Extract named entities from texts using spaCy
Source:R/spacy_extract_entity.R
spacy_extract_entity.RdThis function extracts named entities from texts, based on the entity tag
ent attributes of documents objects parsed by spaCy (see
https://spacy.io/usage/linguistic-features#named-entities).
Arguments
- x
a character object or a TIF-compliant corpus data.frame (see https://github.com/ropenscilabs/tif)
- output
type of returned object, either
"list"or"data.frame".- type
type of named entities, either
named,extended, orall. Extended entities are those of typesDATE,TIME,PERCENT,MONEY,QUANTITY,ORDINALandCARDINAL, and named entities are those of all other types. The entity types are those of the language model; see its "Label scheme" on https://spacy.io/models.- multithread
deprecated; it no longer has any effect, and gives a warning if supplied. Documents are always processed in batches with spaCy's
nlp.pipe(); usen_processandbatch_sizeinstead.- n_process
the number of processes for spaCy's
nlp.pipe()to use:1(the default) processes the documents in the current R session's Python process, a larger number starts that many Python processes, and-1uses all CPUs. Starting the processes and loading the model in each takes time, so more than one process helps only for large numbers of documents. A custom pipeline component must be defined in an importable Python module to be used in other processes.- batch_size
the number of documents that spaCy's
nlp.pipe()processes together;NULL(the default) uses the pipeline's default (nlp.batch_size). Larger batches can be faster, but use more memory.- ...
unused
- offsets
logical; if
TRUE, also return the position of each entity in its text as it was given inx, in the columnsstartandend: the positions of the entity's first and last characters, counted from 1 as inspacy_parse(), so thatsubstring(text, start, end)gives the entities of a text. This needsoutput = "data.frame". It comes after..., so it has to be given by its full name.
Details
When the option output = "data.frame" is selected, the
function returns a data.frame with the following fields, and with
start and end after them when offsets = TRUE.
ent_typetype of entity (e.g.
ORGfor organizations)start_idserial number ID of starting token: its position in the document, starting at 1 and counting every token of spaCy's, as the rows of
spacy_parse(x)do. Whitespace tokens are counted, so where a document has them this is not the position among the tokens ofspacy_tokenize(x)with default options, which leaves them out.lengthnumber of words (tokens) included in a named entity (e.g. for an entity, "New York Stock Exchange"",
length = 4)
Components that are left out
The components of spaCy's pipeline that entities do not need are left out
of the call, which makes it faster: with spaCy's English models, the
tagger, the attribute ruler and the lemmatizer. The output is the same as
when every component runs, so a component stays whenever a later one
that stays may read what it sets, and nothing is left out when the
pipeline has a component that is not one of spaCy's standard ones. See
the details of spacy_parse(), and the article "Improving Performance"
on the package website.
Examples
if (FALSE) { # \dontrun{
spacy_initialize()
txt <- c(doc1 = "The Supreme Court is located in Washington D.C.",
doc2 = "Paul earned a postgraduate degree from MIT.")
spacy_extract_entity(txt)
spacy_extract_entity(txt, output = "list")
} # }