Skip to contents

This function extracts named entities from texts, based on the entity tag ent attributes of documents objects parsed by spaCy (see https://spacy.io/usage/linguistic-features#named-entities).

Usage

spacy_extract_entity(
  x,
  output = c("data.frame", "list"),
  type = c("all", "named", "extended"),
  multithread,
  n_process = 1L,
  batch_size = NULL,
  ...,
  offsets = FALSE
)

Arguments

x

a character object or a TIF-compliant corpus data.frame (see https://github.com/ropenscilabs/tif)

output

type of returned object, either "list" or "data.frame".

type

type of named entities, either named, extended, or all. Extended entities are those of types DATE, TIME, PERCENT, MONEY, QUANTITY, ORDINAL and CARDINAL, and named entities are those of all other types. The entity types are those of the language model; see its "Label scheme" on https://spacy.io/models.

multithread

deprecated; it no longer has any effect, and gives a warning if supplied. Documents are always processed in batches with spaCy's nlp.pipe(); use n_process and batch_size instead.

n_process

the number of processes for spaCy's nlp.pipe() to use: 1 (the default) processes the documents in the current R session's Python process, a larger number starts that many Python processes, and -1 uses all CPUs. Starting the processes and loading the model in each takes time, so more than one process helps only for large numbers of documents. A custom pipeline component must be defined in an importable Python module to be used in other processes.

batch_size

the number of documents that spaCy's nlp.pipe() processes together; NULL (the default) uses the pipeline's default (nlp.batch_size). Larger batches can be faster, but use more memory.

...

unused

offsets

logical; if TRUE, also return the position of each entity in its text as it was given in x, in the columns start and end: the positions of the entity's first and last characters, counted from 1 as in spacy_parse(), so that substring(text, start, end) gives the entities of a text. This needs output = "data.frame". It comes after ..., so it has to be given by its full name.

Value

either a list or data.frame of tokens

Details

When the option output = "data.frame" is selected, the function returns a data.frame with the following fields, and with start and end after them when offsets = TRUE.

ent_type

type of entity (e.g. ORG for organizations)

start_id

serial number ID of starting token: its position in the document, starting at 1 and counting every token of spaCy's, as the rows of spacy_parse(x) do. Whitespace tokens are counted, so where a document has them this is not the position among the tokens of spacy_tokenize(x) with default options, which leaves them out.

length

number of words (tokens) included in a named entity (e.g. for an entity, "New York Stock Exchange"", length = 4)

Components that are left out

The components of spaCy's pipeline that entities do not need are left out of the call, which makes it faster: with spaCy's English models, the tagger, the attribute ruler and the lemmatizer. The output is the same as when every component runs, so a component stays whenever a later one that stays may read what it sets, and nothing is left out when the pipeline has a component that is not one of spaCy's standard ones. See the details of spacy_parse(), and the article "Improving Performance" on the package website.

Examples

if (FALSE) { # \dontrun{
spacy_initialize()

txt <- c(doc1 = "The Supreme Court is located in Washington D.C.",
         doc2 = "Paul earned a postgraduate degree from MIT.")
spacy_extract_entity(txt)
spacy_extract_entity(txt, output = "list")
} # }