Extract noun phrases from texts using spaCy
Source:R/spacy_extract_nounphrases.R
spacy_extract_nounphrases.RdThis function extracts noun phrases from documents, based on the
noun_chunks attributes of documents objects parsed by spaCy (see
https://spacy.io/usage/linguistic-features#noun-chunks).
Usage
spacy_extract_nounphrases(
x,
output = c("data.frame", "list"),
multithread,
n_process = 1L,
batch_size = NULL,
...,
offsets = FALSE
)Arguments
- x
a character object or a TIF-compliant corpus data.frame (see https://github.com/ropenscilabs/tif)
- output
type of returned object, either
"data.frame"or"list"- multithread
deprecated; it no longer has any effect, and gives a warning if supplied. Documents are always processed in batches with spaCy's
nlp.pipe(); usen_processandbatch_sizeinstead.- n_process
the number of processes for spaCy's
nlp.pipe()to use:1(the default) processes the documents in the current R session's Python process, a larger number starts that many Python processes, and-1uses all CPUs. Starting the processes and loading the model in each takes time, so more than one process helps only for large numbers of documents. A custom pipeline component must be defined in an importable Python module to be used in other processes.- batch_size
the number of documents that spaCy's
nlp.pipe()processes together;NULL(the default) uses the pipeline's default (nlp.batch_size). Larger batches can be faster, but use more memory.- ...
unused
- offsets
logical; if
TRUE, also return the position of each noun phrase in its text as it was given inx, in the columnsstartandend: the positions of the noun phrase's first and last characters, counted from 1 as inspacy_parse(), so thatsubstring(text, start, end)gives the noun phrases of a text. This needsoutput = "data.frame". It comes after..., so it has to be given by its full name.
Details
When the option output = "data.frame" is selected, the
function returns a data.frame with the following fields, and with
start and end after them when offsets = TRUE.
root_textcontents of root token
start_idserial number ID of starting token: its position in the document, starting at 1 and counting every token of spaCy's, as the rows of
spacy_parse(x)do. Whitespace tokens are counted, so where a document has them this is not the position among the tokens ofspacy_tokenize(x)with default options, which leaves them out.root_idserial number ID of root token
lengthnumber of words (tokens) included in a noun-phrase (e.g. for a noun-phrase, "individual car owners",
length = 3)
Components that are left out
The components of spaCy's pipeline that noun phrases do not need are left out
of the call, which makes it faster: with spaCy's English
models, the lemmatizer and the named entity recognizer. The output is the same as
when every component runs, so a component stays whenever a later one
that stays may read what it sets, and nothing is left out when the
pipeline has a component that is not one of spaCy's standard ones, or
finds its noun phrases by a function that is not spaCy's own for the
language, which may read any annotation. See
the details of spacy_parse(), and the article "Improving Performance"
on the package website.
Examples
if (FALSE) { # \dontrun{
spacy_initialize()
txt <- c(doc1 = "Natural language processing is a branch of computer science.",
doc2 = "Paul earned a postgraduate degree from MIT.")
spacy_extract_nounphrases(txt)
spacy_extract_nounphrases(txt, output = "list")
} # }