From Text to Auditable Annotations
Kenneth Benoit
2026-11-16
Source:vignettes/articles/annotations.Rmd
annotations.RmdIntroduction
Much analysis of text with spaCy ends in a count: how often presidents speak of Congress, or how many passive verbs a manifesto has. A count does not show what was counted. Before relying on it, you want to see each thing that was found, in the text where it was found, and to change the rule that found it when the rule is wrong.
This article does that for one small example: references to the legislature and to the courts in four inaugural addresses of US presidents. It arrives at this, a passage of George Bush’s address of 1989 with its entities highlighted, and every reference to the legislature in that address in its context:
## Keyword-in-context with 5 matches.
##
## [1989-Bush, 1574:1574] the Executive and the | Congress | . The challenges before
## [1989-Bush, 1585:1586] be thrashed out with | the House | and the Senate.
## [1989-Bush, 1589:1589] the House and the | Senate | . We must bring
## [1989-Bush, 1655:1655] voices. \n\n For | Congress | , too, has
## [1989-Bush, 1926:1926] Majority Leader, the | Congress | and the Executive were
Version 2.0 of spacyr adds what this takes:
- rules written in R, as a quanteda dictionary or as
patterns, which
spacy_match()finds in the texts andspacy_initialize()adds to the pipeline as named entities; - the position of every match, entity and token, in spaCy’s tokens
and, with
offsets = TRUE, in the characters of the original text; -
entity_visualize(), which shows the text with its entities highlighted; -
as_index(), which gives matches toquanteda::kwic()for a concordance, and methods that turn a parse into quanteda tokens, with the document variables of the corpus that was parsed.
The article goes through these in turn. For more detail on each step, see the articles on rule-based matching, rule-based named entities and improving performance.
The texts
The texts are four of the inaugural addresses in
quanteda’s data_corpus_inaugural, a corpus
whose document variables record the year, the president and the
party:
library("spacyr")
library("quanteda")
corp <- corpus_subset(data_corpus_inaugural, Year %in% c(1789, 1861, 1933, 1989))
docvars(corp)
## Year President FirstName Party
## 1 1789 Washington George none
## 2 1861 Lincoln Abraham Republican
## 3 1933 Roosevelt Franklin D. Democratic
## 4 1989 Bush George RepublicanDefine what to find
A quanteda dictionary says what to find. Its keys are the labels, and its values are the words and phrases to find under each:
dict <- dictionary(list(LEGISLATURE = c("Congress", "Senate", "House of Representatives", "House"),
JUDICIARY = "Supreme Court"),
tolower = FALSE)spacy_match() finds them in the texts. It returns a
data.frame with one row for each match: the document, the label, the
matched text, and where the match is.
spacy_initialize()
## successfully initialized (spaCy Version: 3.8.18, language model: en_core_web_sm)
m <- spacy_match(corp, dict)
head(m, 3)
## doc_id match_id label node sentence_id token_id text start_id
## 1 1789-Washington 1 LEGISLATURE <NA> 1 6 Senate 6
## 2 1789-Washington 2 LEGISLATURE <NA> 1 10 House of Representatives 10
## 3 1789-Washington 3 LEGISLATURE <NA> 19 19 House of Representatives 1281
## length
## 1 1
## 2 3
## 3 3
table(m$doc_id, m$label)
##
## JUDICIARY LEGISLATURE
## 1789-Washington 0 3
## 1861-Lincoln 2 5
## 1933-Roosevelt 0 5
## 1989-Bush 0 6That is 21 references. Whether they are what we meant is the next question.
Inspect the matches
quanteda::kwic() shows each occurrence of a pattern with
the words around it. It can do the same for positions found elsewhere,
given as an index, and as_index() makes one from a table of
matches. The positions count spaCy’s tokens, so the tokens to give
kwic() are those of spacy_parse() for the same
texts, which as.tokens() converts. Only the tokens are
needed here, so the parse leaves out the lemmas, the parts of speech and
the entities:
toks <- as.tokens(spacy_parse(corp, lemma = FALSE, pos = FALSE, entity = FALSE))
kwic(toks, index = as_index(m), window = 5)
## Keyword-in-context with 21 matches.
##
## [1789-Washington, 6:6] Fellow - Citizens of the | Senate |
## [1789-Washington, 10:12] the Senate and of the | House of Representatives |
## [1789-Washington, 1281:1283] most properly addressed to the | House of Representatives |
## [1861-Lincoln, 608:608] law. All members of | Congress |
## [1861-Lincoln, 906:906] to specify particular acts of | Congress |
## [1861-Lincoln, 2259:2259] not expressly say. May | Congress |
## [1861-Lincoln, 2274:2274] not expressly say. Must | Congress |
## [1861-Lincoln, 2588:2589] to be decided by the | Supreme Court |
## [1861-Lincoln, 2735:2736] fixed by decisions of the | Supreme Court |
## [1861-Lincoln, 3394:3394] not seen - has passed | Congress |
## [1933-Roosevelt, 1169:1169] presently urge upon a new | Congress |
## [1933-Roosevelt, 1205:1205] to putting our own national | house |
## [1933-Roosevelt, 1788:1788] such other measures as the | Congress |
## [1933-Roosevelt, 1820:1820] in the event that the | Congress |
## [1933-Roosevelt, 1862:1862] . I shall ask the | Congress |
## [1989-Bush, 1487:1487] on this in the White | House |
## [1989-Bush, 1574:1574] between the Executive and the | Congress |
## [1989-Bush, 1586:1586] be thrashed out with the | House |
## [1989-Bush, 1589:1589] with the House and the | Senate |
## [1989-Bush, 1655:1655] discordant voices. \n\n For | Congress |
## [1989-Bush, 1926:1926] Mr. Majority Leader, the | Congress |
##
## and of the House of
## : \n\n Among the vicissitudes
## . It concerns myself,
## swear their support to the
## as proper to be enforced
## prohibit slavery in the Territories
## protect slavery in the Territories
## , nor do I deny
## , the instant they are
## , to the effect that
## in special session detailed measures
## in order and making income
## may build out of its
## shall fail to take one
## for the one remaining instrument
## , in the Cabinet agencies
## . The challenges before us
## and the Senate. We
## . We must bring the
## , too, has changed
## and the Executive were capableTwo of the 21 are not references to the legislature: Roosevelt’s “our
own national house”, and the “House” of Bush’s “White House”. Both come
from the value "House", which matches in any case and
wherever the word is, as a dictionary value does in
quanteda. The table of counts could not have shown
this.
Revise the rules
Two changes remove them: the value becomes "the House",
and the values match in their own case.
as_entity_patterns() makes a set of rules from a dictionary
with that setting. Both spacy_match() and
spacy_initialize() take the rules.
dict <- dictionary(list(LEGISLATURE = c("Congress", "Senate", "House of Representatives", "the House"),
JUDICIARY = "Supreme Court"),
tolower = FALSE)
rules <- as_entity_patterns(dict, case_insensitive = FALSE)
rules
## Entity patterns: 5 patterns (LEGISLATURE: 4, JUDICIARY: 1)
## LEGISLATURE "Congress"
## LEGISLATURE "Senate"
## LEGISLATURE "House of Representatives"
## LEGISLATURE "the House"
## JUDICIARY "Supreme Court"
m <- spacy_match(corp, rules, offsets = TRUE)
kwic(toks, index = as_index(m[grepl("House", m$text), ]), window = 5)
## Keyword-in-context with 3 matches.
##
## [1789-Washington, 10:12] the Senate and of the | House of Representatives |
## [1789-Washington, 1281:1283] most properly addressed to the | House of Representatives |
## [1989-Bush, 1585:1586] will be thrashed out with | the House |
##
## : \n\n Among the vicissitudes
## . It concerns myself,
## and the Senate. We
table(m$doc_id, m$label)
##
## JUDICIARY LEGISLATURE
## 1789-Washington 0 3
## 1861-Lincoln 2 5
## 1933-Roosevelt 0 4
## 1989-Bush 0 5The two false matches are gone, and the House that Bush meant is still found. A rule has its limits too: this one would miss a sentence that began with “The House”, and none of these addresses has one.
Matching does not change the pipeline, so rules can be tried and
revised like this without starting spaCy again. Rules need not be words
and phrases. spacy_match() also takes patterns that
describe tokens by their attributes, such as their part of speech, and
patterns that describe tokens connected in the dependency parse; the
article on rule-based matching uses them to
find passive verbs.
Back to the original text
With offsets = TRUE, each match also has
start and end, the positions of its first and
last characters in the document. They count characters from 1, as R
does, so substring() recovers the match from the original
text, or the match with as much of the text around it as you want:
txt <- as.character(corp)
all(substring(txt[m$doc_id], m$start, m$end) == m$text)
## [1] TRUE
court <- m[m$label == "JUDICIARY", ]
data.frame(doc_id = court$doc_id,
context = substring(txt[court$doc_id], court$start - 30, court$end + 30))
## doc_id context
## 1 1861-Lincoln ions are to be decided by the Supreme Court, nor do I deny that such deci
## 2 1861-Lincoln bly fixed by decisions of the Supreme Court, the instant they are made inUnlike the positions of tokens, these do not depend on how the text was split into tokens. They can mark the matches in the documents themselves, or send them to another program or to a human coder.
Rules as named entities
Rules that find what they should can become part of the pipeline.
spacy_initialize(entity_patterns = ) adds them to it as
spaCy’s entity ruler, and what they find is then among the named
entities of every parse. spaCy is running without the rules, so it is
finalized first:
Then it is initialized with the rules, and the texts are parsed, with the offsets of their tokens:
spacy_initialize(entity_patterns = rules)
## successfully initialized (spaCy Version: 3.8.18, language model: en_core_web_sm)
parsed <- spacy_parse(corp, offsets = TRUE)The entity column of the parse has the labels of the
rules next to those of the language model. These are the first two
sentences in which Bush refers to the legislature, and the last tokens
of the second:
bush <- parsed[parsed$doc_id == "1989-Bush", ]
sentences <- unique(bush$sentence_id[grepl("^LEGISLATURE", bush$entity)])
passage <- bush[bush$sentence_id %in% sentences[1:2], ]
tail(passage, 7)
## doc_id sentence_id token_id token start end lemma pos entity
## 9287 1989-Bush 78 9 with 7315 7318 with ADP
## 9288 1989-Bush 78 10 the 7320 7322 the DET LEGISLATURE_B
## 9289 1989-Bush 78 11 House 7324 7328 House PROPN LEGISLATURE_I
## 9290 1989-Bush 78 12 and 7330 7332 and CCONJ
## 9291 1989-Bush 78 13 the 7334 7336 the DET
## 9292 1989-Bush 78 14 Senate 7338 7343 Senate PROPN LEGISLATURE_B
## 9293 1989-Bush 78 15 . 7344 7344 . PUNCToffsets = TRUE works for spacy_parse() as
it does for spacy_match(), and gives the position of each
token in the text. Counting the entities of each type gives the model’s
entities and ours, and the same 17 and 2 that spacy_match()
found:
table(entity_extract(parsed)$entity_type)
##
## EVENT FAC GPE JUDICIARY LAW LEGISLATURE LOC NORP
## 2 1 27 2 23 17 9 12
## ORG PERSON WORK_OF_ART
## 34 11 3Matches and entities serve different purposes.
spacy_match() leaves the pipeline as it is and returns a
table. Its matches can overlap when their labels differ, and it takes
patterns for things that are not entities. Entities are part of the
parse. No two of them overlap, and where a rule’s entity overlaps one of
the model’s, the rule’s is kept, unless
overwrite_entities = FALSE. Every function that works with
entities then has them: entity_extract(),
entity_consolidate(), spacy_extract_entity()
and entity_visualize().
Highlight the entities
entity_visualize() shows parsed documents with their
entities highlighted and labelled, as HTML. Because the parse has the
offsets of its tokens, the text is spaced as the original was. With
view = TRUE the result opens in the browser. In an R
Markdown document, knitr::raw_html() puts it into the
page:
knitr::raw_html(entity_visualize(passage, colors = c(LEGISLATURE = "#ffd966")))Labels of the model have colours of their own, and
colors sets one for a label of ours, which would otherwise
be grey. The passage also shows what the model does by itself: it takes
“the Executive” for a location. To show only some labels, name them, as
in entity_visualize(parsed, entities = names(dict)).
Back in quanteda
as.tokens() turns the parse into
quanteda tokens, and restores the document variables of
the corpus that was parsed. The counts can then be set beside the year
and the party of each address, and the tokens go on to any analysis in
quanteda:
toks <- as.tokens(parsed)
counts <- table(factor(m$doc_id, levels = docnames(toks)), m$label)
cbind(docvars(toks), unclass(counts))
## Year President FirstName Party JUDICIARY LEGISLATURE
## 1789-Washington 1789 Washington George none 0 3
## 1861-Lincoln 1861 Lincoln Abraham Republican 2 5
## 1933-Roosevelt 1933 Roosevelt Franklin D. Democratic 0 4
## 1989-Bush 1989 Bush George Republican 0 5The levels of the factor are the names of all the documents, so that a document without a match would stay in the table, with zeros.
Larger corpora
For a larger corpus, it matters that the functions run no more of
spaCy’s pipeline than they need. spacy_match() runs only
the components that its patterns ask for, and words and phrases need
only the parser. spacy_parse() leaves out the components of
the columns that are not asked for, as the parse for the concordance did
above. Both take batch_size and n_process, for
the number of documents that spaCy handles at a time and the number of
processes it uses. Whether more processes are faster depends on the
number and the length of the documents, so time them on a sample first.
The article on improving performance has
the measurements.
Upgrading from 1.4.0
Most code written for spacyr 1.4.0 runs unchanged in
2.0. What can need attention is listed under “Upgrading from 1.4.0” in
the news: the versions of R and spaCy
that are needed, n_process and batch_size in
place of multithread, and the functions that are deprecated
or have a new name.