There is a newer version of this record available.

Software Open Access

quanteda/quanteda: CRAN v2.1.0

Kenneth Benoit; Kohei Watanabe; Haiyan Wang; Paul Nulty; Adam Obeng; Stefan Müller; Jiong Wei Lua; Aki Matsuo; Christian Mueller; Will Lowe; Pablo Barberá; Christopher Gandrud; mark padgham; Tyler Rinker; José Tomás Atria; Johannes Gruber; Katrin Leinweber; Michael Chirico; Michael W. Kearney; Stas Malavin; Thomas J. Leeper; hotzeplotz; Chung-hong Chan; etienne-s; hofaichan; lindbrook; nicmer; Tom Paskhalis


  • Added block_size to quanteda_options() to control the number of documents in blocked tokenization.
  • Fixed print.dictionary2() to control the printing of nested levels with max_nkey (#1967)
  • Added textstat_summary() to provide detailed information about dfm, tokens and corpus objects. It will replace summary() in future versions.
  • Fixed a performance issue causing slowdowns in tokenizing (using the default what = "word") corpora with large numbers of documents that contain social media tags and URLs that needed to be preserved (such a large corpus of Tweets).
  • Updated the (default) "word" tokenizer to preserve hashtags and usernames better with non-ASCII text, and made these patterns user-configurable in quanteda_options(). The following are now preserved: "#政治" as well as Weibo-style hashtags such as "#英国首相#".
  • convert(x, to = "data.frame") now outputs the first column as "doc_id" rather than "document" since "document" is a commonly occurring term in many texts. (#1918)
  • Added new methods char_select(), char_keep(), and char_remove() for easy manipulation of character vectors.
  • Added dictionary_edit() for easy, interactive editing of dictionaries, plus the functions char_edit() and list_edit() for editing character and list of character objects.
  • Added a method to textplot_wordcloud() that plots objects from textstat_keyness(), to visualize keywords either by comparison or for the target category only.
  • Improved the performance of kwic() (#1840).
  • Added new logsmooth scheme to dfm_weight().
  • Added new textstat_summary() method, which returns summary information about the tokens/types/features etc in an object. It also caches summary information so that this can be retrieved on subsequent calls, rather than re-computed.
Bug fixes and stability enhancements
  • Stopped returning NA for non-existent features when n > nfeat(x) in textstat_frequency(x, n). (#1929)
  • Fixed a problem in dfm_lookup() and tokens_lookup() in which an error was caused when no dictionary key returned a single match (#1946).
  • Fixed a bug that caused a textstat_simil/dist object converted to a data.frame to drop its document2 labels (#1939).
  • Fixed a bug causing dfm_match() to fail on a dfm that included "pads" (""). (#1960)
  • Updated the data_dfm_lbgexample object using more modern dfm internals.
  • Updates textstat_readability(), textstat_lexdiv(), and nscrabble() so that empty texts are not dropped in the result. (#1976)

Files (38.4 MB)
Name Size
38.4 MB Download
All versions This version
Views 1,54322
Downloads 2101
Data volume 6.1 GB38.4 MB
Unique views 1,44521
Unique downloads 1201


Cite as