MaartenGr/BERTopic: v0.17.0

Maarten Grootendorst; Heinz-Alexander Fuetterer; Freddy Heppell; zilch42; Anubhab Das; Chris Inskip; Ian Randman; Jakub Ciszek; Joshua Sundance Bailey; azikoss; dschwalm; Ahmed Elashry; Alex Gamble; Anastasia Simonoff; Andrea Francis Soria Jimenez; Anoop Thomas Mathew; Aratako; Bob; Carlos Pegueros; Daniel Kapitan; Daniel van Strien; Danny Huang; David Dai; David DiCato; Domenic Rosati; Elton Liao; Felipe Alves Siqueira; Franz Louis Cesista; Gus Moir; Hao LI

doi:10.5281/zenodo.15052846

Published March 19, 2025 | Version v0.17.0

Software Open

MaartenGr/BERTopic: v0.17.0

1. WZB Berlin Social Science Center
2. @GateNLP
3. Textify AI (@T3xtifyai)
4. Google
5. @swca
6. Hugging Face
7. @Docket Inc.
8. Proton AG
9. @Nubank
10. Freelancer and educator in data science & AI @anthology-of-data-science, Fellow @eaisi
11. CloseFactor
12. @scitedotai @dalhousie
13. ICMC, University of São Paulo
14. Mustang Analytics
15. Queen's University

<h3><b>Highlights:</a></b></h3>

Light-weight installation without UMAP and HDBSCAN by @MaartenGr in #2289
Add Model2Vec as an embedding backend by @MaartenGr in #2245
Add LiteLLM as a representation model by @MaartenGr in #2213
Interactive DataMapPlot by @MaartenGr in #2287

<h3><b>Fixes:</a></b></h3>

Lightweight installation: use safetensors without torch by @hedgeho in #2306
Fix missing links by @MaartenGr in #2305
Set up pre-commit hooks by @afuetterer in #2283
Fix handling OpenAI returning None objects by @jeaninejuliettes in #2280
Add support for python 3.13 by @afuetterer in #2173
Added system prompts by @Leo-LiHao in #2145
More documentation for topic reduction by @MaartenGr in #2260
Drop support for python 3.8 by @afuetterer in #2243
Fixed online topic modeling on GPU by @SSivakumar12 in #2181
Fixed hierarchical cluster visualization by @PipaFlores in #2191
Remove duplicated phrase by @AndreaFrancis in #2197

Model2Vec

With Model2Vec, we now have a very interesting pipeline for light-weight embeddings. Combined with the light-weight installation, you can now run BERTopic without using pytorch!

Installation is straightforward:

pip install --no-deps bertopic
pip install --upgrade numpy pandas scikit-learn tqdm plotly pyyaml

This will install BERTopic even without UMAP or HDBSCAN, so you can use other techniques instead. If these are not installed, then it uses PCA with scikit-learn's HDBSCAN instead. You can install them, together with Model2Vec:

pip install model2vec umap-learn hdbscan

Then, creating a BERTopic model is as straightforward as you are used to:

from bertopic import BERTopic
from model2vec import StaticModel

# Model2Vec
embedding_model = StaticModel.from_pretrained("minishlab/potion-base-8M")

# BERTopic
topic_model = BERTopic(embedding_model=embedding_model)

DataMapPlot

To use the interactive version of DataMapPlot, you only need to run the following:

from umap import UMAP

# Reduce your embeddings to 2-dimensions
reduced_embeddings = UMAP(n_neighbors=10, n_components=2, min_dist=0.0, metric='cosine').fit_transform(embeddings)

# Create an interactive DataMapPlot figure
topic_model.visualize_document_datamap(docs, reduced_embeddings=reduced_embeddings, interactive=True

Files