Skip to content

ThematOS

pypi-image license-image

LDA topic modeling on tomotopy. ThematOS is a plain Python library; it also ships HyFI 2.0 config groups and workflows, so a training run, a parameter sweep, or an inference run can be started from the command line and recorded in a run manifest. The name comes from the Greek θέματος, "of the theme".

ThematOS 1.0 drops the HyFI 1.x API. Pin thematos<1 to keep it.

Install

uv add thematos          # or: pip install thematos
pip install pyLDAvis     # optional: LDAvis HTML next to the other artifacts

Python 3.11 or newer. Stopwords come from lexikanon, which pulls in MeCab through fugashi; on Python 3.15 fugashi builds from source and needs the MeCab library (apt-get install libmecab-dev or brew install mecab).

Use it as a library

from thematos.datasets import Corpus
from thematos.models import LdaModel

corpus = Corpus(data_file="news.parquet", text_col="tokens")  # or Corpus(data=dataframe)
model = LdaModel(output_dir="workspace/topic/model", seed=42, model_args={"k": 10})
model.train(corpus)  # trains and computes coherence; writes nothing
model.save()  # model file, log-likelihood trace, distributions, top words
model.summarize()  # tomotopy summary and one JSON line in model-summary.jsonl
model.plot()  # log-likelihood plot and topic word clouds
model.model.perplexity

Every file name starts with the model id, LDA_model_k(10) here. Importing thematos never imports HyFI.

Use it from HyFI

Installing thematos registers its config groups through the hyfi.plugins entry point:

group options
model lda
dataset topic_corpus (stopwords from lexikanon's stopwords group)
words wordprior
plot wordcloud
workflow thematos_lda, thematos_lda_sweep, thematos_lda_infer
hyfi run workflow=thematos_lda dataset.data_file=news.parquet dataset.text_col=tokens \
  model.model_args.k=20 model.seed=42

Artifacts go under the project workspace (workspace/topic/model/ by default) and the run manifest under .hyfi/runs/<run id>/manifest.json. See Usage for the sweep and inference workflows.

Citation

See CITATION.cff, or use GitHub's "Cite this repository" button.

License

MIT