{"url":"/task/topic-coverage","name":"Topic coverage","slug":"topic-coverage","description_markdown":"A prevalent use case of topic models is that of topic discovery.\r\nHowever, most of the topic model evaluation methods rely on abstract metrics such as perplexity or topic coherence. The topic coverage approach is to measure the models' performance by matching model-generated topics to a fixed set of reference topics - topics discovered by humans and represented in a machine-readable format. This way, the models are evaluated in the context of their use, by essentially simulating topic modeling in a fixed setting defined by a text collection and a set of reference topics.\r\nReference topics represent a ground truth that can be used to evaluate both topic models and other measures of model performance. This coverage approach enables large-scale automatic evaluation of existing and future topic models.","categories":[{"name":"Natural Language Processing","url":"/area/natural-language-processing"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":18,"papers_with_code":6,"benchmarks":3,"benchmark_tables_in_archive":3,"benchmark_tables_shown":3,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":1,"subtasks":0,"parent_tasks":1},"benchmarks":[{"leaderboard":"/sota/topic-coverage-on-topic-modeling-topic","slug":"topic-coverage-on-topic-modeling-topic","dataset":"Topic modeling topic coverage dataset - news","dataset_url":null,"rows_in_archive":2,"metrics":["AuCDC","SupCov"],"first_row_in_archive_order":{"model":"PYP","paper_title":"A Topic Coverage Approach to Evaluation of Topic Models","paper_url":"/paper/a-topic-coverage-approach-to-evaluation-of","paper_date":"2020-12-11","arxiv_id":"2012.06274","code_links":[{"title":"dkorenci/topic_coverage","url":"https://github.com/dkorenci/topic_coverage"}],"syntology":null}},{"leaderboard":"/sota/topic-coverage-on-topic-modeling-topic-1","slug":"topic-coverage-on-topic-modeling-topic-1","dataset":"Topic modeling topic coverage dataset - bio","dataset_url":null,"rows_in_archive":2,"metrics":["AuCDC","SupCov"],"first_row_in_archive_order":{"model":"NMF-200","paper_title":"A Topic Coverage Approach to Evaluation of Topic Models","paper_url":"/paper/a-topic-coverage-approach-to-evaluation-of","paper_date":"2020-12-11","arxiv_id":"2012.06274","code_links":[{"title":"dkorenci/topic_coverage","url":"https://github.com/dkorenci/topic_coverage"}],"syntology":null}},{"leaderboard":"/sota/topic-coverage-on-topic-modeling-topic-2","slug":"topic-coverage-on-topic-modeling-topic-2","dataset":"Topic modeling topic coverage dataset","dataset_url":"/dataset/topic-modeling-topic-coverage-dataset","rows_in_archive":1,"metrics":["Spearman Correlation"],"first_row_in_archive_order":{"model":"AuCDC","paper_title":"A Topic Coverage Approach to Evaluation of Topic Models","paper_url":"/paper/a-topic-coverage-approach-to-evaluation-of","paper_date":"2020-12-11","arxiv_id":"2012.06274","code_links":[{"title":"dkorenci/topic_coverage","url":"https://github.com/dkorenci/topic_coverage"}],"syntology":null}}],"datasets":[{"url":"/dataset/topic-modeling-topic-coverage-dataset","name":"Topic modeling topic coverage dataset","full_name":"","num_papers_in_archive":1}],"subtasks":[],"parent_tasks":[{"url":"/task/topic-models","name":"Topic Models"}],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":6,"of":6,"tagged_in_all":18,"items":[{"url":"/paper/neural-topic-modeling-with-large-language","title":"Neural Topic Modeling with Large Language Models in the Loop","date":"2024-11-13","arxiv_id":"2411.08534","repositories_listed":1,"syntology":null},{"url":"/paper/regen-zero-shot-text-classification-via","title":"ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval","date":"2023-05-18","arxiv_id":"2305.10703","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/mug-a-general-meeting-understanding-and","title":"MUG: A General Meeting Understanding and Generation Benchmark","date":"2023-03-24","arxiv_id":"2303.13939","repositories_listed":1,"syntology":null},{"url":"/paper/augesc-large-scale-data-augmentation-for","title":"AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation","date":"2022-02-26","arxiv_id":"2202.13047","repositories_listed":1,"syntology":null},{"url":"/paper/unsupervised-summarization-for-chat-logs-with","title":"Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders","date":"2020-12-14","arxiv_id":"2012.07300","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/a-topic-coverage-approach-to-evaluation-of","title":"A Topic Coverage Approach to Evaluation of Topic Models","date":"2020-12-11","arxiv_id":"2012.06274","repositories_listed":1,"syntology":null}],"syntology_records":2,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}