{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mitigating-data-sparsity-for-short-text-topic","title":"Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning","arxiv_id":"2211.12878","date":"2022-11-23","proceeding":null,"authors":["Xiaobao Wu","Anh Tuan Luu","Xinshuai Dong"],"abstract":"To overcome the data sparsity issue in short text topic modeling, existing methods commonly rely on data augmentation or the data characteristic of short texts to introduce more word co-occurrence information. However, most of them do not make full use of the augmented data or the data characteristic: they insufficiently learn the relations among samples in data, leading to dissimilar topic distributions of semantically similar text pairs. To better address data sparsity, in this paper we propose a novel short text topic modeling framework, Topic-Semantic Contrastive Topic Model (TSCTM). To sufficiently model the relations among samples, we employ a new contrastive learning method with efficient positive and negative sampling strategies based on topic semantics. This contrastive learning method refines the representations, enriches the learning signals, and thus mitigates the sparsity issue. Extensive experimental results show that our TSCTM outperforms state-of-the-art baselines regardless of the data augmentation availability, producing high-quality topics and topic distributions.","url_abs":"https://arxiv.org/abs/2211.12878v1","url_pdf":"https://arxiv.org/pdf/2211.12878v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mitigating-data-sparsity-for-short-text-topic","repo_url":"https://github.com/bobxwu/tsctm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"mitigating-data-sparsity-for-short-text-topic","repo_url":"https://github.com/bobxwu/topmost","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2211.12878","atlas_url":"https://app.syntology.ai/?focus=2211.12878","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.12878"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bobxwu/tsctm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bobxwu/topmost","reach":null}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1},"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"372966abcb972caf","entry":"TSC","repo":"bobxwu/tsctm","repo_kind":"official","path":"models/TSCTM.py","file_url":"https://github.com/bobxwu/tsctm/blob/HEAD/models/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"372966abcb972caf"}},{"code_sha256_prefix":"63e8e671840a4fc3","entry":"TSCTM","repo":"bobxwu/tsctm","repo_kind":"official","path":"models/TSCTM.py","file_url":"https://github.com/bobxwu/tsctm/blob/HEAD/models/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63e8e671840a4fc3"}},{"code_sha256_prefix":"9c19bf351705f1a6","entry":"TSCTM","repo":"bobxwu/topmost","repo_kind":"listed","path":"topmost/models/basic/TSCTM/TSCTM.py","file_url":"https://github.com/bobxwu/topmost/blob/HEAD/topmost/models/basic/TSCTM/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c19bf351705f1a6"}},{"code_sha256_prefix":"9e58a550a72ec7d7","entry":"TopicDistQuant","repo":"bobxwu/tsctm","repo_kind":"official","path":"models/TSCTM.py","file_url":"https://github.com/bobxwu/tsctm/blob/HEAD/models/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e58a550a72ec7d7"}},{"code_sha256_prefix":"b927db1941b8db00","entry":"TopicDistQuant","repo":"bobxwu/topmost","repo_kind":"listed","path":"topmost/models/basic/TSCTM/TSCTM.py","file_url":"https://github.com/bobxwu/topmost/blob/HEAD/topmost/models/basic/TSCTM/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b927db1941b8db00"}},{"code_sha256_prefix":"adaec247f2fff276","entry":"TSC","repo":"bobxwu/topmost","repo_kind":"listed","path":"topmost/models/basic/TSCTM/TSCTM.py","file_url":"https://github.com/bobxwu/topmost/blob/HEAD/topmost/models/basic/TSCTM/TSCTM.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"adaec247f2fff276"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}