Home › Code › preprocess_text

preprocess_text

Syntologyentry name in harvested coderead from the graph 2026-09-24

preprocess_text appears in the code Syntology harvested for 26 papers, as 26 distinct code bodies found in 29 places (a place is one code body under one paper). At least one of them ran in 15 of the papers; 12 of the code bodies carry a behaviour fingerprint.

What this page is not. Routines are grouped here by the exact string of their function or class name. Nothing asserts that two samples named preprocess_text do the same thing, share code, or are comparable; the name is a string, not an identity. Behaviour outputs (what a fingerprinted sample returned on the shared battery) are not in this export and are not shown here; the graph at syntology.ai holds them. "Ran" means executed on a synthesized fixture, not that the code is correct or reproduces a paper.

Samples Syntology

Syntology ran 14 of the 26 distinct code bodies named preprocess_text; 12 are unverified. One tile per status, in the site's fixed vocabulary, each code body counted once:

0ran · honoured contract
0ran · violated contract
6ran · our draft was wrong
0ran · fixture could not drive it
8ran
12unverified
12fingerprinted

Licence is a property of each copy, so it is counted per place: 17 of the 29 places are pointer only (Syntology does not serve that copy's text). This site shows no code text for any sample; every row below links to the file in its repository where the record names one.

“Ran” means the sample executed on a synthesized input; it does not mean the output is correct. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code, and those samples did run. The ran count above is every status except unverified, the same rule as each paper page.

Papers

26 papers shown of 26, newest first; 29 places in the table. A paper with no recorded date is placed by the month its arXiv id encodes, shown in the Date column as YYYY-MM (from id). One row per place: a paper whose repository defines the name more than once appears more than once, and the same code body held for several papers appears once under each, with the same status. Titles and dates are the archive's archive 2025-07-28 for papers in the archive, and the graph's for 6 papers added by Syntology. Status and fingerprint are Syntology's record of each code body; licence is recorded for each place. The File cell ends with the code body's code_sha256, Syntology's identity for that exact code: an agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

PaperDateFileStatus SyntologyLicence
Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions added by Syntology 2026-04 (from id) zhangwenhao6/PERA/dataset/dataset_helper.py f343f672c3ddd483 unverified licence not identified · pointer only
Analyzing Cancer Patients' Experiences with Embedding-based Topic Modeling and LLMs added by Syntology 2026-01 (from id) 4dpicture/TM4health/BERT-topic_main.py 5be1e77cf9455131 unverified no licence file found · pointer only
Analyzing Cancer Patients' Experiences with Embedding-based Topic Modeling and LLMs added by Syntology 2026-01 (from id) 4dpicture/TM4health/Top2Vec_main.py 31c4c5deccef16a5 unverified no licence file found · pointer only
MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching added by Syntology 2026-01 (from id) quchangle1/MatchTIR/Code/verl/utils/reward_score/tool.py 11664393d2a5d440 ran · our draft was wrong fingerprinted no licence file found · pointer only
Can SAEs reveal and mitigate racial biases of LLMs in healthcare? added by Syntology 2025-11 (from id) hibaahsan/sae_bias/src/steer_bhcs.py cdfacc9dd53d6bcd ran · our draft was wrong no licence file found · pointer only
Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents added by Syntology 2025-10 (from id) GuoqingWang1/IGPO/evaluate/cacluate_metrics.py 567311eaeb3eeea6 unverified Apache-2.0 (permissive)
Building Data-Driven Occupation Taxonomies: A Bottom-Up Multi-Stage Approach via Semantic Clustering and Multi-Agent Collaboration added by Syntology 2025-09 (from id) aida-ugent/CLIMB/src/get_embeddings.py daf229f8df309a33 unverified licence not identified · pointer only
MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotations 18 Oct 2024 dair-iitd/MediTOD/metrics/nlg_metrics.py a6ac4d0ef910ae31 ran fingerprinted no licence file found · pointer only
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue 12 Sep 2024 davidjurgens/human-llm-similarity/src/analysis/constituency_parse.py c3e35dbeaa8a7e0c ran fingerprinted no licence file found · pointer only
Selective Prompting Tuning for Personalized Conversations with LLMs 26 Jun 2024 hqsiswiliam/SPT/dataset/dataset_helper.py f343f672c3ddd483 unverified no licence file found · pointer only
MAD: Multi-Alignment MEG-to-Text Decoding 3 Jun 2024 neuspeech/mad-meg2text/evaluation.py 2908e3c7f6faafb0 ran fingerprinted Apache-2.0 (permissive)
What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights 31 May 2024 cvmi-lab/clip-beyond-tail/concept_freq_utils/calc_class_frequency.py 18545f8c98643da8 unverified MIT (permissive)
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series 29 May 2024 multimodal-art-projection/map-neo/Matrix/get_high_quality_data/consturct_dataset.py cafca988e5b20807 unverified no licence file found · pointer only
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems 24 May 2024 dair-iitd/SyncTOD/src/prompting/prompt_gen.py a2572769e9a0fc2d ran · our draft was wrong fingerprinted no licence file found · pointer only
On the Content Bias in Fréchet Video Distance 18 Apr 2024 songweige/tats/tats/coinrun/coinrun_data.py 24674c65e5c3df68 unverified MIT (permissive)
Continuous Language Model Interpolation for Dynamic and Controllable Text Generation 10 Apr 2024 skangasl/continuous-lm-interpolation/finetuning/finetune_classifier.py 756ad99fad4efd57 ran fingerprinted no licence file found · pointer only
InstanceDiffusion: Instance-level Control for Image Generation 5 Feb 2024 frank-xwang/InstanceDiffusion/dataset-generation/submitit_automatic_label_ram_save_json.py cc40c350f2fdbdf0 ran Apache-2.0 (permissive)
SMUTF: Schema Matching Using Generative Tags and Hybrid Features 22 Jan 2024 fireindark707/python-schema-matching/relation_features.py c30d94617013b74c ran fingerprinted Apache-2.0 (permissive)
Instructed Language Models with Retrievers Are Powerful Entity Linkers 6 Nov 2023 mrzilinxiao/insgenentitylinking/data_scripts/wiki_dump.py 8ea3c73453786f73 ran fingerprinted no licence file found · pointer only
Advancing Test-Time Adaptation in Wild Acoustic Test Settings 14 Oct 2023 Waffle-Liu/CEA/corpus/l2arctic.py e6c3900dd3f5b5a3 ran fingerprinted MIT (permissive)
SGEM: Test-Time Adaptation for Automatic Speech Recognition via Sequential-Level Generalized Entropy Minimization 3 Jun 2023 drumpt/sgem/corpus/commonvoice.py e6c3900dd3f5b5a3 ran fingerprinted MIT (permissive)
SGEM: Test-Time Adaptation for Automatic Speech Recognition via Sequential-Level Generalized Entropy Minimization 3 Jun 2023 drumpt/sgem/corpus/preprocess_ted.py 3f1c87031de7b002 unverified MIT (permissive)
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization 25 May 2021 cliang1453/super-structured-lottery-tickets/data_utils/xlnet_utils.py b19697c998adb1ea unverified MIT recorded; this copy not marked cleared · pointer only
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media 26 Jul 2020 alisafaya/OffensEval2020/bert-cnn.py c86227bb02c3af92 ran · our draft was wrong fingerprinted no licence file found · pointer only
Revisiting Pre-Trained Models for Chinese Natural Language Processing 29 Apr 2020 ymcui/Chinese-PreTrained-XLNet/src/prepro_utils.py b19697c998adb1ea unverified Apache-2.0 (permissive)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations 26 Sep 2019 kpe/bert-for-tf2/bert/tokenization/albert_tokenization.py 20e27f52f8beba5e unverified MIT (permissive)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations 26 Sep 2019 lonePatient/albert_pytorch/model/tokenization_albert.py 53ab86e6b6c686ae unverified Apache-2.0 (permissive)
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 11 Oct 2018 JA-Bar/nlp-depression/src/glove_fastext_bert/prediction_suicidal.py 68e228e1a25b69ed ran · our draft was wrong fingerprinted no licence file found · pointer only
Achieving Human Parity on Automatic Chinese to English News Translation 15 Mar 2018 sanxing-chen/NMT2017-ZH-EN/preprocess.py 3ca4d1464b186e3d ran · our draft was wrong fingerprinted no licence file found · pointer only

This site shows no code text; each File cell links to the file on GitHub at the repository's current default branch, which may have changed since the harvest. "Pointer only" means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence cell for the reason. Per-sample records for a paper are on its paper page under "Code Syntology ran".

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections