{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gistembed-guided-in-sample-selection-of","title":"GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning","arxiv_id":"2402.16829","date":"2024-02-26","proceeding":null,"authors":["Aivin V. Solatorio"],"abstract":"Embedding models are integral to AI applications like semantic search, personalized recommendations, and retrieval augmented generation for LLMs, necessitating high-quality training data. However, the limited scalability of manual data curation prompts the need for automated methods to ensure data integrity. Traditional unsupervised triplet mining automates training data generation, crucial for embedding model training, yet inadvertently injects biases and noise, thereby degrading model performance. Addressing this, we introduce GISTEmbed, a novel strategy that enhances in-batch negative selection during contrastive training through a guide model. This approach departs from reliance on random sampling and equal utility assumption of batch negatives, significantly reducing noise from data quality issues and improving model fine-tuning. Benchmarked against the Massive Text Embedding Benchmark (MTEB), GISTEmbed showcases consistent performance improvements across various model sizes and achieves state-of-the-art results in select categories. This framework enables significant enhancements for smaller models by leveraging the capabilities of powerful yet resource-intensive large models. GISTEmbed can potentially revolutionize the creation of highly efficient, smaller models, democratizing access to advanced AI technologies. Making these technologies more accessible and cost-effective, especially for applications constrained by resources, significantly expands the impact and accessibility of state-of-the-art AI solutions across diverse sectors.","url_abs":"https://arxiv.org/abs/2402.16829v1","url_pdf":"https://arxiv.org/pdf/2402.16829v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gistembed-guided-in-sample-selection-of","repo_url":"https://github.com/avsolatorio/gistembed","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"},{"task_slug":null,"task_name":"Triplet"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16829","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16829"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/avsolatorio/gistembed","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"bb58ccd905346a8b","entry":"compute_pair","repo":"avsolatorio/gistembed","repo_kind":"official","path":"evaluation/mteb_eval.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/evaluation/mteb_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bb58ccd905346a8b"}},{"code_sha256_prefix":"0c8b53a0d94dfcd7","entry":"compute_sts","repo":"avsolatorio/gistembed","repo_kind":"official","path":"evaluation/mteb_eval.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/evaluation/mteb_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c8b53a0d94dfcd7"}},{"code_sha256_prefix":"7f91a871ff0d03ed","entry":"get_embeddings","repo":"avsolatorio/gistembed","repo_kind":"official","path":"gist_embed/trainer/loss.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/gist_embed/trainer/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f91a871ff0d03ed"}},{"code_sha256_prefix":"830f213b5c3ae41c","entry":"get_guide_embeddings","repo":"avsolatorio/gistembed","repo_kind":"official","path":"gist_embed/trainer/loss.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/gist_embed/trainer/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"830f213b5c3ae41c"}},{"code_sha256_prefix":"684784c451b19b7c","entry":"get_output_folder","repo":"avsolatorio/gistembed","repo_kind":"official","path":"evaluation/mteb_eval.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/evaluation/mteb_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"684784c451b19b7c"}},{"code_sha256_prefix":"fc0516e02c0038d1","entry":"has_length","repo":"avsolatorio/gistembed","repo_kind":"official","path":"gist_embed/trainer/trainer.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/gist_embed/trainer/trainer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc0516e02c0038d1"}},{"code_sha256_prefix":"55c7889c25e9257c","entry":"remove_diagonal","repo":"avsolatorio/gistembed","repo_kind":"official","path":"gist_embed/trainer/loss.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/gist_embed/trainer/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55c7889c25e9257c"}},{"code_sha256_prefix":"26cf0c99b5908318","entry":"background_hf_model_upload","repo":"avsolatorio/gistembed","repo_kind":"official","path":"gist_embed/util.py","file_url":"https://github.com/avsolatorio/gistembed/blob/HEAD/gist_embed/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26cf0c99b5908318"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}