{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/universal-natural-language-processing-with","title":"Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start","arxiv_id":"2010.02584","date":"2020-10-06","proceeding":"EMNLP 2020 11","authors":["Wenpeng Yin","Nazneen Fatema Rajani","Dragomir Radev","Richard Socher","Caiming Xiong"],"abstract":"A standard way to address different NLP problems is by first constructing a problem-specific dataset, then building a model to fit this dataset. To build the ultimate artificial intelligence, we desire a single machine that can handle diverse new problems, for which task-specific annotations are limited. We bring up textual entailment as a unified solver for such NLP problems. However, current research of textual entailment has not spilled much ink on the following questions: (i) How well does a pretrained textual entailment system generalize across domains with only a handful of domain-specific examples? and (ii) When is it worth transforming an NLP task into textual entailment? We argue that the transforming is unnecessary if we can obtain rich annotations for this task. Textual entailment really matters particularly when the target NLP task has insufficient annotations. Universal NLP can be probably achieved through different routines. In this work, we introduce Universal Few-shot textual Entailment (UFO-Entail). We demonstrate that this framework enables a pretrained entailment model to work well on new entailment domains in a few-shot setting, and show its effectiveness as a unified solver for several downstream NLP tasks such as question answering and coreference resolution when the end-task annotations are limited. Code: https://github.com/salesforce/UniversalFewShotNLP","url_abs":"https://arxiv.org/abs/2010.02584v1","url_pdf":"https://arxiv.org/pdf/2010.02584v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"universal-natural-language-processing-with","repo_url":"https://github.com/salesforce/UniversalFewShotNLP","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"coreference-resolution","task_name":"Coreference Resolution"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"coreference-resolution-1","task_name":"coreference-resolution"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.02584","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.02584"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salesforce/UniversalFewShotNLP","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f6662f438f626dbd","entry":"build_GAP_output_format","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/full.shot.train.on.target.data.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/full.shot.train.on.target.data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f6662f438f626dbd"}},{"code_sha256_prefix":"84120ed8a77f6efa","entry":"calculate_scores","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/gap_scorer.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/gap_scorer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"84120ed8a77f6efa"}},{"code_sha256_prefix":"b2f6b157b471666d","entry":"calculate_scores","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Non-Entail-Approach/gap_scorer_modified.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Non-Entail-Approach/gap_scorer_modified.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"b2f6b157b471666d"}},{"code_sha256_prefix":"65e3e28505f962b0","entry":"convert_examples_to_features","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/0.shot.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/0.shot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"65e3e28505f962b0"}},{"code_sha256_prefix":"30a9889ed4a6e986","entry":"convert_examples_to_features","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/full.shot.train.on.target.data.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/full.shot.train.on.target.data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"30a9889ed4a6e986"}},{"code_sha256_prefix":"6408f45358c253ca","entry":"examples_to_features","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/full.shot.train.on.target.data.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/full.shot.train.on.target.data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6408f45358c253ca"}},{"code_sha256_prefix":"f706083dc1ace8a0","entry":"load_GAP_coreference_data","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/preprocess.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f706083dc1ace8a0"}},{"code_sha256_prefix":"eb1bd7c8cdf42b2e","entry":"read_annotations_test","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Coreference/gap_scorer_modified.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Coreference/gap_scorer_modified.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"eb1bd7c8cdf42b2e"}},{"code_sha256_prefix":"5df41f5886b2a575","entry":"wordpairID_2_tokenpairID","repo":"salesforce/UniversalFewShotNLP","repo_kind":"official","path":"Non-Entail-Approach/train.coreference.py","file_url":"https://github.com/salesforce/UniversalFewShotNLP/blob/HEAD/Non-Entail-Approach/train.coreference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5df41f5886b2a575"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}