{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rationales-for-sequential-predictions","title":"Rationales for Sequential Predictions","arxiv_id":"2109.06387","date":"2021-09-14","proceeding":"EMNLP 2021 11","authors":["Keyon Vafa","Yuntian Deng","David M. Blei","Alexander M. Rush"],"abstract":"Sequence models are a critical component of modern NLP systems, but their predictions are difficult to explain. We consider model explanations though rationales, subsets of context that can explain individual model predictions. We find sequential rationales by solving a combinatorial optimization: the best rationale is the smallest subset of input tokens that would predict the same output as the full sequence. Enumerating all subsets is intractable, so we propose an efficient greedy algorithm to approximate this objective. The algorithm, which is called greedy rationalization, applies to any model. For this approach to be effective, the model should form compatible conditional distributions when making predictions on incomplete subsets of the context. This condition can be enforced with a short fine-tuning step. We study greedy rationalization on language modeling and machine translation. Compared to existing baselines, greedy rationalization is best at optimizing the combinatorial objective and provides the most faithful rationales. On a new dataset of annotated sequential rationales, greedy rationales are most similar to human rationales.","url_abs":"https://arxiv.org/abs/2109.06387v2","url_pdf":"https://arxiv.org/pdf/2109.06387v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rationales-for-sequential-predictions","repo_url":"https://github.com/keyonvafa/sequential-rationales","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"rationales-for-sequential-predictions","repo_url":"https://github.com/keyonvafa/career-code","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.06387","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.06387"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keyonvafa/sequential-rationales","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keyonvafa/career-code","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1},"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6e11afaf3878d5ad","entry":"decode_sequence","repo":"keyonvafa/career-code","repo_kind":"listed","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/career-code/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e11afaf3878d5ad"}},{"code_sha256_prefix":"81307176d91be885","entry":"decode_sequence","repo":"keyonvafa/sequential-rationales","repo_kind":"official","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/sequential-rationales/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81307176d91be885"}},{"code_sha256_prefix":"67860fa68ffc1866","entry":"decode_single_word","repo":"keyonvafa/career-code","repo_kind":"listed","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/career-code/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67860fa68ffc1866"}},{"code_sha256_prefix":"f4b35aa2c181e0e7","entry":"decode_single_word","repo":"keyonvafa/sequential-rationales","repo_kind":"official","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/sequential-rationales/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4b35aa2c181e0e7"}},{"code_sha256_prefix":"f49763cc0d941343","entry":"rationalize_conditional_model","repo":"keyonvafa/sequential-rationales","repo_kind":"official","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/sequential-rationales/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f49763cc0d941343"}},{"code_sha256_prefix":"63536eed85fd01b3","entry":"rationalize_occupation_model","repo":"keyonvafa/career-code","repo_kind":"listed","path":"fairseq/rationalization.py","file_url":"https://github.com/keyonvafa/career-code/blob/HEAD/fairseq/rationalization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63536eed85fd01b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}