{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dont-add-dont-miss-effective-content","title":"Dont Add, dont Miss: Effective Content Preserving Generation from Pre-Selected Text Spans","arxiv_id":"2310.09017","date":"2023-10-13","proceeding":null,"authors":["Aviv Slobodkin","Avi Caciularu","Eran Hirsch","Ido Dagan"],"abstract":"The recently introduced Controlled Text Reduction (CTR) task isolates the text generation step within typical summarization-style tasks. It does so by challenging models to generate coherent text conforming to pre-selected content within the input text (``highlights''). This framing enables increased modularity in summarization-like tasks, allowing to couple a single CTR model with various content-selection setups and modules. However, there are currently no reliable CTR models, while the performance of the existing baseline for the task is mediocre, falling short of practical utility. Here, we address this gap by introducing a high-quality, open-source CTR model that tackles two prior key limitations: inadequate enforcement of the content-preservation constraint, and suboptimal silver training data. Addressing these, we amplify the content-preservation constraint in both training, via RL, and inference, via a controlled decoding strategy. Further, we substantially improve the silver training data quality via GPT-4 distillation. Overall, pairing the distilled dataset with the highlight-adherence strategies yields marked gains over the current baseline, of up to 30 ROUGE-L points, providing a reliable CTR model for downstream use.","url_abs":"https://arxiv.org/abs/2310.09017v3","url_pdf":"https://arxiv.org/pdf/2310.09017v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dont-add-dont-miss-effective-content","repo_url":"https://github.com/lovodkin93/cdr_ctr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.09017","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.09017"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lovodkin93/cdr_ctr","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0ed8bcc22ca9a631","entry":"compute_meteor_metrics","repo":"lovodkin93/cdr_ctr","repo_kind":"official","path":"src/compute_metrics.py","file_url":"https://github.com/lovodkin93/cdr_ctr/blob/HEAD/src/compute_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ed8bcc22ca9a631"}},{"code_sha256_prefix":"f7cda950ed471ba7","entry":"convert_row_spans_str_to_list_of_highlights","repo":"lovodkin93/cdr_ctr","repo_kind":"official","path":"src/lookahead/ctr_code/preprocessor.py","file_url":"https://github.com/lovodkin93/cdr_ctr/blob/HEAD/src/lookahead/ctr_code/preprocessor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f7cda950ed471ba7"}},{"code_sha256_prefix":"41836ad00ba12f98","entry":"fix_highlights","repo":"lovodkin93/cdr_ctr","repo_kind":"official","path":"src/lookahead/ctr_code/concatenate_highlights.py","file_url":"https://github.com/lovodkin93/cdr_ctr/blob/HEAD/src/lookahead/ctr_code/concatenate_highlights.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41836ad00ba12f98"}},{"code_sha256_prefix":"31a96d10c731f0a3","entry":"get_special_tokens_constants","repo":"lovodkin93/cdr_ctr","repo_kind":"official","path":"src/lookahead/ctr_code/preprocessor.py","file_url":"https://github.com/lovodkin93/cdr_ctr/blob/HEAD/src/lookahead/ctr_code/preprocessor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31a96d10c731f0a3"}},{"code_sha256_prefix":"2852c583a31c5ec4","entry":"convert_highlight_rows_to_document_highlights","repo":"lovodkin93/cdr_ctr","repo_kind":"official","path":"src/lookahead/ctr_code/preprocessor.py","file_url":"https://github.com/lovodkin93/cdr_ctr/blob/HEAD/src/lookahead/ctr_code/preprocessor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2852c583a31c5ec4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}