{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/future-events-as-backdoor-triggers","title":"Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs","arxiv_id":"2407.04108","date":"2024-07-04","proceeding":null,"authors":["Sara Price","Arjun Panickssery","Sam Bowman","Asa Cooper Stickland"],"abstract":"Backdoors are hidden behaviors that are only triggered once an AI system has been deployed. Bad actors looking to create successful backdoors must design them to avoid activation during training and evaluation. Since data used in these stages often only contains information about events that have already occurred, a component of a simple backdoor trigger could be a model recognizing data that is in the future relative to when it was trained. Through prompting experiments and by probing internal activations, we show that current large language models (LLMs) can distinguish past from future events, with probes on model activations achieving 90% accuracy. We train models with backdoors triggered by a temporal distributional shift; they activate when the model is exposed to news headlines beyond their training cut-off dates. Fine-tuning on helpful, harmless and honest (HHH) data does not work well for removing simpler backdoor triggers but is effective on our backdoored models, although this distinction is smaller for the larger-scale model we tested. We also find that an activation-steering vector representing a model's internal representation of the date influences the rate of backdoor activation. We take these results as initial evidence that, at least for models at the modest scale we test, standard safety measures are enough to remove these backdoors.","url_abs":"https://arxiv.org/abs/2407.04108v3","url_pdf":"https://arxiv.org/pdf/2407.04108v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"future-events-as-backdoor-triggers","repo_url":"https://github.com/sbp354/future_triggered_backdoors","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.04108","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04108"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sbp354/future_triggered_backdoors","reach":{"status":"ok"}}],"summary":{"ran":9,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"559ab80369898623","entry":"clean_output","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/fcc/utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/fcc/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"559ab80369898623"}},{"code_sha256_prefix":"29fa20cd04b72bb3","entry":"format_sigfigs","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/fcc/utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/fcc/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29fa20cd04b72bb3"}},{"code_sha256_prefix":"80941a6b4b9a6ea8","entry":"get_activations","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/linear_probes/utils/model_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/linear_probes/utils/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80941a6b4b9a6ea8"}},{"code_sha256_prefix":"ed61d87fda9f6a73","entry":"get_agg_stats","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/fcc/plot_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/fcc/plot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed61d87fda9f6a73"}},{"code_sha256_prefix":"abbfddd667e95681","entry":"get_logit_diff","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/mech_interp_experiments/patching_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/mech_interp_experiments/patching_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"abbfddd667e95681"}},{"code_sha256_prefix":"2a75fb471a0321a6","entry":"get_prompt","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/headline_prompting/prompting_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/headline_prompting/prompting_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a75fb471a0321a6"}},{"code_sha256_prefix":"6c8235303d623c0c","entry":"load_activations","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/linear_probes/utils/model_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/linear_probes/utils/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c8235303d623c0c"}},{"code_sha256_prefix":"5114a76c19ed4e1f","entry":"load_secrets","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/headline_prompting/prompting_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/headline_prompting/prompting_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5114a76c19ed4e1f"}},{"code_sha256_prefix":"c69f6de628152645","entry":"logits_to_ave_logit_diff","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/mech_interp_experiments/patching_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/mech_interp_experiments/patching_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c69f6de628152645"}},{"code_sha256_prefix":"f200d942b0a2fc20","entry":"assess_logit_diff","repo":"sbp354/future_triggered_backdoors","repo_kind":"official","path":"future_probing/mech_interp_experiments/patching_utils.py","file_url":"https://github.com/sbp354/future_triggered_backdoors/blob/HEAD/future_probing/mech_interp_experiments/patching_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f200d942b0a2fc20"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}