{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-13981","title":"Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines","arxiv_id":"2605.13981","date":"2026-05-13","proceeding":"ICML","authors":["Katherine Lambert","Sasha Luccioni"],"abstract":"The rise in deployment of large language models has driven a surge in GPU demand and datacenter scaling, raising concerns about electricity use, grid stress, and the impacts of modern AI workloads. Distillation is often promoted as one of the most effective paths to obtain cheaper, more efficient models, yet these claims rarely account for the full end-to-end energy and resource costs, including crucial teacher-side workloads such as data generation, logit caching, and evaluation. We present a comprehensive energy accounting framework that measures the complete computational cost of distillation pipelines via detailed stage-wise tracking of GPU device power consumption. In our experiments, we separate and log empirical energy use across distinct phases and systematically measure the energy and emissions of two common distillation methods: the classic logit-based knowledge distillation and synthetic-data supervised fine-tuning, constructing energy-quality Pareto frontiers that expose the previously ignored costs. From these measurements and analyses, we derive practical design rules for selecting distillation methods and hyperparameters under energy and budget constraints, and release an open-source measurement harness and accounting protocol to provide a standardized foundation for comparable, reproducible distillation research, explicitly accountable for complete pipeline energy impact.","url_abs":"https://arxiv.org/abs/2605.13981","url_pdf":"https://arxiv.org/pdf/2605.13981","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.13981","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.13981"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/StellarLuminosity/Energy","reach":null}],"summary":{"ran":7,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"a203a11333a9a700","entry":"compute_ce_loss","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/pipelines/kd_eval.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/pipelines/kd_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a203a11333a9a700"}},{"code_sha256_prefix":"12780389543962b9","entry":"compute_sft_loss","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/pipelines/sft_main.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/pipelines/sft_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12780389543962b9"}},{"code_sha256_prefix":"a3c4618734a7e045","entry":"eval_model","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/pipelines/sft_main.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/pipelines/sft_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3c4618734a7e045"}},{"code_sha256_prefix":"5493aa802657f082","entry":"load_yaml","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/core/config_loader.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/core/config_loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5493aa802657f082"}},{"code_sha256_prefix":"ec6c6fa1db88bcea","entry":"merge_configs","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/core/config_loader.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/core/config_loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec6c6fa1db88bcea"}},{"code_sha256_prefix":"7aca9ebb37ee1d3e","entry":"prepare_dataset","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/core/utils.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/core/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7aca9ebb37ee1d3e"}},{"code_sha256_prefix":"8b697807fda248c6","entry":"validate_sampling_interval","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/core/prerun.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/core/prerun.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b697807fda248c6"}},{"code_sha256_prefix":"701ac1f85631179f","entry":"load_config","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/core/config_loader.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/core/config_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"701ac1f85631179f"}},{"code_sha256_prefix":"99c76a090a976891","entry":"load_model","repo":"StellarLuminosity/Energy","repo_kind":"found_in_text","path":"distill_bench/pipelines/kd_eval.py","file_url":"https://github.com/StellarLuminosity/Energy/blob/HEAD/distill_bench/pipelines/kd_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99c76a090a976891"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}