{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/buildingsbench-a-large-scale-dataset-of-900k-1","title":"BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecasting","arxiv_id":"2307.00142","date":"2023-06-30","proceeding":"NeurIPS 2023 11","authors":["Patrick Emami","Abhijeet Sahu","Peter Graf"],"abstract":"Short-term forecasting of residential and commercial building energy consumption is widely used in power systems and continues to grow in importance. Data-driven short-term load forecasting (STLF), although promising, has suffered from a lack of open, large-scale datasets with high building diversity. This has hindered exploring the pretrain-then-fine-tune paradigm for STLF. To help address this, we present BuildingsBench, which consists of: 1) Buildings-900K, a large-scale dataset of 900K simulated buildings representing the U.S. building stock; and 2) an evaluation platform with over 1,900 real residential and commercial buildings from 7 open datasets. BuildingsBench benchmarks two under-explored tasks: zero-shot STLF, where a pretrained model is evaluated on unseen buildings without fine-tuning, and transfer learning, where a pretrained model is fine-tuned on a target building. The main finding of our benchmark analysis is that synthetically pretrained models generalize surprisingly well to real commercial buildings. An exploration of the effect of increasing dataset size and diversity on zero-shot commercial building performance reveals a power-law with diminishing returns. We also show that fine-tuning pretrained models on real commercial and residential buildings improves performance for a majority of target buildings. We hope that BuildingsBench encourages and facilitates future research on generalizable STLF. All datasets and code can be accessed from https://github.com/NREL/BuildingsBench.","url_abs":"https://arxiv.org/abs/2307.00142v3","url_pdf":"https://arxiv.org/pdf/2307.00142v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"buildingsbench-a-large-scale-dataset-of-900k-1","repo_url":"https://github.com/nrel/buildingsbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"load-forecasting","task_name":"Load Forecasting"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.00142","atlas_url":"https://app.syntology.ai/?focus=2307.00142","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.00142"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NREL/BuildingsBench","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nrel/buildingsbench","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f0c9e682f0cab1f6","entry":"absolute_error","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/evaluation/metrics.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c9e682f0cab1f6"}},{"code_sha256_prefix":"f975e66c06137e29","entry":"bias_error","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/evaluation/metrics.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f975e66c06137e29"}},{"code_sha256_prefix":"6c1ee89025af6a2e","entry":"get_puma_county_lookup_table","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/utils.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6c1ee89025af6a2e"}},{"code_sha256_prefix":"673a7f585bf3c71b","entry":"load_model_checkpoint","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/utils.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"673a7f585bf3c71b"}},{"code_sha256_prefix":"97e0d2619679e5a7","entry":"squared_error","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/evaluation/metrics.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"97e0d2619679e5a7"}},{"code_sha256_prefix":"40bc59777d0a77cc","entry":"time_features_to_datetime","repo":"NREL/BuildingsBench","repo_kind":"official","path":"buildings_bench/utils.py","file_url":"https://github.com/NREL/BuildingsBench/blob/HEAD/buildings_bench/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"40bc59777d0a77cc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}