{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-latents-benchmark-21-evaluating-latent","title":"Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity","arxiv_id":"2109.04463","date":"2021-09-09","proceeding":null,"authors":["Felix Pei","Joel Ye","David Zoltowski","Anqi Wu","Raeed H. Chowdhury","Hansem Sohn","Joseph E. O'Doherty","Krishna V. Shenoy","Matthew T. Kaufman","Mark Churchland","Mehrdad Jazayeri","Lee E. Miller","Jonathan Pillow","Il Memming Park","Eva L. Dyer","Chethan Pandarinath"],"abstract":"Advances in neural recording present increasing opportunities to study neural activity in unprecedented detail. Latent variable models (LVMs) are promising tools for analyzing this rich activity across diverse neural systems and behaviors, as LVMs do not depend on known relationships between the activity and external experimental variables. However, progress with LVMs for neuronal population activity is currently impeded by a lack of standardization, resulting in methods being developed and compared in an ad hoc manner. To coordinate these modeling efforts, we introduce a benchmark suite for latent variable modeling of neural population activity. We curate four datasets of neural spiking activity from cognitive, sensory, and motor areas to promote models that apply to the wide variety of activity seen across these areas. We identify unsupervised evaluation as a common framework for evaluating models across datasets, and apply several baselines that demonstrate benchmark diversity. We release this benchmark through EvalAI. http://neurallatents.github.io","url_abs":"https://arxiv.org/abs/2109.04463v4","url_pdf":"https://arxiv.org/pdf/2109.04463v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-latents-benchmark-21-evaluating-latent","repo_url":"https://github.com/neurallatents/nlb_tools","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"neural-latents-benchmark-21-evaluating-latent","repo_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"}],"methods":[{"method_slug":"hoc","method_name":"HOC"}],"datasets_introduced":[{"slug":"nlb","name":"NLB","full_name":"Neural Latents Benchmark"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2109.04463","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.04463"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neurallatents/nlb_tools","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kabirdabholkar/nlb_tools_fewshot","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":24},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1},"listed":{"samples":15,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e44e57b951a765b2","entry":"bits_per_spike","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/metrics.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e44e57b951a765b2"}},{"code_sha256_prefix":"0d3d3adb1e0b12f1","entry":"bits_per_spike","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/evaluation.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d3d3adb1e0b12f1"}},{"code_sha256_prefix":"7be81192bfe31963","entry":"chop_data","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/chop.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/chop.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7be81192bfe31963"}},{"code_sha256_prefix":"bd7c0d0bf0c8c24c","entry":"evaluate","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/evaluation.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd7c0d0bf0c8c24c"}},{"code_sha256_prefix":"14fc654bea6d290a","entry":"evaluate","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/evaluation.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14fc654bea6d290a"}},{"code_sha256_prefix":"edfc3fd7ed9643f0","entry":"extract_reallyheldout_by_id","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/make_fewshot.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/make_fewshot.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"edfc3fd7ed9643f0"}},{"code_sha256_prefix":"dd4a6f58dc498993","entry":"fewshot_from_train","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/make_fewshot.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/make_fewshot.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd4a6f58dc498993"}},{"code_sha256_prefix":"2687687849f6dbf2","entry":"fit_poisson","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/sklearn_glm.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/sklearn_glm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2687687849f6dbf2"}},{"code_sha256_prefix":"2477a50a3c586b76","entry":"fit_poisson_parallel","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/sklearn_glm.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/sklearn_glm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2477a50a3c586b76"}},{"code_sha256_prefix":"5cc10defe6e4c731","entry":"generate_unique_prefix","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/collate_results.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/collate_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cc10defe6e4c731"}},{"code_sha256_prefix":"861c3e54aa2c4e05","entry":"load_result_files","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/cross_decoding.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/cross_decoding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"861c3e54aa2c4e05"}},{"code_sha256_prefix":"eff7a97dd8cb0604","entry":"make_eval_input_tensors","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/make_tensors.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/make_tensors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eff7a97dd8cb0604"}},{"code_sha256_prefix":"bcfca1cb80e661c1","entry":"make_eval_target_tensors","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/make_tensors.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/make_tensors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcfca1cb80e661c1"}},{"code_sha256_prefix":"16f126abd979f82f","entry":"make_train_input_tensors","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/make_tensors.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/make_tensors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16f126abd979f82f"}},{"code_sha256_prefix":"3d8a5bc8cf916fb0","entry":"merge_chops","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/chop.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/chop.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d8a5bc8cf916fb0"}},{"code_sha256_prefix":"dd5f9738be104eaf","entry":"neg_log_likelihood","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/evaluation.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd5f9738be104eaf"}},{"code_sha256_prefix":"a27593d38113b6a5","entry":"r2_score","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/metrics.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a27593d38113b6a5"}},{"code_sha256_prefix":"68e6cf9fdc15aae9","entry":"read_concat_results","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/collate_results.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/collate_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68e6cf9fdc15aae9"}},{"code_sha256_prefix":"b2719609b2d567a8","entry":"regional_bits_per_spike","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/metrics.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2719609b2d567a8"}},{"code_sha256_prefix":"de16cf54220b6637","entry":"result_dict_to_pandas","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/fewshot_utils.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/fewshot_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de16cf54220b6637"}},{"code_sha256_prefix":"984d705b60df1c36","entry":"run_model_on_numpy_dummy","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/load_and_save_latents.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/load_and_save_latents.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"984d705b60df1c36"}},{"code_sha256_prefix":"9c1716805b975c6e","entry":"select_best_cobps","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/cross_decoding.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/cross_decoding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c1716805b975c6e"}},{"code_sha256_prefix":"67a8701afb5319da","entry":"smooth_column","repo":"neurallatents/nlb_tools","repo_kind":"official","path":"nlb_tools/nwb_interface.py","file_url":"https://github.com/neurallatents/nlb_tools/blob/HEAD/nlb_tools/nwb_interface.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67a8701afb5319da"}},{"code_sha256_prefix":"abd84c9759241332","entry":"split_into_chunks_of_K","repo":"kabirdabholkar/nlb_tools_fewshot","repo_kind":"listed","path":"nlb_tools/make_fewshot.py","file_url":"https://github.com/kabirdabholkar/nlb_tools_fewshot/blob/HEAD/nlb_tools/make_fewshot.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abd84c9759241332"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}