{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/soft-tensor-product-representations-for-fully","title":"Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products","arxiv_id":"2412.04671","date":"2024-12-05","proceeding":null,"authors":["Bethia Sun","Maurice Pagnucco","Yang song"],"abstract":"Since the inception of the classicalist vs. connectionist debate, it has been argued that the ability to systematically combine symbol-like entities into compositional representations is crucial for human intelligence. In connectionist systems, the field of disentanglement has gained prominence for its ability to produce explicitly compositional representations; however, it relies on a fundamentally symbolic, concatenative representation of compositional structure that clashes with the continuous, distributed foundations of deep learning. To resolve this tension, we extend Smolensky's Tensor Product Representation (TPR) and introduce Soft TPR, a representational form that encodes compositional structure in an inherently distributed, flexible manner, along with Soft TPR Autoencoder, a theoretically-principled architecture designed specifically to learn Soft TPRs. Comprehensive evaluations in the visual representation learning domain demonstrate that the Soft TPR framework consistently outperforms conventional disentanglement alternatives -- achieving state-of-the-art disentanglement, boosting representation learner convergence, and delivering superior sample efficiency and low-sample regime performance in downstream tasks. These findings highlight the promise of a distributed and flexible approach to representing compositional structure by potentially enhancing alignment with the core principles of deep learning over the conventional symbolic approach.","url_abs":"https://arxiv.org/abs/2412.04671v3","url_pdf":"https://arxiv.org/pdf/2412.04671v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"soft-tensor-product-representations-for-fully","repo_url":"https://github.com/gomb0c/soft_tpr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.04671","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.04671"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gomb0c/soft_tpr","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":11},"by_repo_kind":{"official":{"samples":12,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"34f0bd8890d57785","entry":"nearest_embed","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/model/vq_vae.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/model/vq_vae.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34f0bd8890d57785"}},{"code_sha256_prefix":"00ba486ea1f7537e","entry":"bce_recon_loss_fn","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/optim/loss.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/optim/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00ba486ea1f7537e"}},{"code_sha256_prefix":"406c6e802e6ba500","entry":"gaussian_kls","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/utils.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"406c6e802e6ba500"}},{"code_sha256_prefix":"a9ef158cd6f83f6d","entry":"get_activation_name","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/utils.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9ef158cd6f83f6d"}},{"code_sha256_prefix":"343bea31ec8dd2f5","entry":"get_most_recent_dir","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/logger/load.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/logger/load.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"343bea31ec8dd2f5"}},{"code_sha256_prefix":"0ed038365cbe2dc0","entry":"get_n_layers","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/utils.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ed038365cbe2dc0"}},{"code_sha256_prefix":"4217eff3614e5a21","entry":"get_result_from_summary_json","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/vis_results/plot_results_final-tpr_comparison.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/vis_results/plot_results_final-tpr_comparison.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4217eff3614e5a21"}},{"code_sha256_prefix":"76111b30fc874b16","entry":"get_save_path","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/logger/load.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/logger/load.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76111b30fc874b16"}},{"code_sha256_prefix":"2445492633b2e3e7","entry":"is_incorrect_run","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/vis_results/plot_results_final-tpr_comparison.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/vis_results/plot_results_final-tpr_comparison.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2445492633b2e3e7"}},{"code_sha256_prefix":"58f2f3e8317316ec","entry":"mse_recon_loss_fn","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/optim/loss.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/optim/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"58f2f3e8317316ec"}},{"code_sha256_prefix":"663b284fc8ed7db2","entry":"nearest_embed","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/repn_learners/soft_tpr_ae/model/nearest_embed.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/repn_learners/soft_tpr_ae/model/nearest_embed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"663b284fc8ed7db2"}},{"code_sha256_prefix":"b2d4cc7bb63548ca","entry":"parse_results_from_wandb_logs","repo":"gomb0c/soft_tpr","repo_kind":"official","path":"src/vis_results/plot_results_final-tpr_comparison.py","file_url":"https://github.com/gomb0c/soft_tpr/blob/HEAD/src/vis_results/plot_results_final-tpr_comparison.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2d4cc7bb63548ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}