{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-object-representation-learning","title":"Unsupervised Object Representation Learning using Translation and Rotation Group Equivariant VAE","arxiv_id":"2210.12918","date":"2022-10-24","proceeding":null,"authors":["Alireza Nasiri","Tristan Bepler"],"abstract":"In many imaging modalities, objects of interest can occur in a variety of locations and poses (i.e. are subject to translations and rotations in 2d or 3d), but the location and pose of an object does not change its semantics (i.e. the object's essence). That is, the specific location and rotation of an airplane in satellite imagery, or the 3d rotation of a chair in a natural image, or the rotation of a particle in a cryo-electron micrograph, do not change the intrinsic nature of those objects. Here, we consider the problem of learning semantic representations of objects that are invariant to pose and location in a fully unsupervised manner. We address shortcomings in previous approaches to this problem by introducing TARGET-VAE, a translation and rotation group-equivariant variational autoencoder framework. TARGET-VAE combines three core innovations: 1) a rotation and translation group-equivariant encoder architecture, 2) a structurally disentangled distribution over latent rotation, translation, and a rotation-translation-invariant semantic object representation, which are jointly inferred by the approximate inference network, and 3) a spatially equivariant generator network. In comprehensive experiments, we show that TARGET-VAE learns disentangled representations without supervision that significantly improve upon, and avoid the pathologies of, previous methods. When trained on images highly corrupted by rotation and translation, the semantic representations learned by TARGET-VAE are similar to those learned on consistently posed objects, dramatically improving clustering in the semantic latent space. Furthermore, TARGET-VAE is able to perform remarkably accurate unsupervised pose and location inference. We expect methods like TARGET-VAE will underpin future approaches for unsupervised object generation, pose prediction, and object detection.","url_abs":"https://arxiv.org/abs/2210.12918v2","url_pdf":"https://arxiv.org/pdf/2210.12918v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-object-representation-learning","repo_url":"https://github.com/smlc-nysbc/target-vae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"learning-semantic-representations","task_name":"Learning Semantic Representations"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"pose-prediction","task_name":"Pose Prediction"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.12918","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.12918"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/smlc-nysbc/target-vae","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":20},"by_repo_kind":{"official":{"samples":20,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5260d45bd76e16cc","entry":"cluster_acc","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"clustering_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/clustering_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5260d45bd76e16cc"}},{"code_sha256_prefix":"e9e60431bb497e09","entry":"cluster_acc","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"clustering_mnist.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/clustering_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9e60431bb497e09"}},{"code_sha256_prefix":"20061b753ab107e5","entry":"compute_2d_ctf","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/ctf.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/ctf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20061b753ab107e5"}},{"code_sha256_prefix":"69f6f08ec67a0821","entry":"crop","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/image.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69f6f08ec67a0821"}},{"code_sha256_prefix":"1fd769897b5c1e01","entry":"ctf_filter","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/ctf.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/ctf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1fd769897b5c1e01"}},{"code_sha256_prefix":"65871fecaa6f5b52","entry":"downsample","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/image.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65871fecaa6f5b52"}},{"code_sha256_prefix":"ca7bf21d29efd052","entry":"eval_minibatch","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca7bf21d29efd052"}},{"code_sha256_prefix":"f7bf32be6ed3b3ce","entry":"eval_minibatch","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_galaxy.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_galaxy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7bf32be6ed3b3ce"}},{"code_sha256_prefix":"a0c4d1584fb072d5","entry":"eval_model","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0c4d1584fb072d5"}},{"code_sha256_prefix":"9769b6fef90de4d5","entry":"eval_model","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_galaxy.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_galaxy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9769b6fef90de4d5"}},{"code_sha256_prefix":"1d38fa41d0b89adb","entry":"get_latent","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"clustering_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/clustering_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d38fa41d0b89adb"}},{"code_sha256_prefix":"c5f1b20bf55ae6d8","entry":"get_latent","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"clustering_mnist.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/clustering_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5f1b20bf55ae6d8"}},{"code_sha256_prefix":"48088186a5bb2896","entry":"get_mode","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/mrc.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48088186a5bb2896"}},{"code_sha256_prefix":"8c582ad5126b1725","entry":"load_images","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"clustering_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/clustering_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c582ad5126b1725"}},{"code_sha256_prefix":"3e39b8fba4e5dc36","entry":"make_header","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/mrc.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e39b8fba4e5dc36"}},{"code_sha256_prefix":"39e9adc0e88f7f86","entry":"normalize","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/image.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39e9adc0e88f7f86"}},{"code_sha256_prefix":"e23c9b574323efb9","entry":"parse","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/mrc.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/mrc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e23c9b574323efb9"}},{"code_sha256_prefix":"9ef31dc8425cf565","entry":"parse_ctf","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"src/ctf.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/src/ctf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ef31dc8425cf565"}},{"code_sha256_prefix":"fef77345332c002c","entry":"train_epoch","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_dsprites.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_dsprites.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fef77345332c002c"}},{"code_sha256_prefix":"d4b03d98b21b85d0","entry":"train_epoch","repo":"smlc-nysbc/target-vae","repo_kind":"official","path":"train_galaxy.py","file_url":"https://github.com/smlc-nysbc/target-vae/blob/HEAD/train_galaxy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4b03d98b21b85d0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}