{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unmasking-the-inductive-biases-of","title":"Benchmarking Unsupervised Object Representations for Video Sequences","arxiv_id":"2006.07034","date":"2020-06-12","proceeding":null,"authors":["Marissa A. Weis","Kashyap Chitta","Yash Sharma","Wieland Brendel","Matthias Bethge","Andreas Geiger","Alexander S. Ecker"],"abstract":"Perceiving the world in terms of objects and tracking them through time is a crucial prerequisite for reasoning and scene understanding. Recently, several methods have been proposed for unsupervised learning of object-centric representations. However, since these models were evaluated on different downstream tasks, it remains unclear how they compare in terms of basic perceptual abilities such as detection, figure-ground segmentation and tracking of objects. To close this gap, we design a benchmark with four data sets of varying complexity and seven additional test sets featuring challenging tracking scenarios relevant for natural videos. Using this benchmark, we compare the perceptual abilities of four object-centric approaches: ViMON, a video-extension of MONet, based on recurrent spatial attention, OP3, which exploits clustering via spatial mixture models, as well as TBA and SCALOR, which use explicit factorization via spatial transformers. Our results suggest that the architectures with unconstrained latent representations learn more powerful representations in terms of object detection, segmentation and tracking than the spatial transformer based architectures. We also observe that none of the methods are able to gracefully handle the most challenging tracking scenarios despite their synthetic nature, suggesting that our benchmark may provide fruitful guidance towards learning more robust object-centric video representations.","url_abs":"https://arxiv.org/abs/2006.07034v2","url_pdf":"https://arxiv.org/pdf/2006.07034v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unmasking-the-inductive-biases-of","repo_url":"https://github.com/ecker-lab/object-centric-representation-benchmark","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"multi-object-tracking","task_name":"Multi-Object Tracking"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatial-transformer","method_name":"Spatial Transformer"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2006.07034","atlas_url":"https://app.syntology.ai/?focus=2006.07034","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.07034"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ecker-lab/object-centric-representation-benchmark","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9910e2fc297f8665","entry":"identity","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/op3/networks.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/op3/networks.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9910e2fc297f8665"}},{"code_sha256_prefix":"5cb0c55f58279f59","entry":"simple_separated_format","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/op3/core/tabulate.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/op3/core/tabulate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cb0c55f58279f59"}},{"code_sha256_prefix":"42612b635bfa2e19","entry":"add_prefix","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/op3/core/logging.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/op3/core/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42612b635bfa2e19"}},{"code_sha256_prefix":"1029760706cbba76","entry":"append_log","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/op3/core/logging.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/op3/core/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1029760706cbba76"}},{"code_sha256_prefix":"575f1a1d9fa46573","entry":"binarize_masks","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/eval/eval_utils.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/eval/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"575f1a1d9fa46573"}},{"code_sha256_prefix":"07dca4e4203d7201","entry":"calculate_iou","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/eval/eval_utils.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/eval/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07dca4e4203d7201"}},{"code_sha256_prefix":"dbd33c458e391c84","entry":"compute_mot_metrics","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/eval/eval_utils.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/eval/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbd33c458e391c84"}},{"code_sha256_prefix":"ebb9f263536c12da","entry":"exclude_bg","repo":"ecker-lab/object-centric-representation-benchmark","repo_kind":"official","path":"ocrb/eval/eval_mot.py","file_url":"https://github.com/ecker-lab/object-centric-representation-benchmark/blob/HEAD/ocrb/eval/eval_mot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebb9f263536c12da"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}