{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sornet-spatial-object-centric-representations","title":"SORNet: Spatial Object-Centric Representations for Sequential Manipulation","arxiv_id":"2109.03891","date":"2021-09-08","proceeding":null,"authors":["Wentao Yuan","Chris Paxton","Karthik Desingh","Dieter Fox"],"abstract":"Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state. In such tasks, the ability to reason about spatial relations among object entities from raw sensor inputs is crucial in order to determine when a task has been completed and which actions can be executed. In this work, we propose SORNet (Spatial Object-Centric Representation Network), a framework for learning object-centric representations from RGB images conditioned on a set of object queries, represented as image patches called canonical object views. With only a single canonical view per object and no annotation, SORNet generalizes zero-shot to object entities whose shape and texture are both unseen during training. We evaluate SORNet on various spatial reasoning tasks such as spatial relation classification and relative direction regression in complex tabletop manipulation scenarios and show that SORNet significantly outperforms baselines including state-of-the-art representation learning techniques. We also demonstrate the application of the representation learned by SORNet on visual-servoing and task planning for sequential manipulation on a real robot.","url_abs":"https://arxiv.org/abs/2109.03891v3","url_pdf":"https://arxiv.org/pdf/2109.03891v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sornet-spatial-object-centric-representations","repo_url":"https://github.com/wentaoyuan/sornet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"relation-classification","task_name":"Relation Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"},{"task_slug":"task-planning","task_name":"Task Planning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.03891","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.03891"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wentaoyuan/sornet","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0c414c16502ebb56","entry":"build_predicates","repo":"wentaoyuan/sornet","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c414c16502ebb56"}},{"code_sha256_prefix":"baad60b39603aa26","entry":"calc_acc","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_clevr.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_clevr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"baad60b39603aa26"}},{"code_sha256_prefix":"340a4f2d4088b73e","entry":"calc_acc","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_leonardo.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_leonardo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"340a4f2d4088b73e"}},{"code_sha256_prefix":"e42cc40dfb34d87e","entry":"calc_dist","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_regression.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_regression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e42cc40dfb34d87e"}},{"code_sha256_prefix":"70cbeab60da02673","entry":"step","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_clevr.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_clevr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70cbeab60da02673"}},{"code_sha256_prefix":"263ba8b7b267c3e4","entry":"step","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_leonardo.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_leonardo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"263ba8b7b267c3e4"}},{"code_sha256_prefix":"565a7402728223c4","entry":"step","repo":"wentaoyuan/sornet","repo_kind":"official","path":"train_regression.py","file_url":"https://github.com/wentaoyuan/sornet/blob/HEAD/train_regression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"565a7402728223c4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}