{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-se-3-equivariant-geometric-reasoning-for","title":"Deep SE(3)-Equivariant Geometric Reasoning for Precise Placement Tasks","arxiv_id":"2404.13478","date":"2024-04-20","proceeding":null,"authors":["Ben Eisner","Yi Yang","Todor Davchev","Mel Vecerik","Jonathan Scholz","David Held"],"abstract":"Many robot manipulation tasks can be framed as geometric reasoning tasks, where an agent must be able to precisely manipulate an object into a position that satisfies the task from a set of initial conditions. Often, task success is defined based on the relationship between two objects - for instance, hanging a mug on a rack. In such cases, the solution should be equivariant to the initial position of the objects as well as the agent, and invariant to the pose of the camera. This poses a challenge for learning systems which attempt to solve this task by learning directly from high-dimensional demonstrations: the agent must learn to be both equivariant as well as precise, which can be challenging without any inductive biases about the problem. In this work, we propose a method for precise relative pose prediction which is provably SE(3)-equivariant, can be learned from only a few demonstrations, and can generalize across variations in a class of objects. We accomplish this by factoring the problem into learning an SE(3) invariant task-specific representation of the scene and then interpreting this representation with novel geometric reasoning layers which are provably SE(3) equivariant. We demonstrate that our method can yield substantially more precise placement predictions in simulated placement tasks than previous methods trained with the same amount of data, and can accurately represent relative placement relationships data collected from real-world demonstrations. Supplementary information and videos can be found at https://sites.google.com/view/reldist-iclr-2023.","url_abs":"https://arxiv.org/abs/2404.13478v1","url_pdf":"https://arxiv.org/pdf/2404.13478v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-se-3-equivariant-geometric-reasoning-for","repo_url":"https://github.com/r-pad/taxpose","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"pose-prediction","task_name":"Pose Prediction"},{"task_slug":null,"task_name":"Position"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2404.13478","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.13478"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/r-pad/taxpose","reach":null}],"summary":{"ran":10,"ran_honours":1,"ran_fixture":2,"unverified":9},"by_repo_kind":{"listed":{"samples":21,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"0256b22bbf6fc0bd","entry":"DGCNN","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0256b22bbf6fc0bd"}},{"code_sha256_prefix":"511f66707327f995","entry":"Decoder","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"511f66707327f995"}},{"code_sha256_prefix":"6fa6bcc8cc01f9be","entry":"DecoderLayer","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6fa6bcc8cc01f9be"}},{"code_sha256_prefix":"065e92ade1cc3217","entry":"Encoder","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"065e92ade1cc3217"}},{"code_sha256_prefix":"0b1c98d83e7c5bed","entry":"EncoderDecoder","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b1c98d83e7c5bed"}},{"code_sha256_prefix":"24ccd00d4adcc3a9","entry":"EncoderLayer","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24ccd00d4adcc3a9"}},{"code_sha256_prefix":"2aaa428ab3d2f019","entry":"MultiHeadedAttention","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2aaa428ab3d2f019"}},{"code_sha256_prefix":"79a4b9f82b00cd47","entry":"PointNet","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79a4b9f82b00cd47"}},{"code_sha256_prefix":"49c1ccb5ad835ec4","entry":"PositionwiseFeedForward","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49c1ccb5ad835ec4"}},{"code_sha256_prefix":"dd481d1dfb89ace0","entry":"SublayerConnection","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd481d1dfb89ace0"}},{"code_sha256_prefix":"70ba9ecc422a6a48","entry":"attention","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"70ba9ecc422a6a48"}},{"code_sha256_prefix":"a4df418e59bd929e","entry":"get_graph_feature","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4df418e59bd929e"}},{"code_sha256_prefix":"35d6887261f3e988","entry":"knn","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35d6887261f3e988"}},{"code_sha256_prefix":"db755fe5ed8d6e0e","entry":"CustomTransformer","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db755fe5ed8d6e0e"}},{"code_sha256_prefix":"06d70af2876337b8","entry":"MLPKernel","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06d70af2876337b8"}},{"code_sha256_prefix":"ef5dbe43a9efc444","entry":"MultilaterationHead","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef5dbe43a9efc444"}},{"code_sha256_prefix":"8908af7a125f3b6b","entry":"ResidualFlow_DiffEmbTransformer","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8908af7a125f3b6b"}},{"code_sha256_prefix":"fe5f0050772bc142","entry":"ResidualMLPHead","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5f0050772bc142"}},{"code_sha256_prefix":"e77d7759eb725806","entry":"VNArgs","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e77d7759eb725806"}},{"code_sha256_prefix":"9c5f4a44ab78a987","entry":"VN_DGCNN","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c5f4a44ab78a987"}},{"code_sha256_prefix":"97b9e03315e765bc","entry":"create_embedding_network","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97b9e03315e765bc"}},{"code_sha256_prefix":"92c2f28b7b961ce2","entry":"estimate_p","repo":"r-pad/taxpose","repo_kind":"listed","path":"taxpose/nets/transformer_flow.py","file_url":"https://github.com/r-pad/taxpose/blob/HEAD/taxpose/nets/transformer_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92c2f28b7b961ce2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}