Papers › VisualPatchWorld: Code World Models as Latent Structured Representations for Planning

VisualPatchWorld: Code World Models as Latent Structured Representations for Planning

28 Jul 2026arXiv:2607.25236added by Syntology

Jiaxin Bai, Jiaxuan Xiong

Title, abstract, authors and date from arXiv's metadata (CC0); this paper is not in the Papers with Code archive (frozen 2025-07-28).

Different research lines use the term world model in different ways, yet they share a common aim: to capture how the world evolves under action in a form that supports perception, simulation, and planning. Two prominent realizations are neural predictors that learn dynamics in continuous vector spaces, and hand-built physics engines that expose explicit state and physical laws. Neural predictors scale from data but leave the form of the dynamics implicit; physics engines are inspectable and editable but difficult to construct at scale. We introduce VisualPatchWorld (VPW), which represents world dynamics as code. VPW first selects a qualitative dynamical form with short active probes, then fits that form's free parameters from recorded state-action traces by minimizing multi-step prediction error. The resulting programs can be rolled forward like a simulator, inspected in source form, and used inside model-predictive control; image-derived scene graphs can supply the live state at replan time. Across comparisons with prior code-based world models, VPW attains 69.0% mean planning success and exceeds the strongest code baseline by 23.5 points. The largest gains arise when choosing the correct qualitative dynamics is essential. Under the same planner, the induced models approach ground-truth engine success on navigation and grasp-rich control; a residual gap remains for contact-rich pushing, and checking a shortlist of promising plans in the engine closes most of that gap. These results establish a practical route toward automatically constructed code world models that are useful for planning. Code is available at https://github.com/HKBU-KnowComp/VisualPatchWorld/.

PaperPDFCode Syntology ranMethod as code

In Syntology View this paper on Syntology, its page in Syntology's graph. That page adds which of its functions are also present in other papers' code Syntology holds, with dates, and its dated run record as JSON and a README badge. It also lists the repositories linked to the paper, the abstract and the calls for agents.

For agents, on Syntology's MCP service (how to connect):

Code

Syntology 44 code samples harvested from 1 repository linked to this paper: 23 ran (of which 7 constructed an object rather than computing a result; 15 with no instrument failure: 7 honoured, 0 violated, 8 with no contract checked; 8 where Syntology's instrument failed) · 21 unverified. Where Syntology's instrument failed: 7 where our contract draft was wrong; 1 where the contract check could not run; failures of Syntology's instrument, not of the code.

By repository: found in paper text by Syntology: 44 samples from 1 repository, 23 ran (15 with no instrument failure; 8 where Syntology's instrument failed). The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

Levels written on 2026-09-22 (UTC): the dates Syntology's graph wrote each sample's verification level, not when the code executed.

HKBU-KnowComp/VisualPatchWorld found in paper text by SyntologySyntology: 44 samples, 23 ran (8 instrument-failed) · 21 unverified report

Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

44 samples harvested: 23 ran (of which 7 constructed an object rather than computing a result; 15 with no instrument failure: 7 honoured, 0 violated, 8 with no contract checked; 8 where Syntology's instrument failed) · 21 unverified. “Where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. That date is not when the samples ran.

7ran · honoured contract
7ran · our draft was wrong
1ran; the contract check could not run
8ran
21unverified

Licence: 0 of the 44 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from HKBU-KnowComp/VisualPatchWorld. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. A line beginning “fixture:” under a sample that ran gives the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right. “Level recorded” gives the UTC date when Syntology's graph wrote the sample's verification level, not when the code executed; where the level was written before Syntology's graph kept that date, the row says “level recorded, date not recorded”. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself. “Our draft was wrong” and “the contract check could not run” are failures of Syntology's instrument, not of the code: the sample ran on Syntology's synthesized input, and then the drafted contract did not describe it, or the check itself raised an error (RAISES), lacked a dependency (DEP_MISSING) or ran out of time (TIMEOUT); hover the label for which. A metamorphic tier, where shown, is evidence of the output's shape, not that the sample implements the method.

Each sample ends with its code_sha256. A sample's code_sha256 is Syntology's identity for that exact code. It is printed in full (16 hexadecimal characters for every sample in this build), and get_code(code_sha256="…") takes it exactly as printed. An agent calls it on Syntology's MCP service (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples are grouped by repository: a repository marked official first, then by how directly the repository is tied to the paper, then in the archive's order; never by how many ran. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of six reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar); this paper's copy was not located, so its licence is not recorded; or this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not. Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

A function or class name links to its page when the name occurs in the harvested code of at least 5 papers; the name is a string, not an identity.

HKBU-KnowComp/VisualPatchWorld found in paper text by Syntology 44 samples, 23 ran (8 instrument-failed) · 21 unverified
EnvStructureReport src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · 3b3ac601d6efe942 · report fixture: returned EnvStructureReport · level recorded 2026-09-22
EpisodeSlice src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · b973a7b79d96cfa9 · report fixture: returned EpisodeSlice · level recorded 2026-09-22
LeWMEnvSpec src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · c6ccd43fd3e2fbc1 · report fixture: returned LeWMEnvSpec · level recorded 2026-09-22
ProbeResult src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · 8c59c27d7893b0a2 · report fixture: returned ProbeResult · level recorded 2026-09-22
TaggedTransition src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · 7dd0f6681042a092 · report fixture: returned TaggedTransition · level recorded 2026-09-22
Trajectory in 7 papers src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · cec67a4407d01cf3 · report fixture: returned Trajectory · level recorded 2026-09-22
Transition in 7 papers src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · ccc986c237c99cc1 · report fixture: returned Transition · level recorded 2026-09-22
_agent_xy src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · 8eae1dea8284248c · report fixture: signature row:dict/0/?, env_name:str/0/str · returned ndarray · level recorded 2026-09-22
_as_int_list src/visualpatchworld/structure_induction.py ran · our draft was wrong fingerprinted MIT (permissive) · e1995fb8c273c632 · report fixture: signature values:list/1/float · returned list · level recorded 2026-09-22
_cube_block_xy src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · 30c840c93b496349 · report fixture: signature row:dict/0/? · returned ndarray · level recorded 2026-09-22
_first_valid_start src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · b190ceebd9f6cad0 · report fixture: signature h5:MockH5/0/?, goal_offset_steps:int/0/int, seed:int/0/int · returned tuple · level recorded 2026-09-22
_reacher_qpos src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · dac6b3bc02a8cc8e · report fixture: signature row:dict/0/? · returned ndarray · level recorded 2026-09-22
_reacher_qvel src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · 2c071d1d8488c3db · report fixture: signature row:dict/0/? · returned ndarray · level recorded 2026-09-22
_sim_obs_vector src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · ef44efed8670fb6c · report fixture: signature env:MockEnv/0/?, spec:LeWMEnvSpec/0/?, obs:ndarray/1/float · returned ndarray · level recorded 2026-09-22
_target_xy src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · e1f7c095971a2c84 · report fixture: signature row:dict/0/?, env_name:str/0/str · returned ndarray · level recorded 2026-09-22
goal_row_from_start src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · 10040f293d33680e · report fixture: signature h5:MockH5/0/?, spec:LeWMEnvSpec/0/?, episode_index:int/0/int, start_step:int/0/int, goal_offset:int/0/int · returned dict · level recorded 2026-09-22
probe_pusht_structure src/visualpatchworld/structure_induction.py ran; the contract check could not run (DEP_MISSING) · metamorphic tier: deterministic MIT (permissive) · e0b03e59ea78068d · report fixture: signature geometry:dict/0/? · returned list · level recorded 2026-09-22
require_h5py src/visualpatchworld/structure_induction.py ran · metamorphic tier: deterministic MIT (permissive) · a04c4f5412023862 · report fixture: returned module · level recorded 2026-09-22
reset_env_from_rows src/visualpatchworld/structure_induction.py ran · honoured contract MIT (permissive) · 74b8d14d662e95eb · report fixture: signature env:MockEnv/0/?, spec:LeWMEnvSpec/0/?, start_row:dict/0/?, goal_row:dict/0/? · returned ndarray · level recorded 2026-09-22
resolve_dataset_path src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · 119bc8041bb90077 · report fixture: signature dataset:str/0/str, stablewm_home:str/0/str · returned PosixPath · level recorded 2026-09-22
row_dict_from_h5 src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · f666f32a7d3abc1c · report fixture: signature h5:MockH5/0/?, idx:int/0/int, spec:LeWMEnvSpec/0/? · returned dict · level recorded 2026-09-22
sim_step_with_info src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · 3e34eae5424d18e7 · report fixture: signature env:MockEnv/0/?, spec:LeWMEnvSpec/0/?, action:ndarray/1/float · returned tuple · level recorded 2026-09-22
structure_sketch_from_probes src/visualpatchworld/structure_induction.py ran · our draft was wrong MIT (permissive) · f5b09b8e9f3a9e62 · report fixture: signature results:list/1/ProbeResult · returned dict · level recorded 2026-09-22
Timeline src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 0c9a53a186fd8ffa · report
_macro_step src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 58202bf4e83babab · report
_sync_row_after_reset src/visualpatchworld/structure_induction.py unverified MIT (permissive) · ef695b35deb9f72b · report
episode_slices src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 4a1f6f624b72e1f1 · report
get_env_spec src/visualpatchworld/structure_induction.py unverified MIT (permissive) · c0101a75a31bb147 · report
identify_env_structure src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 714431e8d86d6708 · report
induce_from_structure src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 23e0b564c40c423e · report
load_probe_rows src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 7d1ef43169eb3872 · report
make_gym_env src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 25e26316ca1ded08 · report
probe_cube_far_decoupled src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 3ad9bab8b5188774 · report
probe_cube_goal_static src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 32fdce9d67977a6f · report
probe_cube_grip_gated_contact src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 17a5da2bc84b4e82 · report
probe_reacher_first_order_damping src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 5514ead367d0d241 · report
probe_reacher_joint_actuation src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 38b67be0819ba60c · report
probe_reacher_target_static src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 6373bf5e387130a9 · report
probe_tworoom_action_direction src/visualpatchworld/structure_induction.py unverified MIT (permissive) · ecf9ce5d1007d6d2 · report
probe_tworoom_hold_still src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 5733e0fabc40a424 · report
probe_tworoom_target_static src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 8b27f799573cc037 · report
run_structure_probes src/visualpatchworld/structure_induction.py unverified MIT (permissive) · c8e1b07d03eb3a6c · report
select_template_name src/visualpatchworld/structure_induction.py unverified MIT (permissive) · 10445722117444c9 · report
sim_row_from_obs src/visualpatchworld/structure_induction.py unverified MIT (permissive) · b770491d28e9dd15 · report

Method as code Syntology

What Syntology's graph records as this paper's claimed method, lifted into code: each assembly names its bricks (functions and classes), each with its own role and its own verification level. An assembly is not a check that the code implements the method correctly or reproduces the paper's results. Levels are each brick's own verification level in Syntology's graph (level 0: no recorded run), listed brick by brick and never combined into one level; “ran” means the brick executed on a synthesized input, not that its output is correct, and the instrument figure counts failures of Syntology's instrument, not of the code. Where a model read the code against the paper's claim, each assembly's label verdict is one model's judgement, kept in this page's JSON twin; it is not calibrated, so no card prints it.

1 assembly on this paper: 1 harvested, 0 generated.

harvested Assembly for VisualPatchWorld (graph method: VisualPatchWorld). 44 bricks; individually on a fixture: 23 ran (of which 7 constructed an object rather than computing a result; 15 with no instrument failure: 7 honoured, 0 violated, 8 with no contract checked; 8 where Syntology's instrument failed) · 21 unverified. The assembly has not been run end-to-end.

  1. induce_from_structure entry · level 0 · unverified · 23e0b564c40c423e
  2. EnvStructureReport helper · level 1 · ran · 3b3ac601d6efe942
  3. get_env_spec helper · level 0 · unverified · c0101a75a31bb147
  4. identify_env_structure model · level 0 · unverified · 714431e8d86d6708
  5. ProbeResult helper · level 1 · ran · 8c59c27d7893b0a2
  6. LeWMEnvSpec helper · level 1 · ran · c6ccd43fd3e2fbc1
  7. load_probe_rows helper · level 0 · unverified · 7d1ef43169eb3872
  8. run_structure_probes helper · level 0 · unverified · c8e1b07d03eb3a6c
  9. select_template_name helper · level 0 · unverified · 10445722117444c9
  10. structure_sketch_from_probes helper · level 1 · ran · our draft was wrong · f5b09b8e9f3a9e62
  11. _first_valid_start helper · level 1 · ran · honoured contract · b190ceebd9f6cad0
  12. goal_row_from_start helper · level 1 · ran · our draft was wrong · 10040f293d33680e
  13. require_h5py helper · level 1 · ran · a04c4f5412023862
  14. resolve_dataset_path helper · level 1 · ran · our draft was wrong · 119bc8041bb90077
  15. row_dict_from_h5 helper · level 1 · ran · our draft was wrong · f666f32a7d3abc1c
  16. Timeline helper · level 0 · unverified · 0c9a53a186fd8ffa
  17. make_gym_env helper · level 0 · unverified · 25e26316ca1ded08
  18. probe_cube_far_decoupled helper · level 0 · unverified · 3ad9bab8b5188774
  19. probe_cube_goal_static helper · level 0 · unverified · 32fdce9d67977a6f
  20. probe_cube_grip_gated_contact helper · level 0 · unverified · 17a5da2bc84b4e82
  21. probe_pusht_structure helper · level 1 · ran; the contract check could not run (DEP_MISSING) · e0b03e59ea78068d
  22. probe_reacher_first_order_damping helper · level 0 · unverified · 5514ead367d0d241
  23. probe_reacher_joint_actuation helper · level 0 · unverified · 38b67be0819ba60c
  24. probe_reacher_target_static helper · level 0 · unverified · 6373bf5e387130a9
  25. probe_tworoom_action_direction helper · level 0 · unverified · ecf9ce5d1007d6d2
  26. probe_tworoom_hold_still helper · level 0 · unverified · 5733e0fabc40a424
  27. probe_tworoom_target_static helper · level 0 · unverified · 8b27f799573cc037
  28. episode_slices helper · level 0 · unverified · 4a1f6f624b72e1f1
  29. TaggedTransition helper · level 1 · ran · 7dd0f6681042a092
  30. Trajectory helper · level 1 · ran · cec67a4407d01cf3
  31. Transition helper · level 1 · ran · ccc986c237c99cc1
  32. _agent_xy helper · level 1 · ran · honoured contract · 8eae1dea8284248c
  33. _cube_block_xy helper · level 1 · ran · honoured contract · 30c840c93b496349
  34. _macro_step helper · level 0 · unverified · 58202bf4e83babab
  35. _sync_row_after_reset helper · level 0 · unverified · ef695b35deb9f72b
  36. reset_env_from_rows helper · level 1 · ran · honoured contract · 74b8d14d662e95eb
  37. _target_xy helper · level 1 · ran · honoured contract · e1f7c095971a2c84
  38. _reacher_qvel helper · level 1 · ran · honoured contract · 2c071d1d8488c3db
  39. _reacher_qpos helper · level 1 · ran · honoured contract · dac6b3bc02a8cc8e
  40. EpisodeSlice helper · level 1 · ran · b973a7b79d96cfa9
  41. _as_int_list helper · level 1 · ran · our draft was wrong · e1995fb8c273c632
  42. sim_row_from_obs helper · level 0 · unverified · b770491d28e9dd15
  43. sim_step_with_info helper · level 1 · ran · our draft was wrong · 3e34eae5424d18e7
  44. _sim_obs_vector helper · level 1 · ran · our draft was wrong · ef44efed8670fb6c

Each brick ends with its code_sha256, Syntology's identity for that exact code; click one to copy the get_code call that returns it. For agents, on Syntology's MCP service: assembly(arxiv_id="2607.25236") returns these assemblies with each brick's role and wiring (how to connect).

Results from the paper

The Papers with Code archive ends with its 2025-07-28 snapshot. This paper's arXiv identifier, 2607.25236, was issued in July 2026, after that date, so the archive has no leaderboard rows for it.

Placed on leaderboards by Syntology Syntology

Syntology's extractor, the model Claude Sonnet 4.5, judged this paper's tables to report on 1 leaderboard (a model's judgement, not a result) and has not placed the paper on it: MuJoCo Games · Reacher (refused by rules: the row is an ablation variant, not the method itself; the table's column header does not name the leaderboard's metric). What is not shown.

Syntology has checked 8,886 of the 18,637 papers on this site that are newer than the archive (for 3,717 of them no archive leaderboard matched the paper's tables, so there was nothing further to check); 733 were read and have nothing to place (arXiv has no HTML version of the paper, or that version has no tables), 41 could not be read (the extractor's reply could not be parsed), and results from the other 8,977 appear after they are checked.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections