Papers › Charting and navigating the space of solutions for recurrent neural networks

Charting and navigating the space of solutions for recurrent neural networks

17 Nov 2021NeurIPS 2021 12arXiv:2111.09356archive 2025-07-28

Elia Turner, Kabir Dabholkar, Omri Barak

Recurrent Neural Networks (RNNs) were recently successfully used to model the way neural activity drives task-related behavior in animals, operating under the implicit assumption that the obtained solutions are universal. Observations in both neuroscience and machine learning challenge this assumption. Animals can approach a given task with a variety of strategies, and training machine learning algorithms introduces the phenomenon of underspecification. These observations imply that every task is associated with a space of solutions. To date, the structure of this space is not understood, limiting the approach of comparing RNNs with neural data. Here, we characterize the space of solutions associated with various tasks. We first study a simple two-neuron network on a task that leads to multiple solutions. We trace the nature of the final solution back to the network's initial connectivity and identify discrete dynamical regimes that underlie this diversity. We then examine three neuroscience-inspired tasks: Delayed and interval discrimination, and Time reproduction. For each task, we find a rich set of solutions. Variability can be found directly in the neural activity of the networks, and additionally by testing the trained networks' ability to extrapolate, as a perturbation to a system often reveals hidden structure. Furthermore, we relate extrapolation patterns to specific dynamical objects and effective algorithms found by the networks. We introduce a tool to derive the reduced dynamics of networks by generating a compact directed graph describing the essence of the dynamics with regards to behavioral inputs and outputs. Using this representation, we can partition the solutions to each task into a handful of types and partially predict them from neural features. Our results shed light on the concept of the space of solutions and its uses in Machine learning and in Neuroscience.

PaperPDFConference PDFCodeCode Syntology ranMethod as code

In Syntology View this paper on Syntology, its page in Syntology's graph. That page adds which of its functions are also present in other papers' code Syntology holds, with dates, and its dated run record as JSON and a README badge. It also lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

Syntology 46 code samples harvested from 1 repository linked to this paper: 21 ran (of which 1 constructed an object rather than computing a result; 14 with no instrument failure: 8 honoured, 4 violated, 2 with no contract checked; 7 where Syntology's instrument failed) · 25 unverified. Where Syntology's instrument failed: 6 where our contract draft was wrong; 1 where the contract check could not run; failures of Syntology's instrument, not of the code.

By repository: official repository: 46 samples from 1 repository, 21 ran (14 with no instrument failure; 7 where Syntology's instrument failed). The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

Levels written on 2026-09-22 (UTC): the dates Syntology's graph wrote each sample's verification level, not when the code executed. 5 of the 21 samples that ran have no recorded date.

eliaturner/space-of-solutions-rnn officialmentioned in paperpytorchSyntology: 46 samples, 21 ran (7 instrument-failed) · 25 unverified report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

46 samples harvested: 21 ran (of which 1 constructed an object rather than computing a result; 14 with no instrument failure: 8 honoured, 4 violated, 2 with no contract checked; 7 where Syntology's instrument failed) · 25 unverified. “Where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Syntology's record for this page has not changed since , the first build that kept a record date for it; when this build read Syntology's graph is in the build record. That date is not when the samples ran.

8ran · honoured contract
4ran · violated contract
6ran · our draft was wrong
1ran; the contract check could not run
2ran
25unverified

Licence: 46 of the 46 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from eliaturner/space-of-solutions-rnn. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. A line beginning “fixture:” under a sample that ran gives the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right. “Level recorded” gives the UTC date when Syntology's graph wrote the sample's verification level, not when the code executed; where the level was written before Syntology's graph kept that date, the row says “level recorded, date not recorded”. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself. “Our draft was wrong” and “the contract check could not run” are failures of Syntology's instrument, not of the code: the sample ran on Syntology's synthesized input, and then the drafted contract did not describe it, or the check itself raised an error (RAISES), lacked a dependency (DEP_MISSING) or ran out of time (TIMEOUT); hover the label for which. A metamorphic tier, where shown, is evidence of the output's shape, not that the sample implements the method.

Each sample ends with its code_sha256. A sample's code_sha256 is Syntology's identity for that exact code. It is printed in full (16 hexadecimal characters for every sample in this build), and get_code(code_sha256="…") takes it exactly as printed. An agent calls it on Syntology's MCP service (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples are grouped by repository: a repository marked official first, then by how directly the repository is tied to the paper, then in the archive's order; never by how many ran. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of six reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar); this paper's copy was not located, so its licence is not recorded; or this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not. Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

A function or class name links to its page when the name occurs in the harvested code of at least 5 papers; the name is a string, not an identity.

eliaturner/space-of-solutions-rnn official repository 46 samples, 21 ran (7 instrument-failed) · 25 unverified
DynamicsToGraphAbstract analysis/dynamics_to_graph.py ran no licence file found · pointer only · faccc1c689a2ed0e · report fixture: returned DynamicsToGraphAbstract · level recorded 2026-09-22
calc_normalized_q_value analysis/dynamics_to_graph.py ran · violated contract fingerprinted no licence file found · pointer only · 25dff3a0ce8e52c6 · report fixture: signature states:ndarray/3/float · returned ndarray · level recorded 2026-09-22
calculate_merge_index analysis/dynamics_to_graph.py ran · violated contract fingerprinted no licence file found · pointer only · aec9430f7428319d · report fixture: signature mat:ndarray/2/float · returned tuple · level recorded 2026-09-22
check_condition_zero_edge analysis/dynamics_to_graph.py ran · violated contract no licence file found · pointer only · b2b0352f37d9319d · report fixture: signature G:DiGraph/0/?, s:str/0/str, t:str/0/str · returned bool · level recorded 2026-09-22
clip_dynamics analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · caaa11a04fbc5814 · report fixture: signature dynamics:ndarray/2/float, outputs:ndarray/2/float · returned tuple · level recorded 2026-09-22
diagonal_index_to_matrix_index analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · df12b0b8f73f171f · report fixture: signature location:int/0/int, k:int/0/int · returned tuple · level recorded 2026-09-22
find_isomorphic_classes find_solution_types.py ran · our draft was wrong no licence file found · pointer only · 001ff5f383a4130c · report fixture: signature graph_list:list/1/DiGraph · returned tuple · level recorded, date not recorded
flatten_list in 30 papers predict_class_from_training_features.py ran · violated contract no licence file found · pointer only · 615bb1ff169f4a10 · report fixture: signature l:list/1/tuple · returned ndarray · level recorded, date not recorded
get_max_peak analysis/dynamics_to_graph.py ran; the contract check could not run (RAISES) · metamorphic tier: invariant fingerprinted no licence file found · pointer only · 62d98cd9762d8b0c · report fixture: signature array:ndarray/1/float · returned int64 · level recorded 2026-09-22
graphs_to_labels find_solution_types.py ran · honoured contract no licence file found · pointer only · 93f7c97198955ef7 · report fixture: signature class_to_list:dict/0/?, size:int/0/int · returned ndarray · level recorded, date not recorded
load_pickle in 115 papers analysis/dynamics_to_graph.py ran · our draft was wrong no licence file found · pointer only · 96a5aa0968aad06b · report fixture: signature file_name:str/0/str · returned dict · level recorded 2026-09-22
max_bottom_left_diagonal analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · f9445822175bc7b9 · report fixture: signature mat:Tensor/2/float · returned Tensor · level recorded 2026-09-22
merge_edges analysis/dynamics_to_graph.py ran · our draft was wrong no licence file found · pointer only · 93cc45cabb504e93 · report fixture: signature G:DiGraph/0/? · returned DiGraph · level recorded 2026-09-22
merge_path_to_edge analysis/dynamics_to_graph.py ran · our draft was wrong no licence file found · pointer only · 73b031ebc624843d · report fixture: returned DiGraph · level recorded 2026-09-22
merge_square analysis/dynamics_to_graph.py ran no licence file found · pointer only · a4d852cc0fe10648 · report fixture: returned DiGraph · level recorded 2026-09-22
merge_zero_one_edges analysis/dynamics_to_graph.py ran · our draft was wrong no licence file found · pointer only · 76e2a3048ddb706e · report fixture: signature G:DiGraph/0/? · returned DiGraph · level recorded 2026-09-22
return_dict_sorted_by_size find_solution_types.py ran · our draft was wrong no licence file found · pointer only · 732dcfc3729281c6 · report fixture: signature d:dict/0/? · returned dict · level recorded, date not recorded
scale in 38 papers predict_class_from_training_features.py ran · honoured contract fingerprinted no licence file found · pointer only · b62a3520a7e372d0 · report fixture: signature X:ndarray/2/float · returned ndarray · level recorded, date not recorded
speed_vector analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · 80645396bf42b8e6 · report fixture: signature vec:ndarray/2/float · returned ndarray · level recorded 2026-09-22
states_distance_matrix analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · c005caf129e972e0 · report fixture: signature a:ndarray/2/float, b:ndarray/2/float · returned ndarray · level recorded 2026-09-22
svm_if_separable analysis/dynamics_to_graph.py ran · honoured contract fingerprinted no licence file found · pointer only · 5c8ef092b4630ff8 · report fixture: signature points1:ndarray/2/float, points2:ndarray/2/float · returned float64 · level recorded 2026-09-22
Analyzer analysis/dynamics_to_graph.py unverified no licence file found · pointer only · da28a85f859ed730 · report
check_condition_mixed_edge analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 4f2ff1b68a2d77a9 · report
check_condition_nodes analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 45c3a4640f99a73a · report
check_condition_parallel_nodes analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 0a38342bb02c942f · report
check_square_condition analysis/dynamics_to_graph.py unverified no licence file found · pointer only · acbe66815fb07933 · report
check_square_condition1 analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 27a61359fe068a73 · report
check_square_condition2 analysis/dynamics_to_graph.py unverified no licence file found · pointer only · ba8644dc8e550e55 · report
connect_cycle analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 23d8262dc927f1eb · report
connect_lace_to_graph analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 28285443ad039dbd · report
contract_nodes analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 595eb81aacd577ec · report
dump_pickle analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 8cdf4e586d5d62e7 · report
edge_to_node analysis/dynamics_to_graph.py unverified no licence file found · pointer only · a21a4d05dd51c431 · report
get_merging_location analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 40e93d32b8f92389 · report
get_path_from_source analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 82a7792ae91bb0d1 · report
get_paths in 32 papers analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 65f3109a43521871 · report
get_zero_paths analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 457ecfefaa97e3bb · report
is_node_uninformative analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 35768b563789ac19 · report
make_compact analysis/dynamics_to_graph.py unverified no licence file found · pointer only · d9e98cf7dab8b4c8 · report
merge_cycle_to_graph analysis/dynamics_to_graph.py unverified no licence file found · pointer only · d2126a6a6bab8b28 · report
merge_lateral_nodes analysis/dynamics_to_graph.py unverified no licence file found · pointer only · f036f25b71b02505 · report
merge_two_cycles analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 93523148bf1dd544 · report
modified_cycle_finder analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 58516f1e4a6763b1 · report
redirect_output_routes analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 02127c61512197da · report
remove_uninformative_paths analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 774f3a2dbcc9fdf3 · report
try_merge_dynamics_to_graph analysis/dynamics_to_graph.py unverified no licence file found · pointer only · 1b7b69f6648f93d1 · report

Method as code Syntology

What Syntology's graph records as this paper's claimed method, lifted into code: each assembly names its bricks (functions and classes), each with its own role and its own verification level. An assembly is not a check that the code implements the method correctly or reproduces the paper's results. Levels are each brick's own verification level in Syntology's graph (level 0: no recorded run), listed brick by brick and never combined into one level; “ran” means the brick executed on a synthesized input, not that its output is correct, and the instrument figure counts failures of Syntology's instrument, not of the code. Where a model read the code against the paper's claim, each assembly's label verdict is one model's judgement, kept in this page's JSON twin; it is not calibrated, so no card prints it.

1 assembly on this paper: 1 harvested, 0 generated.

harvested Assembly for a tool to derive the reduced dynamics of networks (graph method: a tool to derive the reduced dynamics of networks). 41 bricks; individually on a fixture: 16 ran (of which 1 constructed an object rather than computing a result; 11 with no instrument failure: 6 honoured, 3 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 25 unverified. The assembly has not been run end-to-end.

  1. DynamicsToGraphAbstract entry · level 1 · ran · faccc1c689a2ed0e
  2. Analyzer helper · level 0 · unverified · da28a85f859ed730
  3. clip_dynamics helper · level 1 · ran · honoured contract · caaa11a04fbc5814
  4. connect_cycle helper · level 0 · unverified · 23d8262dc927f1eb
  5. connect_lace_to_graph helper · level 0 · unverified · 28285443ad039dbd
  6. make_compact helper · level 0 · unverified · d9e98cf7dab8b4c8
  7. modified_cycle_finder helper · level 0 · unverified · 58516f1e4a6763b1
  8. try_merge_dynamics_to_graph helper · level 0 · unverified · 1b7b69f6648f93d1
  9. dump_pickle helper · level 0 · unverified · 8cdf4e586d5d62e7
  10. load_pickle helper · level 1 · ran · our draft was wrong · 96a5aa0968aad06b
  11. calc_normalized_q_value helper · level 1 · ran · violated contract · 25dff3a0ce8e52c6
  12. merge_edges helper · level 1 · ran · our draft was wrong · 93cc45cabb504e93
  13. merge_lateral_nodes helper · level 0 · unverified · f036f25b71b02505
  14. merge_square helper · level 1 · ran · a4d852cc0fe10648
  15. merge_zero_one_edges helper · level 1 · ran · our draft was wrong · 76e2a3048ddb706e
  16. redirect_output_routes helper · level 0 · unverified · 02127c61512197da
  17. remove_uninformative_paths helper · level 0 · unverified · 774f3a2dbcc9fdf3
  18. get_max_peak helper · level 1 · ran; the contract check could not run (RAISES) · 62d98cd9762d8b0c
  19. get_merging_location helper · level 0 · unverified · 40e93d32b8f92389
  20. get_path_from_source helper · level 0 · unverified · 82a7792ae91bb0d1
  21. merge_cycle_to_graph helper · level 0 · unverified · d2126a6a6bab8b28
  22. check_condition_zero_edge helper · level 1 · ran · violated contract · b2b0352f37d9319d
  23. edge_to_node helper · level 0 · unverified · a21a4d05dd51c431
  24. check_condition_parallel_nodes helper · level 0 · unverified · 0a38342bb02c942f
  25. contract_nodes helper · level 0 · unverified · 595eb81aacd577ec
  26. check_square_condition helper · level 0 · unverified · acbe66815fb07933
  27. check_condition_mixed_edge helper · level 0 · unverified · 4f2ff1b68a2d77a9
  28. get_paths helper · level 0 · unverified · 65f3109a43521871
  29. is_node_uninformative helper · level 0 · unverified · 35768b563789ac19
  30. merge_path_to_edge helper · level 1 · ran · our draft was wrong · 73b031ebc624843d
  31. calculate_merge_index helper · level 1 · ran · violated contract · aec9430f7428319d
  32. get_zero_paths helper · level 0 · unverified · 457ecfefaa97e3bb
  33. max_bottom_left_diagonal helper · level 1 · ran · honoured contract · f9445822175bc7b9
  34. states_distance_matrix helper · level 1 · ran · honoured contract · c005caf129e972e0
  35. merge_two_cycles helper · level 0 · unverified · 93523148bf1dd544
  36. svm_if_separable helper · level 1 · ran · honoured contract · 5c8ef092b4630ff8
  37. check_condition_nodes helper · level 0 · unverified · 45c3a4640f99a73a
  38. check_square_condition1 helper · level 0 · unverified · 27a61359fe068a73
  39. check_square_condition2 helper · level 0 · unverified · ba8644dc8e550e55
  40. diagonal_index_to_matrix_index helper · level 1 · ran · honoured contract · df12b0b8f73f171f
  41. speed_vector helper · level 1 · ran · honoured contract · 80645396bf42b8e6

Each brick ends with its code_sha256, Syntology's identity for that exact code; click one to copy the get_code call that returns it. For agents, on Syntology's MCP service: assembly(arxiv_id="2111.09356") returns these assemblies with each brick's role and wiring (how to connect).

Tasks

BIG-bench Machine Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections