Papers › FoundationStereo: Zero-Shot Stereo Matching

FoundationStereo: Zero-Shot Stereo Matching

17 Jan 2025CVPR 2025 1arXiv:2501.09898archive 2025-07-28

Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield

Tremendous progress has been made in deep stereo matching to excel on benchmark datasets through per-domain fine-tuning. However, achieving strong zero-shot generalization - a hallmark of foundation models in other computer vision tasks - remains challenging for stereo matching. We introduce FoundationStereo, a foundation model for stereo depth estimation designed to achieve strong zero-shot generalization. To this end, we first construct a large-scale (1M stereo pairs) synthetic training dataset featuring large diversity and high photorealism, followed by an automatic self-curation pipeline to remove ambiguous samples. We then design a number of network architecture components to enhance scalability, including a side-tuning feature backbone that adapts rich monocular priors from vision foundation models to mitigate the sim-to-real gap, and long-range context reasoning for effective cost volume filtering. Together, these components lead to strong robustness and accuracy across domains, establishing a new standard in zero-shot stereo depth estimation. Project page: https://nvlabs.github.io/FoundationStereo/

PaperPDFConference PDFCodeCode Syntology ranMethod as code

In Syntology View this paper on Syntology, its page in Syntology's graph. That page adds which of its functions are also present in other papers' code Syntology holds, with dates, and its dated run record as JSON and a README badge. It also lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

Syntology 46 code samples harvested from 2 repositories linked to this paper: 36 ran (of which 23 constructed an object rather than computing a result; 26 with no instrument failure: 1 honoured, 1 violated, 24 with no contract checked; 10 where Syntology's instrument failed) · 10 unverified. Where Syntology's instrument failed: 2 where our contract draft was wrong; 8 where the contract check could not run; failures of Syntology's instrument, not of the code.

By repository: official repository: 42 samples from 1 repository, 35 ran (25 with no instrument failure; 10 where Syntology's instrument failed); community (archive-listed): 4 samples from 1 repository, 1 ran (1 with no instrument failure; 0 where Syntology's instrument failed). The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

Levels written between 2026-09-22 and 2026-09-24 (UTC): the dates Syntology's graph wrote each sample's verification level, not when the code executed.

NVlabs/FoundationStereo officialmentioned on GitHubpytorchSyntology: 42 samples, 35 ran (10 instrument-failed) · 7 unverified report
xiandaguo/openstereo mentioned on GitHubpytorchSyntology: 4 samples, 1 ran (0 instrument-failed) · 3 unverified report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

46 samples harvested: 36 ran (of which 23 constructed an object rather than computing a result; 26 with no instrument failure: 1 honoured, 1 violated, 24 with no contract checked; 10 where Syntology's instrument failed) · 10 unverified. “Where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Syntology's record for this page has not changed since , the first build that kept a record date for it; when this build read Syntology's graph is in the build record. That date is not when the samples ran.

1ran · honoured contract
1ran · violated contract
2ran · our draft was wrong
8ran; the contract check could not run
24ran
10unverified

Licence: 46 of the 46 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 2 repositories linked to this paper, official or community; the list below is grouped by repository, each group headed by its name and what it is to the paper. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. A line beginning “fixture:” under a sample that ran gives the arguments Syntology's fixture called the sample with, each as name:type/rank/element kind (? where none was recorded), and the type of what the fixture's run returned, exactly as the harness recorded them; the arguments and the returned type were recorded by two separate runs of the same fixture. No output value is shown, and a returned type says nothing about whether the output was right. “Level recorded” gives the UTC date when Syntology's graph wrote the sample's verification level, not when the code executed; where the level was written before Syntology's graph kept that date, the row says “level recorded, date not recorded”. For 12 samples that ran, the graph records a signature this page does not show: for a class the fixture built without calling, the harness called the object with inputs it made up itself, so the recorded signature is the harness's probe, not the fixture's arguments. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself. “Our draft was wrong” and “the contract check could not run” are failures of Syntology's instrument, not of the code: the sample ran on Syntology's synthesized input, and then the drafted contract did not describe it, or the check itself raised an error (RAISES), lacked a dependency (DEP_MISSING) or ran out of time (TIMEOUT); hover the label for which. A metamorphic tier, where shown, is evidence of the output's shape, not that the sample implements the method.

Each sample ends with its code_sha256. A sample's code_sha256 is Syntology's identity for that exact code. It is printed in full (16 hexadecimal characters for every sample in this build), and get_code(code_sha256="…") takes it exactly as printed. An agent calls it on Syntology's MCP service (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples are grouped by repository: a repository marked official first, then by how directly the repository is tied to the paper, then in the archive's order; never by how many ran. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of six reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar); this paper's copy was not located, so its licence is not recorded; or this paper's copy is cleared, but the stored copy of the same code that Syntology serves is not. Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

A function or class name links to its page when the name occurs in the harvested code of at least 5 papers; the name is a string, not an identity. A census tag appears when Syntology's census ran implementations with that name on one shared input per bucket: it gives how many were compared, from how many papers, and how many distinct outputs they produced, and names this sample's bucket when this sample was one of them.

NVlabs/FoundationStereo official repository 42 samples, 35 ran (10 instrument-failed) · 7 unverified
BasicConv in 14 papers core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · e3f111e6ae0ed124 · report fixture: returned BasicConv · level recorded 2026-09-22
BasicConv_IN core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · ddabfd0e685bdb50 · report fixture: returned BasicConv_IN · level recorded 2026-09-22
BasicMotionEncoder in 10 papers core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 1a491b3a069a6863 · report fixture: returned BasicMotionEncoder · level recorded 2026-09-22
ChannelAttentionEnhancement core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · d124cf44c00a85ca · report fixture: returned ChannelAttentionEnhancement · level recorded 2026-09-22
Combined_Geo_Encoding_Volume core/foundation_stereo.py ran licence not identified · pointer only · d27f188e59ba3f19 · report fixture: returned Combined_Geo_Encoding_Volume · level recorded 2026-09-22
Conv2x core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · dc9531dcc0ac5e5a · report fixture: returned Conv2x · level recorded 2026-09-22
Conv2x_IN core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · a9dafb790655568b · report fixture: returned Conv2x_IN · level recorded 2026-09-22
Conv3dNormActReduced core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 24e41de38cdd0713 · report fixture: returned Conv3dNormActReduced · level recorded 2026-09-22
CostVolumeDisparityAttention core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 71ecd3c91f81ba88 · report fixture: returned CostVolumeDisparityAttention · level recorded 2026-09-22
DispHead core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 6952b269b7e1b933 · report fixture: returned DispHead · level recorded 2026-09-22
EdgeNextConvEncoder core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · b275671235adca7e · report fixture: returned EdgeNextConvEncoder · level recorded 2026-09-22
FeatureAtt core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 64763e7a4b5dc4b0 · report fixture: returned FeatureAtt · level recorded 2026-09-22
FlashAttentionTransformerEncoderLayer core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 2f4d923b48603a14 · report fixture: returned FlashAttentionTransformerEncoderLayer · level recorded 2026-09-22
FlashMultiheadAttention core/foundation_stereo.py ran licence not identified · pointer only · 3b4f608f12ce8154 · report fixture: signature query:Tensor/3/float, key:Tensor/3/float, value:Tensor/3/float, attn_mask:NoneType/0/?, window_size:tuple/1/int · returned Tensor · level recorded 2026-09-24
InputPadder core/foundation_stereo.py ran licence not identified · pointer only · 7c2c552f07055a81 · report fixture: returned InputPadder · level recorded 2026-09-22
LayerNorm2d in 18 papers core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · 5cadecd58430ba24 · report fixture: returned LayerNorm2d · level recorded 2026-09-22
MultiBasicEncoder core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 0df4b279744da77a · report fixture: returned MultiBasicEncoder · level recorded 2026-09-22
PositionalEmbedding in 43 papers
census: 6 implementations from 6 papers, 4 distinct outputs across 3 buckets, one shared input per bucket
core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · c70a8bfd36e2875a · report fixture: returned PositionalEmbedding · level recorded 2026-09-22
ResidualBlock in 100 papers core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 338d3118e7677358 · report fixture: returned ResidualBlock · level recorded 2026-09-22
ResnetBasicBlock core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · f9b07433177efbd6 · report fixture: returned ResnetBasicBlock · level recorded 2026-09-22
ResnetBasicBlock3D core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 8c2695d5c3cdf485 · report fixture: returned ResnetBasicBlock3D · level recorded 2026-09-22
SelectiveConvGRU core/foundation_stereo.py ran · metamorphic tier: deterministic licence not identified · pointer only · 53375bb95d131457 · report fixture: returned SelectiveConvGRU · level recorded 2026-09-22
SpatialAttentionExtractor core/foundation_stereo.py ran · metamorphic tier: deterministic fingerprinted licence not identified · pointer only · b15de3eac28b8aa9 · report fixture: returned SpatialAttentionExtractor · level recorded 2026-09-22
_is_contiguous core/foundation_stereo.py ran · violated contract fingerprinted licence not identified · pointer only · 97a5bc16c7b55659 · report fixture: signature tensor:Tensor/2/float · returned bool · level recorded 2026-09-22
bilinear_sampler in 23 papers core/foundation_stereo.py ran · our draft was wrong fingerprinted licence not identified · pointer only · 100f55ad0ce6efe9 · report fixture: signature img:Tensor/4/float, coords:Tensor/4/float, mode:str/0/str, mask:bool/0/bool, low_memory:bool/0/bool · returned Tensor · level recorded 2026-09-22
build_concat_volume core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · c7a66c78764c1b17 · report fixture: signature refimg_fea:Tensor/4/float, targetimg_fea:Tensor/4/float, maxdisp:int/0/int · returned Tensor · level recorded 2026-09-22
build_gwc_volume core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · 0fa9626ac5b73f03 · report fixture: signature refimg_fea:Tensor/4/float, targetimg_fea:Tensor/4/float, maxdisp:int/0/int, num_groups:int/0/int, stride:int/0/int · returned Tensor · level recorded 2026-09-22
context_upsample core/foundation_stereo.py ran; the contract check could not run (RAISES) licence not identified · pointer only · 0122ace55bf73208 · report fixture: signature disp_low:Tensor/4/float, up_weights:Tensor/4/float · returned Tensor · level recorded 2026-09-22
disparity_regression in 6 papers core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · 8faecdb9ac9e6cb1 · report fixture: signature x:Tensor/4/float, maxdisp:int/0/int · returned Tensor · level recorded 2026-09-22
freeze_model in 12 papers core/foundation_stereo.py ran · our draft was wrong licence not identified · pointer only · 566179b5eb4285ba · report fixture: signature model:callable/0/? · returned SimpleModel · level recorded 2026-09-22
get_resize_keep_aspect_ratio core/foundation_stereo.py ran · honoured contract fingerprinted licence not identified · pointer only · 76c8886ff1321a65 · report fixture: signature H:int/0/int, W:int/0/int, divider:int/0/int, max_H:int/0/int, max_W:int/0/int · returned tuple · level recorded 2026-09-22
groupwise_correlation in 7 papers core/foundation_stereo.py ran; the contract check could not run (RAISES) licence not identified · pointer only · 9c850a02a2ecd21e · report fixture: signature fea1:Tensor/4/float, fea2:Tensor/4/float, num_groups:int/0/int · returned Tensor · level recorded 2026-09-22
interp in 22 papers core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · 08bedbc0c0447f5d · report fixture: signature x:Tensor/4/float, dest:Tensor/4/float · returned Tensor · level recorded 2026-09-22
normalize_image in 28 papers core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · 1c5a401752019e06 · report fixture: signature img:Tensor/4/float · returned Tensor · level recorded 2026-09-22
pool2x in 8 papers core/foundation_stereo.py ran; the contract check could not run (RAISES) fingerprinted licence not identified · pointer only · 25894ad2a73a8142 · report fixture: signature x:Tensor/4/float · returned Tensor · level recorded 2026-09-22
BasicSelectiveMultiUpdateBlock core/foundation_stereo.py unverified licence not identified · pointer only · d1835da015616021 · report
ContextNetDino core/foundation_stereo.py unverified licence not identified · pointer only · b52511bd3dbf9dd0 · report
DepthAnythingFeature core/foundation_stereo.py unverified licence not identified · pointer only · 6cdaebf982b3607c · report
Feature core/foundation_stereo.py unverified licence not identified · pointer only · 795265c536b80ad1 · report
FoundationStereo core/foundation_stereo.py unverified licence not identified · pointer only · a9e5da2f1e457d33 · report
RaftConvGRU core/foundation_stereo.py unverified licence not identified · pointer only · 8b79f67c9839926d · report
hourglass in 7 papers core/foundation_stereo.py unverified licence not identified · pointer only · 9a2d5c3211b11558 · report
xiandaguo/openstereo community (archive-listed) 4 samples, 1 ran (0 instrument-failed) · 3 unverified
Combined_Geo_Encoding_Volume stereo/modeling/models/foundationstereo/core/foundation_stereo.py ran no licence file found · pointer only · ab36937264210e6a · report fixture: returned Combined_Geo_Encoding_Volume · level recorded 2026-09-22
FoundationStereo stereo/modeling/models/foundationstereo/core/foundation_stereo.py unverified no licence file found · pointer only · ebdb7f80d2a21131 · report
bilinear_sampler in 23 papers stereo/modeling/models/foundationstereo/core/foundation_stereo.py unverified no licence file found · pointer only · 90fc17b50a12390d · report
hourglass in 7 papers stereo/modeling/models/foundationstereo/core/foundation_stereo.py unverified no licence file found · pointer only · fe3bd1e7fff7cb5c · report

Method as code Syntology

What Syntology's graph records as this paper's claimed method, lifted into code: each assembly names its bricks (functions and classes), each with its own role and its own verification level. An assembly is not a check that the code implements the method correctly or reproduces the paper's results. Levels are each brick's own verification level in Syntology's graph (level 0: no recorded run), listed brick by brick and never combined into one level; “ran” means the brick executed on a synthesized input, not that its output is correct, and the instrument figure counts failures of Syntology's instrument, not of the code. Where a model read the code against the paper's claim, each assembly's label verdict is one model's judgement, kept in this page's JSON twin; it is not calibrated, so no card prints it. Siblings are assemblies lifted from this paper in the same batch for the same method: alternative readings kept side by side; none replaces or supersedes another.

2 assemblies on this paper: 2 harvested, 0 generated, listed in 1 groups (siblings lifted in one batch form one group).

2 sibling assemblies for FoundationStereo (harvested), lifted in the same batch on 2026-09-09: alternative readings kept side by side; none replaces or supersedes another.

harvested Assembly for FoundationStereo (graph method: FoundationStereo). 42 bricks; individually on a fixture: 35 ran (of which 22 constructed an object rather than computing a result; 25 with no instrument failure: 1 honoured, 1 violated, 23 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified. The assembly has not been run end-to-end.

one of 2 siblings

  1. FoundationStereo entry · level 0 · unverified · a9e5da2f1e457d33
  2. BasicConv helper · level 1 · ran · e3f111e6ae0ed124
  3. BasicConv_IN helper · level 1 · ran · ddabfd0e685bdb50
  4. BasicSelectiveMultiUpdateBlock helper · level 0 · unverified · d1835da015616021
  5. ChannelAttentionEnhancement helper · level 1 · ran · d124cf44c00a85ca
  6. Combined_Geo_Encoding_Volume helper · level 1 · ran · d27f188e59ba3f19
  7. ContextNetDino helper · level 0 · unverified · b52511bd3dbf9dd0
  8. Conv2x helper · level 1 · ran · dc9531dcc0ac5e5a
  9. Feature helper · level 0 · unverified · 795265c536b80ad1
  10. FeatureAtt helper · level 1 · ran · 64763e7a4b5dc4b0
  11. InputPadder helper · level 1 · ran · 7c2c552f07055a81
  12. ResnetBasicBlock3D helper · level 1 · ran · 8c2695d5c3cdf485
  13. SpatialAttentionExtractor helper · level 1 · ran · b15de3eac28b8aa9
  14. build_concat_volume helper · level 1 · ran; the contract check could not run (RAISES) · c7a66c78764c1b17
  15. build_gwc_volume helper · level 1 · ran; the contract check could not run (RAISES) · 0fa9626ac5b73f03
  16. context_upsample helper · level 1 · ran; the contract check could not run (RAISES) · 0122ace55bf73208
  17. disparity_regression helper · level 1 · ran; the contract check could not run (RAISES) · 8faecdb9ac9e6cb1
  18. hourglass model · level 0 · unverified · 9a2d5c3211b11558
  19. normalize_image data_transform · level 1 · ran; the contract check could not run (RAISES) · 1c5a401752019e06
  20. BasicMotionEncoder helper · level 1 · ran · 1a491b3a069a6863
  21. DispHead helper · level 1 · ran · 6952b269b7e1b933
  22. SelectiveConvGRU helper · level 1 · ran · 53375bb95d131457
  23. interp helper · level 1 · ran; the contract check could not run (RAISES) · 08bedbc0c0447f5d
  24. pool2x helper · level 1 · ran; the contract check could not run (RAISES) · 25894ad2a73a8142
  25. bilinear_sampler helper · level 1 · ran · our draft was wrong · 100f55ad0ce6efe9
  26. DepthAnythingFeature helper · level 0 · unverified · 6cdaebf982b3607c
  27. LayerNorm2d helper · level 1 · ran · 5cadecd58430ba24
  28. MultiBasicEncoder helper · level 1 · ran · 0df4b279744da77a
  29. ResidualBlock helper · level 1 · ran · 338d3118e7677358
  30. get_resize_keep_aspect_ratio helper · level 1 · ran · honoured contract · 76c8886ff1321a65
  31. Conv2x_IN helper · level 1 · ran · a9dafb790655568b
  32. freeze_model helper · level 1 · ran · our draft was wrong · 566179b5eb4285ba
  33. groupwise_correlation helper · level 1 · ran; the contract check could not run (RAISES) · 9c850a02a2ecd21e
  34. Conv3dNormActReduced helper · level 1 · ran · 24e41de38cdd0713
  35. CostVolumeDisparityAttention helper · level 1 · ran · 71ecd3c91f81ba88
  36. EdgeNextConvEncoder helper · level 1 · ran · b275671235adca7e
  37. RaftConvGRU helper · level 0 · unverified · 8b79f67c9839926d
  38. _is_contiguous helper · level 1 · ran · violated contract · 97a5bc16c7b55659
  39. ResnetBasicBlock helper · level 1 · ran · f9b07433177efbd6
  40. FlashAttentionTransformerEncoderLayer helper · level 1 · ran · 2f4d923b48603a14
  41. PositionalEmbedding helper · level 1 · ran · c70a8bfd36e2875a
  42. FlashMultiheadAttention helper · level 1 · ran · 3b4f608f12ce8154

harvested Assembly for FoundationStereo (graph method: FoundationStereo). 5 bricks; individually on a fixture: 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified; every one of the 2 samples that ran constructed an object rather than computing a result. The assembly has not been run end-to-end.

one of 2 siblings

  1. FoundationStereo entry · level 0 · unverified · ebdb7f80d2a21131
  2. Combined_Geo_Encoding_Volume helper · level 1 · ran · ab36937264210e6a
  3. InputPadder helper · level 1 · ran · 7c2c552f07055a81
  4. hourglass model · level 0 · unverified · fe3bd1e7fff7cb5c
  5. bilinear_sampler helper · level 0 · unverified · 90fc17b50a12390d

Each brick ends with its code_sha256, Syntology's identity for that exact code; click one to copy the get_code call that returns it. For agents, on Syntology's MCP service: assembly(arxiv_id="2501.09898") returns these assemblies with each brick's role and wiring (how to connect).

Tasks

Depth EstimationDiversityStereo Depth EstimationStereo MatchingZero-shot Generalization

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections