Papers › Repeat and Concatenate: 2D to 3D Image Translation with 3D to 3D Generative Modeling

Repeat and Concatenate: 2D to 3D Image Translation with 3D to 3D Generative Modeling

26 Jun 2024arXiv:2406.18422archive 2025-07-28

Abril Corona-Figueroa, Hubert P. H. Shum, Chris G. Willcocks

This paper investigates a 2D to 3D image translation method with a straightforward technique, enabling correlated 2D X-ray to 3D CT-like reconstruction. We observe that existing approaches, which integrate information across multiple 2D views in the latent space, lose valuable signal information during latent encoding. Instead, we simply repeat and concatenate the 2D views into higher-channel 3D volumes and approach the 3D reconstruction challenge as a straightforward 3D to 3D generative modeling problem, sidestepping several complex modeling issues. This method enables the reconstructed 3D volume to retain valuable information from the 2D inputs, which are passed between channel states in a Swin UNETR backbone. Our approach applies neural optimal transport, which is fast and stable to train, effectively integrating signal information across multiple views without the requirement for precise alignment; it produces non-collapsed reconstructions that are highly faithful to the 2D views, even after limited training. We demonstrate correlated results, both qualitatively and quantitatively, having trained our model on a single dataset and evaluated its generalization ability across six datasets, including out-of-distribution samples.

PaperPDFCode

Code

abrilcf/3d-3d_repeat-concatenate officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Reconstruction

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 ConvolutionAttentionBatch NormalizationConcatenated Skip ConnectionConvolutionDense ConnectionsLinear LayerMax PoolingMulti-Head AttentionPosition-Wise Feed-Forward LayerReLUResidual ConnectionSoftmaxU-NetUNETR

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections