</> Code with Papers by Syntology

Papers › Analysis of NaN Divergence in Training Monocular Depth Estimation Model

Analysis of NaN Divergence in Training Monocular Depth Estimation Model

7 Nov 2023arXiv:2311.03938archive 2025-07-28

Bum Jun Kim, Hyeonah Jang, Sang Woo Kim

The latest advances in deep learning have facilitated the development of highly accurate monocular depth estimation models. However, when training a monocular depth estimation network, practitioners and researchers have observed not a number (NaN) loss, which disrupts gradient descent optimization. Although several practitioners have reported the stochastic and mysterious occurrence of NaN loss that bothers training, its root cause is not discussed in the literature. This study conducted an in-depth analysis of NaN loss during training a monocular depth estimation network and identified three types of vulnerabilities that cause NaN loss: 1) the use of square root loss, which leads to an unstable gradient; 2) the log-sigmoid function, which exhibits numerical stability issues; and 3) certain variance implementations, which yield incorrect computations. Furthermore, for each vulnerability, the occurrence of NaN loss was demonstrated and practical guidelines to prevent NaN loss were presented. Experiments showed that both optimization stability and performance on monocular depth estimation could be improved by following our guidelines.

PaperPDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Depth EstimationMonocular Depth Estimation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 Delta < 1.25 0.9757 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 Delta < 1.25^2 0.9974 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 Delta < 1.25^3 0.9994 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 RMSE 2.0373 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 RMSE log 0.077 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 Sq Rel 0.1458 #23 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split MIM-Swin-V2 absolute relative error 0.0508 #23 of 79 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 Delta < 1.25 0.9361 #33 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 Delta < 1.25^2 0.9916 #33 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 Delta < 1.25^3 0.9981 #33 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 RMSE 0.3046 #33 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 absolute relative error 0.0864 #33 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 MIM-Swin-V2 log 10 0.0365 #33 of 85 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections