Papers › Effective Benchmarks for Optical Turbulence Modeling

Effective Benchmarks for Optical Turbulence Modeling

7 Jan 2024arXiv:2401.03573archive 2025-07-28

Christopher Jellen, Charles Nelson, Cody Brownell, John Burkhardt

Optical turbulence presents a significant challenge for communication, directed energy, and imaging systems, especially in the atmospheric boundary layer. Effective modeling of optical turbulence strength is critical for the development and deployment of these systems. The lack of standard evaluation tools, especially long-term data sets, modeling tasks, metrics, and baseline models, prevent effective comparisons between approaches and models. This reduces the ease of reproducing results and contributes to over-fitting on local micro-climates. Performance characterized using evaluation metrics provides some insight into the applicability of a model for predicting the strength of optical turbulence. However, these metrics are not sufficient for understanding the relative quality of a model. We introduce the \texttt{otbench} package, a Python package for rigorous development and evaluation of optical turbulence strength prediction models. The package provides a consistent interface for evaluating optical turbulence models on a variety of benchmark tasks and data sets. The \texttt{otbench} package includes a range of baseline models, including statistical, data-driven, and deep learning models, to provide a sense of relative model quality. \texttt{otbench} also provides support for adding new data sets, tasks, and evaluation metrics. The package is available at \url{https://github.com/cdjellen/otbench}.

PaperPDFCode

Code

cdjellen/otbench officialmentioned in paper report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Time Series ForecastingTime Series Regression

Datasets

Introduced by this paper, per the archive.

MLO-Cn2USNA-Cn2 (long-term)USNA-Cn2 (short-duration)

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Time Series Forecasting MLO-Cn2 GBRT RMSE 0.428 #1 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 Mean Window Forecast RMSE 0.481 #2 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 Minute Climatology RMSE 0.551 #3 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 RNN RMSE 0.581 #4 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 Climatology RMSE 0.658 #5 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 Linear Forecast RMSE 0.930 #6 of 7 Archive leaderboard report
Time Series Forecasting MLO-Cn2 Persistence RMSE 1.227 #7 of 7 Archive leaderboard report
Time Series Forecasting USNA-Cn2 (short-duration) GBRT RMSE 0.160 #1 of 5 Archive leaderboard report
Time Series Forecasting USNA-Cn2 (short-duration) Mean Window Forecast RMSE 0.182 #2 of 5 Archive leaderboard report
Time Series Forecasting USNA-Cn2 (short-duration) RNN RMSE 0.187 #3 of 5 Archive leaderboard report
Time Series Forecasting USNA-Cn2 (short-duration) Minute Climatology RMSE 0.453 #4 of 5 Archive leaderboard report
Time Series Forecasting USNA-Cn2 (short-duration) Persistence RMSE 0.821 #5 of 5 Archive leaderboard report
Time Series Regression MLO-Cn2 GBRT RMSE 0.212 #1 of 5 Archive leaderboard report
Time Series Regression MLO-Cn2 RNN RMSE 0.336 #2 of 5 Archive leaderboard report
Time Series Regression MLO-Cn2 Minute Climatology RMSE 0.504 #3 of 5 Archive leaderboard report
Time Series Regression MLO-Cn2 Climatology RMSE 0.661 #4 of 5 Archive leaderboard report
Time Series Regression MLO-Cn2 Persistence RMSE 1.209 #5 of 5 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Hybrid Air-Water Temperature Difference RMSE 0.458 #1 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) RNN RMSE 0.530 #2 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Minute Climatology RMSE 0.625 #3 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Climatology RMSE 0.632 #4 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Offshore Macro Meteorological RMSE 0.675 #5 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Air-Water Temperature Difference RMSE 1.046 #6 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Persistence RMSE 1.208 #7 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) Macro Meteorological RMSE 1.217 #8 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (long-term) GBRT RMSE 1.340 #9 of 9 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Offshore Macro Meteorological RMSE 0.178 #1 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) GBRT RMSE 0.299 #2 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Hybrid Air-Water Temperature Difference RMSE 0.303 #3 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Linear Forecast RMSE 0.358 #4 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) RNN RMSE 0.375 #5 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Minute Climatology RMSE 0.452 #6 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Climatology RMSE 0.480 #7 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Persistence RMSE 0.758 #8 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Macro Meteorological RMSE 0.864 #9 of 10 Archive leaderboard report
Time Series Regression USNA-Cn2 (short-duration) Air-Water Temperature Difference RMSE 0.910 #10 of 10 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Hybrid AWT

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections