Papers › Image-to-LaTeX Converter for Mathematical Formulas and Text

Image-to-LaTeX Converter for Mathematical Formulas and Text

7 Aug 2024arXiv:2408.04015archive 2025-07-28

Daniil Gurgurov, Aleksey Morshnev

In this project, we train a vision encoder-decoder model to generate LaTeX code from images of mathematical formulas and text. Utilizing a diverse collection of image-to-LaTeX data, we build two models: a base model with a Swin Transformer encoder and a GPT-2 decoder, trained on machine-generated images, and a fine-tuned version enhanced with Low-Rank Adaptation (LoRA) trained on handwritten formulas. We then compare the BLEU performance of our specialized model on a handwritten test set with other similar models, such as Pix2Text, TexTeller, and Sumen. Through this project, we contribute open-source models for converting images to LaTeX and provide from-scratch code for building these models with distributed training and GPU optimizations.

PaperPDFCode

Code

d-gurgurov/im2latex officialmentioned in papermentioned on GitHubpytorchMIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Decoder

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Absolute Position EncodingsAdamAttentionAttention DropoutBASEBPECosine AnnealingDense ConnectionsDiscriminative Fine-TuningDropoutGPT-2Label SmoothingLayer NormalizationLinear LayerLinear Warmup With Cosine AnnealingMulti-Head AttentionPosition-Wise Feed-Forward LayerResidual ConnectionSETSoftmaxStochastic DepthSwin TransformerTransformerWeight Decay

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections