Papers › Supervised Multimodal Bitransformers for Classifying Images and Text

Supervised Multimodal Bitransformers for Classifying Images and Text

6 Sep 2019arXiv:1909.02950archive 2025-07-28

Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Ethan Perez, Davide Testuggine

Self-supervised bidirectional transformer models such as BERT have led to dramatic improvements in a wide variety of textual classification tasks. The modern digital world is increasingly multimodal, however, and textual information is often accompanied by other modalities such as images. We introduce a supervised multimodal bitransformer model that fuses information from text and image encoders, and obtain state-of-the-art performance on various multimodal classification benchmark tasks, outperforming strong baselines, including on hard test sets specifically designed to measure multimodal performance.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

facebookresearch/mmbt officialmentioned in papermentioned on GitHubpytorchNOASSERTION report
IsaacRodgz/mmbt_experiments mentioned on GitHubpytorchNOASSERTION report
IsaacRodgz/multimodal-transformers-movies mentioned on GitHubpytorchNOASSERTION report
ThilinaRajapakse/simpletransformers mentioned on GitHubpytorch report
adriangrepo/mmbt_lightning mentioned on GitHubpytorchNOASSERTION report
huggingface/transformers mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

General ClassificationNatural Language Inference

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Natural Language Inference V-SNLI MMBT Accuracy 90.5 #1 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position EncodingsAdamAttentionAttention DropoutBERTBPEDense ConnectionsDropoutLabel SmoothingLayer NormalizationLinear LayerLinear Warmup With Linear DecayMulti-Head AttentionPosition-Wise Feed-Forward LayerReLUResidual ConnectionSoftmaxTransformerWeight DecayWordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections