Papers › VL-BERT: Pre-training of Generic Visual-Linguistic Representations

VL-BERT: Pre-training of Generic Visual-Linguistic Representations

22 Aug 2019ICLR 2020 1arXiv:1908.08530archive 2025-07-28

Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, Jifeng Dai

We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from the input image. It is designed to fit for most of the visual-linguistic downstream tasks. To better exploit the generic representation, we pre-train VL-BERT on the massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better align the visual-linguistic clues and benefit the downstream tasks, such as visual commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single model on the leaderboard of the VCR benchmark. Code is released at \url{https://github.com/jackroos/VL-BERT}.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1908.08530")

Code

Syntology Ran 0 of 1 code samples harvested from 1 repository linked to this paper; 1 has no recorded run.

By repository: official repository: 1 sample from 1 repository, 0 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

jackroos/VL-BERT officialmentioned in papermentioned on GitHubpytorchMIT report
ImperialNLP/BertGen mentioned on GitHubpytorch report
jules-samaran/vl-bert mentioned on GitHubpytorchMIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

1 sample harvested; 0 ran; 0 honoured the contract we drafted; 1 has no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

1unverified

Licence: 0 of the 1 sample are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from jackroos/VL-BERT. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

to_cuda jackroos/VL-BERT/common/trainer.py official repository unverified MIT (permissive) · a905f7851b56e39c · report

Tasks

Image-text matchingLanguage ModellingQuestion AnsweringReferring ExpressionReferring Expression ComprehensionSentenceVisual Commonsense ReasoningVisual Question AnsweringVisual Question Answering (VQA)

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image-text matching CommercialAdsDataset VL-BERT ADD(S) AUC 86.27 #5 of 8 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-A) dev VL-BERTLARGE Accuracy 75.5 #1 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-A) dev VL-BERTBASE Accuracy 73.8 #2 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-A) test VL-BERTLARGE Accuracy 75.8 #7 of 11 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-AR) dev VL-BERTLARGE Accuracy 58.9 #1 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-AR) dev VL-BERTBASE Accuracy 55.2 #2 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (Q-AR) test VL-BERTLARGE Accuracy 59.7 #5 of 7 Archive leaderboard report
Visual Question Answering (VQA) VCR (QA-R) dev VL-BERTLARGE Accuracy 77.9 #1 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (QA-R) dev VL-BERTBASE Accuracy 74.4 #2 of 3 Archive leaderboard report
Visual Question Answering (VQA) VCR (QA-R) test VL-BERTLARGE Accuracy 78.4 #6 of 8 Archive leaderboard report
Visual Question Answering (VQA) VQA v2 test-dev VL-BERTLARGE Accuracy 71.79 #24 of 56 Archive leaderboard report
Visual Question Answering (VQA) VQA v2 test-dev VL-BERTBASE Accuracy 71.16 #27 of 56 Archive leaderboard report
Visual Question Answering (VQA) VQA v2 test-std VL-BERTLARGE overall 72.2 #22 of 38 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: VL-BERT

Absolute Position EncodingsAdamAttentionAttention DropoutBERTBPEDense ConnectionsDropoutLabel SmoothingLayer NormalizationLinear LayerLinear Warmup With Linear DecayMulti-Head AttentionPosition-Wise Feed-Forward LayerReLUResidual ConnectionSoftmaxTransformerVL-BERTWeight DecayWordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections