Papers › IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on...

IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages

25 Apr 2024arXiv:2404.16816archive 2025-07-28

Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, Partha Talukdar

As large language models (LLMs) see increasing adoption across the globe, it is imperative for LLMs to be representative of the linguistic diversity of the world. India is a linguistically diverse country of 1.4 Billion people. To facilitate research on multilingual LLM evaluation, we release IndicGenBench - the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set 29 of Indic languages covering 13 scripts and 4 language families. IndicGenBench is composed of diverse generation tasks like cross-lingual summarization, machine translation, and cross-lingual question answering. IndicGenBench extends existing benchmarks to many Indic languages through human curation providing multi-way parallel evaluation data for many under-represented Indic languages for the first time. We evaluate a wide range of proprietary and open-source LLMs including GPT-3.5, GPT-4, PaLM-2, mT5, Gemma, BLOOM and LLaMA on IndicGenBench in a variety of settings. The largest PaLM-2 models performs the best on most tasks, however, there is a significant performance gap in all languages compared to English showing that further research is needed for the development of more inclusive multilingual language models. IndicGenBench is released at www.github.com/google-research-datasets/indic-gen-bench

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

google-research-datasets/indic-gen-bench officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Cross-Lingual Question AnsweringDiversityMachine TranslationQuestion Answering

Datasets

Introduced by this paper, per the archive.

CrossSum-INFlores-INXQuAD-INXorQA-IN:

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Absolute Position EncodingsAdafactorAdamAttentionAttention DropoutBLOOMBPECosine AnnealingDense ConnectionsDropoutGPT-3GPT-4Gated Linear UnitInverse Square Root ScheduleLLaMALabel SmoothingLayer NormalizationLinear LayerLinear Warmup With Cosine AnnealingMulti-Head AttentionPosition-Wise Feed-Forward LayerResidual ConnectionSETSentencePieceSoftmaxT5TransformerWeight DecaymT5

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections