MCoNaLa (Multilingual CoNaLa)

MCoNaLa is a multilingual dataset to benchmark code generation from natural language commands extending beyond English. Modeled off of the methodology from the English Code/Natural Language Challenge (CoNALa) dataset, the authors annotated a total of 896 NL-code pairs in three languages: Spanish, Japanese, and Russian.

Due to the limited sample of multiple languages, we use English CoNaLa samples for training, where the intents are originally written in English. Spanish, Japanese, and Russian are of the Target Language (TL), whose samples are always (only) used for testing purpose due to the limited amount.

English is the High-Resource Language (HRL) for which the samples can be leveraged for model training.