FewGLUE_64_labeled (A new version of FewGLUE with 64 training examples)

Introduction

The FewGLUE_64_labeled dataset is a new version of FewGLUE dataset. It contains a 64-sample training set, a development set (the original SuperGLUE development set), a test set, and an unlabeled set. It is constructed to facilitate the research of few-shot learning for natural language understanding tasks.

Compared with the original FewGLUE dataset, it differs in the number of labeled data examples in the training set, where the original FewGLUE has 32 training examples while FewGLUE_64_labeled has 64 labeled examples. Purposes for constructing a new version of FewGLUE dataset include:

1. To answer the questions that what is the best performance that few-shot learning can achieve and whether it is possible to further close the performance gap between few-shot learning and fully-supervised systems.

2. To explore to which degree the number of labeled training examples influences the few-shot performance.

Please refer to the FewNLU paper as well as the FewNLU leaderboard for more details.

Acknowledgement

Part of the FewGLUE_64_labeled dataset is based on the original 32-sample version of FewGLUE. We collect them together in one package for the convenience of usage. We appreciate all the contributors who made their dataset public, which greatly advanced few-shot learning as well as the FewNLU project.