CII-Bench (Chinese Image Implication understanding Benchmark)

We introduce the Chinese Image Implication Understanding Benchmark CII-Bench, a new benchmark measuring the higher-order perceptual, reasoning and comprehension abilities of MLLMs when presented with complex Chinese implication images. These images, including abstract artworks, comics and posters, possess visual implications that require an understanding of visual details and reasoning ability. CII-Bench reveals whether current MLLMs, leveraging their inherent comprehension abilities, can accurately decode the metaphors embedded within the complex and abstract information presented in these images.

CII-Bench comprises 698 Chinese images, each accompanied by 1 to 3 multiple-choice questions, totaling 800 questions. CII-Bench encompasses images from six distinct domains: Life, Art, Society, Environment, Politics, and Chinese Traditional Culture. It also features a diverse array of image types, including Illustrations, Memes, Posters, Multi-panel Comics, Single-panel Comics, and Paintings.