{"url":"/dataset/mimic-it","name":"MIMIC-IT","full_name":null,"description_markdown":"**MultI-Modal In-Context Instruction Tuning (MIMIC-IT)** is a dataset for instruction tuning into multi-modal models, motivated by the Flamingo model's upstream interleaved format pretraining dataset. The data sample consists of a queried image-instruction-answer triplet, with the instruction-answer tailored to the image, and context. The context contains a series of image-instruction-answer triplets that contextually correlate with the queried triplet, emulating the relationship between the context and the queried image-text pair found in the MMC4 dataset.\r\n\r\nSource: [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://arxiv.org/pdf/2305.03726v1.pdf)\r\n\r\nImage Source: [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://arxiv.org/pdf/2305.03726v1.pdf)","description_withheld":null,"homepage":"https://github.com/luodian/otter","introduced_date":"2023-05-05","introduced_date_note":null,"introduced_by":{"paper":"/paper/otter-a-multi-modal-model-with-in-context","title":"Otter: A Multi-Modal Model with In-Context Instruction Tuning","first_author":"Bo Li","url":null},"license":{"name":"MIT license","url":"https://github.com/Luodian/Otter/blob/main/LICENSE"},"modalities":[{"name":"Images","url":"/datasets/modality/images"},{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Instruction Following","url":"/task/instruction-following","datasets_with_task":"/datasets/task/instruction-following"}],"languages":[],"variants":["MIMIC-IT"],"data_loaders":[],"num_papers_in_archive":11,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}