{"url":"/task/prompt-engineering","name":"Prompt Engineering","slug":"prompt-engineering","description_markdown":"**Prompt engineering** is the process of designing and refining the prompts used to generate text from language models, such as GPT-3 or similar models. The goal of prompt engineering is to improve the quality and relevance of the generated text by carefully crafting the prompts to elicit the desired responses from the model.\r\n\r\nPrompt engineering involves several steps, including selecting the appropriate model architecture and parameters, designing the prompt format and structure, selecting the appropriate task and training data, and fine-tuning the model using the selected prompt and data.\r\n\r\nPrompt engineering is a crucial step in the development of language models, as it can greatly influence the quality and effectiveness of the model's responses. By carefully designing and refining the prompts used to generate text, researchers and developers can improve the accuracy and relevance of the model's output, making it more useful for a wide range of applications, including chatbots, language translation, content creation, and more.","categories":[{"name":"Computer Vision","url":"/area/computer-vision"},{"name":"Natural Language Processing","url":"/area/natural-language-processing"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":1236,"papers_with_code":454,"benchmarks":16,"benchmark_tables_in_archive":16,"benchmark_tables_shown":16,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":16,"subtasks":1,"parent_tasks":0},"benchmarks":[{"leaderboard":"/sota/prompt-engineering-on-imagenet","slug":"prompt-engineering-on-imagenet","dataset":"ImageNet","dataset_url":"/dataset/imagenet","rows_in_archive":15,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-caltech-101","slug":"prompt-engineering-on-caltech-101","dataset":"Caltech-101","dataset_url":"/dataset/caltech-101","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-dtd","slug":"prompt-engineering-on-dtd","dataset":"DTD","dataset_url":"/dataset/dtd","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-eurosat","slug":"prompt-engineering-on-eurosat","dataset":"EuroSAT","dataset_url":"/dataset/eurosat","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"MMRL++","paper_title":"MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models","paper_url":"/paper/mmrl-parameter-efficient-and-interaction","paper_date":"2025-05-15","arxiv_id":"2505.10088","code_links":[{"title":"yunncheng/MMRL","url":"https://github.com/yunncheng/MMRL"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-fgvc-aircraft","slug":"prompt-engineering-on-fgvc-aircraft","dataset":"FGVC-Aircraft","dataset_url":"/dataset/fgvc-aircraft-1","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-oxford-102-flower","slug":"prompt-engineering-on-oxford-102-flower","dataset":"Oxford 102 Flower","dataset_url":"/dataset/oxford-102-flower","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-oxford-iiit-pet-dataset","slug":"prompt-engineering-on-oxford-iiit-pet-dataset","dataset":"Oxford-IIIT Pet Dataset","dataset_url":"/dataset/oxford-iiit-pets","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-stanford-cars-1","slug":"prompt-engineering-on-stanford-cars-1","dataset":"Stanford Cars","dataset_url":"/dataset/stanford-cars","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-sun397","slug":"prompt-engineering-on-sun397","dataset":"SUN397","dataset_url":"/dataset/sun397","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-ucf101","slug":"prompt-engineering-on-ucf101","dataset":"UCF101","dataset_url":"/dataset/ucf101","rows_in_archive":14,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-food-101","slug":"prompt-engineering-on-food-101","dataset":"Food-101","dataset_url":"/dataset/food-101","rows_in_archive":13,"metrics":["Harmonic mean"],"first_row_in_archive_order":{"model":"PromptKD","paper_title":"PromptKD: Unsupervised Prompt Distillation for Vision-Language Models","paper_url":"/paper/promptkd-unsupervised-prompt-distillation-for","paper_date":"2024-03-05","arxiv_id":"2403.02781","code_links":[{"title":"zhengli97/promptkd","url":"https://github.com/zhengli97/promptkd"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-imagenet-a","slug":"prompt-engineering-on-imagenet-a","dataset":"ImageNet-A","dataset_url":"/dataset/imagenet-a","rows_in_archive":9,"metrics":["Top-1 accuracy %"],"first_row_in_archive_order":{"model":"POMP","paper_title":"Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition","paper_url":"/paper/prompt-pre-training-with-twenty-thousand-1","paper_date":"2023-04-10","arxiv_id":"2304.04704","code_links":[{"title":"amazon-science/prompt-pretraining","url":"https://github.com/amazon-science/prompt-pretraining"}],"syntology":{"n":7,"n_ran":5,"n_unverified":2,"n_pointer_only":0}}},{"leaderboard":"/sota/prompt-engineering-on-imagenet-r","slug":"prompt-engineering-on-imagenet-r","dataset":"ImageNet-R","dataset_url":"/dataset/imagenet-r","rows_in_archive":9,"metrics":["Top-1 accuracy %"],"first_row_in_archive_order":{"model":"POMP","paper_title":"Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition","paper_url":"/paper/prompt-pre-training-with-twenty-thousand-1","paper_date":"2023-04-10","arxiv_id":"2304.04704","code_links":[{"title":"amazon-science/prompt-pretraining","url":"https://github.com/amazon-science/prompt-pretraining"}],"syntology":{"n":7,"n_ran":5,"n_unverified":2,"n_pointer_only":0}}},{"leaderboard":"/sota/prompt-engineering-on-imagenet-s","slug":"prompt-engineering-on-imagenet-s","dataset":"ImageNet-S","dataset_url":"/dataset/imagenet-s","rows_in_archive":9,"metrics":["Top-1 accuracy %"],"first_row_in_archive_order":{"model":"POMP","paper_title":"Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition","paper_url":"/paper/prompt-pre-training-with-twenty-thousand-1","paper_date":"2023-04-10","arxiv_id":"2304.04704","code_links":[{"title":"amazon-science/prompt-pretraining","url":"https://github.com/amazon-science/prompt-pretraining"}],"syntology":{"n":7,"n_ran":5,"n_unverified":2,"n_pointer_only":0}}},{"leaderboard":"/sota/prompt-engineering-on-imagenet-v2","slug":"prompt-engineering-on-imagenet-v2","dataset":"ImageNet V2","dataset_url":"/dataset/imagenet","rows_in_archive":8,"metrics":["Top-1 accuracy %"],"first_row_in_archive_order":{"model":"HPT++","paper_title":"HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling","paper_url":"/paper/hpt-hierarchically-prompting-vision-language","paper_date":"2024-08-27","arxiv_id":"2408.14812","code_links":[{"title":"vill-lab/2024-aaai-hpt","url":"https://github.com/vill-lab/2024-aaai-hpt"},{"title":"ThomasWangY/2024-AAAI-HPT","url":"https://github.com/ThomasWangY/2024-AAAI-HPT"}],"syntology":null}},{"leaderboard":"/sota/prompt-engineering-on-imagenet-21k","slug":"prompt-engineering-on-imagenet-21k","dataset":"ImageNet-21k","dataset_url":null,"rows_in_archive":2,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"POMP","paper_title":"Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition","paper_url":"/paper/prompt-pre-training-with-twenty-thousand-1","paper_date":"2023-04-10","arxiv_id":"2304.04704","code_links":[{"title":"amazon-science/prompt-pretraining","url":"https://github.com/amazon-science/prompt-pretraining"}],"syntology":{"n":7,"n_ran":5,"n_unverified":2,"n_pointer_only":0}}}],"datasets":[{"url":"/dataset/imagenet","name":"ImageNet","full_name":"","num_papers_in_archive":15430},{"url":"/dataset/ucf101","name":"UCF101","full_name":"UCF101 Human Actions dataset","num_papers_in_archive":1863},{"url":"/dataset/oxford-102-flower","name":"Oxford 102 Flower","full_name":"102 Category Flower Dataset","num_papers_in_archive":1307},{"url":"/dataset/dtd","name":"DTD","full_name":"Describable Textures Dataset","num_papers_in_archive":870},{"url":"/dataset/food-101","name":"Food-101","full_name":"","num_papers_in_archive":805},{"url":"/dataset/stanford-cars","name":"Stanford Cars","full_name":"","num_papers_in_archive":790},{"url":"/dataset/caltech-101","name":"Caltech-101","full_name":"","num_papers_in_archive":709},{"url":"/dataset/eurosat","name":"EuroSAT","full_name":"EuroSAT","num_papers_in_archive":687},{"url":"/dataset/fgvc-aircraft-1","name":"FGVC-Aircraft","full_name":"","num_papers_in_archive":520},{"url":"/dataset/imagenet-r","name":"ImageNet-R","full_name":"ImageNet-Rendition","num_papers_in_archive":481},{"url":"/dataset/imagenet-a","name":"ImageNet-A","full_name":"","num_papers_in_archive":431},{"url":"/dataset/oxford-iiit-pets","name":"Oxford-IIIT Pet Dataset","full_name":"Oxford-IIIT Pet Dataset","num_papers_in_archive":90},{"url":"/dataset/oxford-iiit-pets-1","name":"Oxford-IIIT Pets","full_name":"","num_papers_in_archive":59},{"url":"/dataset/sun397","name":"SUN397","full_name":"SUN397","num_papers_in_archive":52},{"url":"/dataset/imagenet-s","name":"ImageNet-S","full_name":"ImageNet Semantic Segmentation","num_papers_in_archive":43},{"url":"/dataset/omnibenchmark","name":"OmniBenchmark","full_name":"","num_papers_in_archive":29}],"subtasks":[{"url":"/task/visual-prompting","name":"Visual Prompting"}],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":454,"tagged_in_all":1236,"items":[{"url":"/paper/learning-transferable-visual-models-from","title":"Learning Transferable Visual Models From Natural Language Supervision","date":"2021-02-26","arxiv_id":"2103.00020","repositories_listed":82,"syntology":{"n":20,"n_ran":16,"n_unverified":4,"n_pointer_only":16}},{"url":"/paper/learning-to-prompt-for-vision-language-models","title":"Learning to Prompt for Vision-Language Models","date":"2021-09-02","arxiv_id":"2109.01134","repositories_listed":18,"syntology":{"n":12,"n_ran":3,"n_unverified":9,"n_pointer_only":0}},{"url":"/paper/conditional-prompt-learning-for-vision","title":"Conditional Prompt Learning for Vision-Language Models","date":"2022-03-10","arxiv_id":"2203.05557","repositories_listed":12,"syntology":{"n":6,"n_ran":4,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/gpt-understands-too","title":"GPT Understands, Too","date":"2021-03-18","arxiv_id":"2103.10385","repositories_listed":10,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/multitask-prompted-training-enables-zero-shot-1","title":"Multitask Prompted Training Enables Zero-Shot Task Generalization","date":"2021-10-15","arxiv_id":"2110.08207","repositories_listed":8,"syntology":{"n":15,"n_ran":8,"n_unverified":7,"n_pointer_only":0}},{"url":"/paper/visual-prompt-tuning","title":"Visual Prompt Tuning","date":"2022-03-23","arxiv_id":"2203.12119","repositories_listed":6,"syntology":{"n":27,"n_ran":17,"n_unverified":10,"n_pointer_only":15}},{"url":"/paper/gpt-takes-the-bar-exam","title":"GPT Takes the Bar Exam","date":"2022-12-29","arxiv_id":"2212.14402","repositories_listed":5,"syntology":null},{"url":"/paper/large-language-models-are-human-level-prompt","title":"Large Language Models Are Human-Level Prompt Engineers","date":"2022-11-03","arxiv_id":"2211.01910","repositories_listed":5,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/langgpt-rethinking-structured-reusable-prompt","title":"LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language","date":"2024-02-26","arxiv_id":"2402.16929","repositories_listed":4,"syntology":null},{"url":"/paper/ip-adapter-text-compatible-image-prompt","title":"IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models","date":"2023-08-13","arxiv_id":"2308.06721","repositories_listed":4,"syntology":{"n":8,"n_ran":3,"n_unverified":5,"n_pointer_only":0}},{"url":"/paper/mme-a-comprehensive-evaluation-benchmark-for","title":"MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models","date":"2023-06-23","arxiv_id":"2306.13394","repositories_listed":4,"syntology":null},{"url":"/paper/differentiable-prompt-makes-pre-trained","title":"Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners","date":"2021-08-30","arxiv_id":"2108.13161","repositories_listed":4,"syntology":{"n":4,"n_ran":3,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/the-prompt-report-a-systematic-survey-of","title":"The Prompt Report: A Systematic Survey of Prompting Techniques","date":"2024-06-06","arxiv_id":"2406.06608","repositories_listed":3,"syntology":{"n":5,"n_ran":3,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/chain-of-thought-reasoning-without-prompting","title":"Chain-of-Thought Reasoning Without Prompting","date":"2024-02-15","arxiv_id":"2402.10200","repositories_listed":3,"syntology":null},{"url":"/paper/optimizing-prompts-for-text-to-image-1","title":"Optimizing Prompts for Text-to-Image Generation","date":"2022-12-19","arxiv_id":"2212.09611","repositories_listed":3,"syntology":{"n":9,"n_ran":1,"n_unverified":8,"n_pointer_only":0}},{"url":"/paper/maple-multi-modal-prompt-learning","title":"MaPLe: Multi-modal Prompt Learning","date":"2022-10-06","arxiv_id":"2210.03117","repositories_listed":3,"syntology":{"n":6,"n_ran":4,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/ask-me-anything-a-simple-strategy-for","title":"Ask Me Anything: A simple strategy for prompting language models","date":"2022-10-05","arxiv_id":"2210.02441","repositories_listed":3,"syntology":{"n":2,"n_ran":2,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/clip-adapter-better-vision-language-models","title":"CLIP-Adapter: Better Vision-Language Models with Feature Adapters","date":"2021-10-09","arxiv_id":"2110.04544","repositories_listed":3,"syntology":{"n":3,"n_ran":0,"n_unverified":3,"n_pointer_only":0}},{"url":"/paper/time-series-forecasting-as-reasoning-a-slow","title":"Time Series Forecasting as Reasoning: A Slow-Thinking Approach with Reinforced LLMs","date":"2025-06-12","arxiv_id":"2506.10630","repositories_listed":2,"syntology":null},{"url":"/paper/risk-aware-driving-scenario-analysis-with","title":"From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios","date":"2025-02-04","arxiv_id":"2502.02145","repositories_listed":2,"syntology":null},{"url":"/paper/a-comparison-of-prompt-engineering-techniques","title":"A Comparison of Prompt Engineering Techniques for Task Planning and Execution in Service Robotics","date":"2024-10-30","arxiv_id":"2410.22997","repositories_listed":2,"syntology":null},{"url":"/paper/hpt-hierarchically-prompting-vision-language","title":"HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling","date":"2024-08-27","arxiv_id":"2408.14812","repositories_listed":2,"syntology":null},{"url":"/paper/lapt-label-driven-automated-prompt-tuning-for","title":"LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models","date":"2024-07-12","arxiv_id":"2407.08966","repositories_listed":2,"syntology":{"n":17,"n_ran":8,"n_unverified":9,"n_pointer_only":0}},{"url":"/paper/paraphrase-types-elicit-prompt-engineering","title":"Paraphrase Types Elicit Prompt Engineering Capabilities","date":"2024-06-28","arxiv_id":"2406.19898","repositories_listed":2,"syntology":null},{"url":"/paper/understanding-sounds-missing-the-questions","title":"Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models","date":"2024-06-12","arxiv_id":"2406.08402","repositories_listed":2,"syntology":null},{"url":"/paper/on-the-worst-prompt-performance-of-large","title":"On the Worst Prompt Performance of Large Language Models","date":"2024-06-08","arxiv_id":"2406.10248","repositories_listed":2,"syntology":null},{"url":"/paper/optimizing-large-language-models-for-openapi","title":"Optimizing Large Language Models for OpenAPI Code Completion","date":"2024-05-24","arxiv_id":"2405.15729","repositories_listed":2,"syntology":null},{"url":"/paper/can-we-use-large-language-models-to-fill","title":"Can We Use Large Language Models to Fill Relevance Judgment Holes?","date":"2024-05-09","arxiv_id":"2405.05600","repositories_listed":2,"syntology":null},{"url":"/paper/semantic-routing-for-enhanced-performance-of","title":"Semantic Routing for Enhanced Performance of LLM-Assisted Intent-Based 5G Core Network Management and Orchestration","date":"2024-04-24","arxiv_id":"2404.15869","repositories_listed":2,"syntology":{"n":3,"n_ran":0,"n_unverified":3,"n_pointer_only":0}},{"url":"/paper/prism-a-promptable-and-robust-interactive","title":"PRISM: A Promptable and Robust Interactive Segmentation Model with Visual Prompts","date":"2024-04-23","arxiv_id":"2404.15028","repositories_listed":2,"syntology":null}],"syntology_records":16,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}