{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-archive-query-log-mining-millions-of","title":"The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives","arxiv_id":"2304.00413","date":"2023-04-02","proceeding":null,"authors":["Jan Heinrich Reimer","Sebastian Schmidt","Maik Fröbe","Lukas Gienapp","Harrisen Scells","Benno Stein","Matthias Hagen","Martin Potthast"],"abstract":"The Archive Query Log (AQL) is a previously unused, comprehensive query log collected at the Internet Archive over the last 25 years. Its first version includes 356 million queries, 166 million search result pages, and 1.7 billion search results across 550 search providers. Although many query logs have been studied in the literature, the search providers that own them generally do not publish their logs to protect user privacy and vital business data. Of the few query logs publicly available, none combines size, scope, and diversity. The AQL is the first to do so, enabling research on new retrieval models and (diachronic) search engine analyses. Provided in a privacy-preserving manner, it promotes open research as well as more transparency and accountability in the search industry.","url_abs":"https://arxiv.org/abs/2304.00413v2","url_pdf":"https://arxiv.org/pdf/2304.00413v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-archive-query-log-mining-millions-of","repo_url":"https://github.com/webis-de/archive-query-log","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"the-archive-query-log-mining-millions-of","repo_url":"https://github.com/webis-de/sigir-23","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"privacy-preserving","task_name":"Privacy Preserving"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":null,"method_name":"None"}],"datasets_introduced":[{"slug":"aql-22","name":"AQL-22","full_name":"Archive Query Log"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}