{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-dataset-for-analysing-complex-document","title":"A Dataset for Analysing Complex Document Layouts in the Digital Humanities and Its Evaluation with Krippendorff’s Alpha","arxiv_id":null,"date":"2022-09-20","proceeding":"GCPR 2022 9","authors":["David Tschirschwitz","Franziska Klemstein","Benno Stein","Volker Rodehorst"],"abstract":"We introduce a new research resource in the form of a high-quality, domain-specific dataset for analysing the document layout of historical documents. The dataset provides an instance segmentation ground truth with 19 classes based on historical layout structures that stem (a) from the publication production process and the respective genres (life sciences, architecture, art, decorative arts, etc.) and, (b) from selected text registers (such as monograph, trade journal, illustrated magazine). Altogether, the dataset contains more than 52,000 instances annotated by experts. A baseline has been tested with the well-known Mask R-CNN and compared to the state-of-the-art model VSR. Inspired by evaluation practices from the field of Natural Language Processing (NLP), we have developed a new method for evaluating annotation consistency. Our method is based on Krippendorff’s alpha (K-\r\n), a statistic for quantifying the so-called “inter-annotator-agreement”. In particular, we propose an adaptation of K-alpha\r\n that treats annotations as a multipartite graph for assessing the agreement of a variable number of annotators. The method is adjustable with regard to evaluation strictness, and it can be used in 2D or 3D as well as for a variety of tasks such as semantic segmentation, instance segmentation, and 3D point cloud segmentation.","url_abs":"https://link.springer.com/chapter/10.1007/978-3-031-16788-1_22","url_pdf":"https://madave94.github.io/assets/pdf/a_dataset_for_analysing_complex_layouts.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-dataset-for-analysing-complex-document","repo_url":"https://github.com/Madave94/kalphacv","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"a-dataset-for-analysing-complex-document","repo_url":"https://github.com/Madave94/VSR-TexBiG-Dataset","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"point-cloud-segmentation","task_name":"Point Cloud Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[{"slug":"texbig","name":"TexBiG","full_name":"Text-Bild-Gefüge"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/instance-segmentation-on-texbig-2022-test","task":"Instance Segmentation","dataset":"TexBiG 2022 test","model":"VSR (Vison, Semantics and Relation Model)","rank_in_archive_order":1,"of":1,"metrics":{"mAP@0.5:0.95:0.05":"65.8"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-texbig-2022-test","task":"Object Detection","dataset":"TexBiG 2022 test","model":"VSR (Vison, Semantics and Relation Model)","rank_in_archive_order":1,"of":1,"metrics":{"mAP@0.5:0.95:0.05":"75.9"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}