{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/peneo-unifying-line-extraction-line-grouping","title":"PEneo: Unifying Line Extraction, Line Grouping, and Entity Linking for End-to-end Document Pair Extraction","arxiv_id":"2401.03472","date":"2024-01-07","proceeding":null,"authors":["Zening Lin","Jiapeng Wang","Teng Li","Wenhui Liao","Dayi Huang","Longfei Xiong","Lianwen Jin"],"abstract":"Document pair extraction aims to identify key and value entities as well as their relationships from visually-rich documents. Most existing methods divide it into two separate tasks: semantic entity recognition (SER) and relation extraction (RE). However, simply concatenating SER and RE serially can lead to severe error propagation, and it fails to handle cases like multi-line entities in real scenarios. To address these issues, this paper introduces a novel framework, PEneo (Pair Extraction new decoder option), which performs document pair extraction in a unified pipeline, incorporating three concurrent sub-tasks: line extraction, line grouping, and entity linking. This approach alleviates the error accumulation problem and can handle the case of multi-line entities. Furthermore, to better evaluate the model's performance and to facilitate future research on pair extraction, we introduce RFUND, a re-annotated version of the commonly used FUNSD and XFUND datasets, to make them more accurate and cover realistic situations. Experiments on various benchmarks demonstrate PEneo's superiority over previous pipelines, boosting the performance by a large margin (e.g., 19.89%-22.91% F1 score on RFUND-EN) when combined with various backbones like LiLT and LayoutLMv3, showing its effectiveness and generality. Codes and the new annotations are available at https://github.com/ZeningLin/PEneo.","url_abs":"https://arxiv.org/abs/2401.03472v3","url_pdf":"https://arxiv.org/pdf/2401.03472v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"peneo-unifying-line-extraction-line-grouping","repo_url":"https://github.com/ZeningLin/PEneo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"key-information-extraction","task_name":"Key Information Extraction"},{"task_slug":"key-value-pair-extraction","task_name":"Key-value Pair Extraction"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"semantic-entity-labeling","task_name":"Semantic entity labeling"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"layoutlmv2","method_name":"LayoutLMv2"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"resnext","method_name":"ResNeXt"},{"method_slug":"resnext-block","method_name":"ResNeXt Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[{"slug":"rfund","name":"RFUND","full_name":"Revised FUNSD and XFUND"},{"slug":"rfund-en","name":"RFUND-EN","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"PEneo\n(LayoutLMv3_base)","rank_in_archive_order":1,"of":13,"metrics":{"key-value pair F1":"79.27"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"PEneo\n(LiLT[InfoXLM]_base)","rank_in_archive_order":2,"of":13,"metrics":{"key-value pair F1":"74.29"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"PEneo\n(LayoutXLM_base)","rank_in_archive_order":3,"of":13,"metrics":{"key-value pair F1":"74.25"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"PEneo\n(LiLT[EN-R]_base)","rank_in_archive_order":4,"of":13,"metrics":{"key-value pair F1":"74.22"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"PEneo\n(LayoutLMv2_base)","rank_in_archive_order":5,"of":13,"metrics":{"key-value pair F1":"71.97"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-sibr","task":"Key-value Pair Extraction","dataset":"SIBR","model":"PEneo\n(LayoutLMv3_base_chinese)","rank_in_archive_order":1,"of":7,"metrics":{"key-value pair F1":"82.52"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-sibr","task":"Key-value Pair Extraction","dataset":"SIBR","model":"PEneo\n(LiLT[InfoXLM]_base)","rank_in_archive_order":2,"of":7,"metrics":{"key-value pair F1":"82.36"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-sibr","task":"Key-value Pair Extraction","dataset":"SIBR","model":"PEneo\n(LayoutXLM_base)","rank_in_archive_order":3,"of":7,"metrics":{"key-value pair F1":"82.23"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}