{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lilt-a-simple-yet-effective-language","title":"LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding","arxiv_id":"2202.13669","date":"2022-02-28","proceeding":"ACL 2022 5","authors":["Jiapeng Wang","Lianwen Jin","Kai Ding"],"abstract":"Structured document understanding has attracted considerable attention and made significant progress recently, owing to its crucial role in intelligent document processing. However, most existing related models can only deal with the document data of specific language(s) (typically English) included in the pre-training collection, which is extremely limited. To address this issue, we propose a simple yet effective Language-independent Layout Transformer (LiLT) for structured document understanding. LiLT can be pre-trained on the structured documents of a single language and then directly fine-tuned on other languages with the corresponding off-the-shelf monolingual/multilingual pre-trained textual models. Experimental results on eight languages have shown that LiLT can achieve competitive or even superior performance on diverse widely-used downstream benchmarks, which enables language-independent benefit from the pre-training of document layout structure. Code and model are publicly available at https://github.com/jpWang/LiLT.","url_abs":"https://arxiv.org/abs/2202.13669v1","url_pdf":"https://arxiv.org/pdf/2202.13669v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lilt-a-simple-yet-effective-language","repo_url":"https://github.com/jpwang/lilt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"lilt-a-simple-yet-effective-language","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"lilt-a-simple-yet-effective-language","repo_url":"https://github.com/MS-P3/code3/tree/main/lilt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"lilt-a-simple-yet-effective-language","repo_url":"https://github.com/MindSpore-scientific-2/code-14/tree/main/lilt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"lilt-a-simple-yet-effective-language","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/lilt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"document-image-classification","task_name":"Document Image Classification"},{"task_slug":"key-information-extraction","task_name":"Key Information Extraction"},{"task_slug":"key-value-pair-extraction","task_name":"Key-value Pair Extraction"},{"task_slug":"semantic-entity-labeling","task_name":"Semantic entity labeling"},{"task_slug":"document-understanding","task_name":"document understanding"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/document-image-classification-on-rvl-cdip","task":"Document Image Classification","dataset":"RVL-CDIP","model":"LiLT[EN-R]BASE","rank_in_archive_order":5,"of":31,"metrics":{"Accuracy":"95.68%"},"uses_additional_data":false},{"leaderboard":"/sota/key-information-extraction-on-cord","task":"Key Information Extraction","dataset":"CORD","model":"LILT","rank_in_archive_order":7,"of":9,"metrics":{"F1":"96.07"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"LiLT\n([EN-R]_base)","rank_in_archive_order":8,"of":13,"metrics":{"key-value pair F1":"54.33"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-rfund-en","task":"Key-value Pair Extraction","dataset":"RFUND-EN","model":"LiLT\n([InfoXLM]_base)","rank_in_archive_order":10,"of":13,"metrics":{"key-value pair F1":"52.18"},"uses_additional_data":false},{"leaderboard":"/sota/key-value-pair-extraction-on-sibr","task":"Key-value Pair Extraction","dataset":"SIBR","model":"LiLT\n([InfoXLM]_base)","rank_in_archive_order":5,"of":7,"metrics":{"key-value pair F1":"72.76"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-entity-labeling-on-funsd","task":"Semantic entity labeling","dataset":"FUNSD","model":"LILT","rank_in_archive_order":10,"of":15,"metrics":{"F1":"88.41"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.13669","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.13669"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MS-P3/code3/tree/main/lilt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/lilt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-14/tree/main/lilt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jpwang/lilt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/jpWang/LiLT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"da1c54dffed5d60c","entry":"create_position_ids_from_input_ids","repo":"jpWang/LiLT","repo_kind":"official","path":"LiLTfinetune/models/LiLTRobertaLike/modeling_LiLTRobertaLike.py","file_url":"https://github.com/jpWang/LiLT/blob/HEAD/LiLTfinetune/models/LiLTRobertaLike/modeling_LiLTRobertaLike.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da1c54dffed5d60c"}},{"code_sha256_prefix":"9f1130d32c6430f8","entry":"get_last_checkpoint","repo":"jpWang/LiLT","repo_kind":"official","path":"LiLTfinetune/evaluation.py","file_url":"https://github.com/jpWang/LiLT/blob/HEAD/LiLTfinetune/evaluation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f1130d32c6430f8"}},{"code_sha256_prefix":"e12a52020d4a022c","entry":"re_score","repo":"jpWang/LiLT","repo_kind":"official","path":"LiLTfinetune/evaluation.py","file_url":"https://github.com/jpWang/LiLT/blob/HEAD/LiLTfinetune/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e12a52020d4a022c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}