{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-10547","title":"Automatic End-to-End Data Integration using Large Language Models","arxiv_id":"2603.10547","date":"2026-03-11","proceeding":null,"authors":["Aaron Steiner","Christian Bizer"],"abstract":"Designing data integration pipelines typically requires substantial manual effort from data engineers to configure pipeline components and label training data. While LLMs have shown promise in handling individual steps of the integration process, their potential to replace all human input across end-to-end data integration pipelines has not been investigated. As a step toward exploring this potential, we present an automatic data integration pipeline that uses GPT-5.2 to generate all artifacts required to adapt the pipeline to specific use cases. These artifacts are schema mappings, value mappings for data normalization, training data for entity matching, and validation data for selecting conflict resolution heuristics in data fusion. We compare the performance of this LLM-based pipeline to the performance of human-designed pipelines along three case studies requiring the integration of video game, music, and company related data. Our experiments show that the LLM-based pipeline is able to produce similar results, for some tasks even better results, as the human-designed pipelines. End-to-end, the human and the LLM pipelines produce integrated datasets of comparable size and density. Having the LLM configure the pipelines costs approximately \\$10 per case study, which represents only a small fraction of the cost of having human data engineers perform the same tasks.","url_abs":"https://arxiv.org/abs/2603.10547","url_pdf":"https://arxiv.org/pdf/2603.10547","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.10547","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.10547"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration","reach":null}],"summary":{"ran":2,"ran_draft_wrong":4,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9d0f64593f5d0f0d","entry":"LLMCallLogger","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d0f64593f5d0f0d"}},{"code_sha256_prefix":"50c993a04b30b76a","entry":"LLMCallRecord","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50c993a04b30b76a"}},{"code_sha256_prefix":"e1227f408c9b5a1b","entry":"_extract_usage_metadata","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e1227f408c9b5a1b"}},{"code_sha256_prefix":"1142f19e8e3c5854","entry":"_get_model_name","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1142f19e8e3c5854"}},{"code_sha256_prefix":"afd325a26f60dce5","entry":"_safe_getattr","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"afd325a26f60dce5"}},{"code_sha256_prefix":"e718190212e9b76c","entry":"_serialize_messages","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e718190212e9b76c"}},{"code_sha256_prefix":"8a3c801e24b27dcd","entry":"BaseMatcher","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a3c801e24b27dcd"}},{"code_sha256_prefix":"0a99423a5452f0f1","entry":"LLMBasedMatcher","repo":"wbsg-uni-mannheim/automatic-data-integration","repo_kind":"found_in_text","path":"PyDI/entitymatching/llm_based.py","file_url":"https://github.com/wbsg-uni-mannheim/automatic-data-integration/blob/HEAD/PyDI/entitymatching/llm_based.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a99423a5452f0f1"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}