{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-copilot-bridging-billions-of-data-and","title":"Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow","arxiv_id":"2306.07209","date":"2023-06-12","proceeding":null,"authors":["Wenqi Zhang","Yongliang Shen","Zeqi Tan","Guiyang Hou","Weiming Lu","Yueting Zhuang"],"abstract":"Industries such as finance, meteorology, and energy generate vast amounts of data daily. Efficiently managing, processing, and displaying this data requires specialized expertise and is often tedious and repetitive. Leveraging large language models (LLMs) to develop an automated workflow presents a highly promising solution. However, LLMs are not adept at handling complex numerical computations and table manipulations and are also constrained by a limited context budget. Based on this, we propose Data-Copilot, a data analysis agent that autonomously performs querying, processing, and visualization of massive data tailored to diverse human requests. The advancements are twofold: First, it is a code-centric agent that receives human requests and generates code as an intermediary to handle massive data, which is quite flexible for large-scale data processing tasks. Second, Data-Copilot involves a data exploration phase in advance, which explores how to design more universal and error-free interfaces for real-time response. Specifically, it actively explores data sources, discovers numerous common requests, and abstracts them into many universal interfaces for daily invocation. When deployed in real-time requests, Data-Copilot only needs to invoke these pre-designed interfaces, transforming raw data into visualized outputs (e.g., charts, tables) that best match the user's intent. Compared to generating code from scratch, invoking these pre-designed and compiler-validated interfaces can significantly reduce errors during real-time requests. Additionally, interface workflows are more efficient and offer greater interpretability than code. We open-sourced Data-Copilot with massive Chinese financial data, such as stocks, funds, and news, demonstrating promising application prospects.","url_abs":"https://arxiv.org/abs/2306.07209v8","url_pdf":"https://arxiv.org/pdf/2306.07209v8.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-copilot-bridging-billions-of-data-and","repo_url":"https://github.com/zwq2018/data-copilot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2306.07209","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.07209"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zwq2018/data-copilot","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5ebf0c719c1ea902","entry":"get_last_year_date","repo":"zwq2018/data-copilot","repo_kind":"official","path":"tool.py","file_url":"https://github.com/zwq2018/data-copilot/blob/HEAD/tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ebf0c719c1ea902"}},{"code_sha256_prefix":"31eea34ea770ab20","entry":"get_stock_code","repo":"zwq2018/data-copilot","repo_kind":"official","path":"tool.py","file_url":"https://github.com/zwq2018/data-copilot/blob/HEAD/tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31eea34ea770ab20"}},{"code_sha256_prefix":"2931ccb773e192df","entry":"retry","repo":"zwq2018/data-copilot","repo_kind":"official","path":"lab_gpt4_call.py","file_url":"https://github.com/zwq2018/data-copilot/blob/HEAD/lab_gpt4_call.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2931ccb773e192df"}},{"code_sha256_prefix":"86f1244e80ed9ec9","entry":"send_chat_request","repo":"zwq2018/data-copilot","repo_kind":"official","path":"lab_gpt4_call.py","file_url":"https://github.com/zwq2018/data-copilot/blob/HEAD/lab_gpt4_call.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86f1244e80ed9ec9"}},{"code_sha256_prefix":"29fa9421abf1d7d7","entry":"timeout_decorator","repo":"zwq2018/data-copilot","repo_kind":"official","path":"lab_gpt4_call.py","file_url":"https://github.com/zwq2018/data-copilot/blob/HEAD/lab_gpt4_call.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29fa9421abf1d7d7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}