{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mg-verilog-multi-grained-dataset-towards","title":"MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation","arxiv_id":"2407.01910","date":"2024-07-02","proceeding":null,"authors":["Yongan Zhang","Zhongzhi Yu","Yonggan Fu","Cheng Wan","Yingyan Celine Lin"],"abstract":"Large Language Models (LLMs) have recently shown promise in streamlining hardware design processes by encapsulating vast amounts of domain-specific data. In addition, they allow users to interact with the design processes through natural language instructions, thus making hardware design more accessible to developers. However, effectively leveraging LLMs in hardware design necessitates providing domain-specific data during inference (e.g., through in-context learning), fine-tuning, or pre-training. Unfortunately, existing publicly available hardware datasets are often limited in size, complexity, or detail, which hinders the effectiveness of LLMs in hardware design tasks. To address this issue, we first propose a set of criteria for creating high-quality hardware datasets that can effectively enhance LLM-assisted hardware design. Based on these criteria, we propose a Multi-Grained-Verilog (MG-Verilog) dataset, which encompasses descriptions at various levels of detail and corresponding code samples. To benefit the broader hardware design community, we have developed an open-source infrastructure that facilitates easy access, integration, and extension of the dataset to meet specific project needs. Furthermore, to fully exploit the potential of the MG-Verilog dataset, which varies in complexity and detail, we introduce a balanced fine-tuning scheme. This scheme serves as a unique use case to leverage the diverse levels of detail provided by the dataset. Extensive experiments demonstrate that the proposed dataset and fine-tuning scheme consistently improve the performance of LLMs in hardware design tasks.","url_abs":"https://arxiv.org/abs/2407.01910v2","url_pdf":"https://arxiv.org/pdf/2407.01910v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mg-verilog-multi-grained-dataset-towards","repo_url":"https://github.com/luke-avionics/mg-verilog","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.01910","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.01910"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/luke-avionics/mg-verilog","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f327928b4d07faf0","entry":"check_correctness","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"verilog_eval/verilog_eval/execution.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/verilog_eval/verilog_eval/execution.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f327928b4d07faf0"}},{"code_sha256_prefix":"0fe5b6fed06ae660","entry":"estimate_pass_at_k","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"verilog_eval/verilog_eval/evaluation.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/verilog_eval/verilog_eval/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0fe5b6fed06ae660"}},{"code_sha256_prefix":"77262c8569201c83","entry":"eval_file_to_part_data","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"auto_data_gen_val/verilog_eval_to_part_data.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/auto_data_gen_val/verilog_eval_to_part_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77262c8569201c83"}},{"code_sha256_prefix":"5a448926543c89fa","entry":"gpt4_request","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"auto_data_gen_val/chain_utils.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/auto_data_gen_val/chain_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a448926543c89fa"}},{"code_sha256_prefix":"c06721d357c61091","entry":"read_problems","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"verilog_eval/verilog_eval/data.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/verilog_eval/verilog_eval/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c06721d357c61091"}},{"code_sha256_prefix":"acfc201fecb4a16a","entry":"completion_request","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"auto_data_gen_val/chain_utils.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/auto_data_gen_val/chain_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acfc201fecb4a16a"}},{"code_sha256_prefix":"812adf6e1ab9441d","entry":"gpt3_5_request","repo":"luke-avionics/mg-verilog","repo_kind":"official","path":"auto_data_gen_val/chain_utils.py","file_url":"https://github.com/luke-avionics/mg-verilog/blob/HEAD/auto_data_gen_val/chain_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"812adf6e1ab9441d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}