{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2607-15557","title":"SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents","arxiv_id":"2607.15557","date":"2026-07-17","proceeding":null,"authors":["Yanze Wang","Pengfei Yao","Tianyi Sun","Chuanrui Hu","Yan Xiao","Xiaotian Luo","Yunyun Han","Yifan Chen","Jun Sun","Yafeng Deng"],"abstract":"Agent skills, SKILL files that package reusable procedural knowledge for an LLM agent, are a popular mechanism for extending agent capabilities. Public repositories now host them in large and growing numbers, yet these artifacts are fragmented, redundant, and uneven in quality, and their value in practice is unclear. A core question remains open, namely how to consolidate this open-source SKILL ecosystem into a single usable corpus, and what bounds its benefit on real-world agent tasks. We present SkillCorpus, a framework that aggregates, curates, matches, and evaluates the open skill ecosystem at scale. It filters ~821,000 crawled skills through a multi-stage pipeline into 96,401 skills organised by a 16-class taxonomy and three quality facets (utility, robustness, safety), and pairs them with a fine-tuned retrieval-and-selection stack that matches task-relevant skills. We evaluate end-to-end across three benchmarks (SkillsBench, GDPVal, QwenClawBench), two harnesses, and two open backbones with a frontier robustness check. Integrating SkillCorpus yields consistent gains across all three benchmarks, largest on SkillsBench (+7.5 pp). An operational analysis traces the gains to a coverage boundary and a harness boundary. SkillCorpus is, to our knowledge, the first end-to-end account of when a curated, retrieval-served community corpus improves real agent tasks, and where it does not. The dataset, models, and code are available at https://github.com/EverMind-AI/SkillCorpus.","url_abs":"https://arxiv.org/abs/2607.15557","url_pdf":"https://arxiv.org/pdf/2607.15557","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2607.15557","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2607.15557"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/EverMind-AI/SkillCorpus","reach":null}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ae8c5c9549bb4f1d","entry":"canonical_name","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/hashing.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/hashing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae8c5c9549bb4f1d"}},{"code_sha256_prefix":"0e60c4342a5d78b9","entry":"content_hash","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/hashing.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/hashing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e60c4342a5d78b9"}},{"code_sha256_prefix":"b5683cb70653c983","entry":"copy_skill_to_library","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/fsstore.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/fsstore.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b5683cb70653c983"}},{"code_sha256_prefix":"0ef97577735cd676","entry":"extract_repos_from_readme","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/aggregate/discover.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/aggregate/discover.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ef97577735cd676"}},{"code_sha256_prefix":"3cc4e2cdd9c10fed","entry":"format_embedding_text","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/embed.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/embed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3cc4e2cdd9c10fed"}},{"code_sha256_prefix":"4ec7800b8f2ff5ff","entry":"load_config","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/config.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ec7800b8f2ff5ff"}},{"code_sha256_prefix":"fb1bf6e4bcaf471a","entry":"normalize_body","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/hashing.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/hashing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb1bf6e4bcaf471a"}},{"code_sha256_prefix":"573cb6ec2099c222","entry":"remove_skill_from_library","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/core/fsstore.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/core/fsstore.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"573cb6ec2099c222"}},{"code_sha256_prefix":"231e234a1f07500a","entry":"clone_or_pull","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/aggregate/clone.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/aggregate/clone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"231e234a1f07500a"}},{"code_sha256_prefix":"4433c492b87a50fe","entry":"discover_repos","repo":"EverMind-AI/SkillCorpus","repo_kind":"found_in_text","path":"skillcorpus/aggregate/discover.py","file_url":"https://github.com/EverMind-AI/SkillCorpus/blob/HEAD/skillcorpus/aggregate/discover.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4433c492b87a50fe"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}