{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/i-sheep-self-alignment-of-llm-from-scratch","title":"I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm","arxiv_id":"2408.08072","date":"2024-08-15","proceeding":null,"authors":["Yiming Liang","Ge Zhang","Xingwei Qu","Tianyu Zheng","Jiawei Guo","Xinrun Du","Zhenzhu Yang","Jiaheng Liu","Chenghua Lin","Lei Ma","Wenhao Huang","Jiajun Zhang"],"abstract":"Large Language Models (LLMs) have achieved significant advancements, however, the common learning paradigm treats LLMs as passive information repositories, neglecting their potential for active learning and alignment. Some approaches train LLMs using their own generated synthetic data, exploring the possibility of active alignment. However, there is still a huge gap between these one-time alignment methods and the continuous automatic alignment of humans. In this paper, we introduce \\textbf{I-SHEEP}, an \\textbf{I}terative \\textbf{S}elf-En\\textbf{H}anc\\textbf{E}m\\textbf{E}nt \\textbf{P}aradigm.This human-like paradigm enables LLMs to \\textbf{continuously self-align from scratch with nothing}. Compared to the one-time alignment method Dromedary \\cite{sun2023principledriven}, which refers to the first iteration in this paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama models. I-SHEEP achieves a maximum relative improvement of 78.2\\% in the Alpaca Eval, 24.0\\% in the MT Bench, and an absolute increase of 8.88\\% in the IFEval accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally, I-SHEEP surpasses the base model in various standard benchmark generation tasks, achieving an average improvement of 24.77\\% in code generation tasks, 12.04\\% in TrivialQA, and 20.29\\% in SQuAD. We also provide new insights based on the experiment results. Our codes, datasets, and models are available at \\textbf{https://anonymous.4open.science/r/I-SHEEP}.","url_abs":"https://arxiv.org/abs/2408.08072v3","url_pdf":"https://arxiv.org/pdf/2408.08072v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"i-sheep-self-alignment-of-llm-from-scratch","repo_url":"https://github.com/multimodal-art-projection/I-SHEEP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"code-generation","task_name":"Code Generation"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"llama","method_name":"LLaMA"}],"datasets_introduced":[{"slug":"i-sheep","name":"I-SHEEP","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.08072","atlas_url":"https://app.syntology.ai/?focus=2408.08072","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.08072"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/multimodal-art-projection/I-SHEEP","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"ca84d7c5495591c0","entry":"encode_prompt","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/generate_instruction.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/generate_instruction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ca84d7c5495591c0"}},{"code_sha256_prefix":"b45a72bd84f8f3ae","entry":"extract_number_from_eval","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/evaluate_prompt.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/evaluate_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b45a72bd84f8f3ae"}},{"code_sha256_prefix":"ce139503666c147f","entry":"generate_prompt_eval_following","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/evaluate_prompt.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/evaluate_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce139503666c147f"}},{"code_sha256_prefix":"440b5e7abb72e1c4","entry":"generate_prompt_eval_quality","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/evaluate_prompt.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/evaluate_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"440b5e7abb72e1c4"}},{"code_sha256_prefix":"f59e6e21f977bebc","entry":"sample_by_ppl","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/generate_instruction.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/generate_instruction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f59e6e21f977bebc"}},{"code_sha256_prefix":"684e541204aa6d36","entry":"weighted_sample_without_replacement","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/generate_instruction.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/generate_instruction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"684e541204aa6d36"}},{"code_sha256_prefix":"68e88d11a1f9f4f4","entry":"encode_input","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/generate_output.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/generate_output.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"68e88d11a1f9f4f4"}},{"code_sha256_prefix":"db753eb30a65a8ff","entry":"encode_rawoutput","repo":"multimodal-art-projection/I-SHEEP","repo_kind":"official","path":"seed_ppl/generate_output.py","file_url":"https://github.com/multimodal-art-projection/I-SHEEP/blob/HEAD/seed_ppl/generate_output.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"db753eb30a65a8ff"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}