{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/direct-preference-optimization-your-language","title":"Direct Preference Optimization: Your Language Model is Secretly a Reward Model","arxiv_id":"2305.18290","date":"2023-05-29","proceeding":"NeurIPS 2023 11","authors":["Rafael Rafailov","Archit Sharma","Eric Mitchell","Stefano Ermon","Christopher D. Manning","Chelsea Finn"],"abstract":"While large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely unsupervised nature of their training. Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF). However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model. In this paper we introduce a new parameterization of the reward model in RLHF that enables extraction of the corresponding optimal policy in closed form, allowing us to solve the standard RLHF problem with only a simple classification loss. The resulting algorithm, which we call Direct Preference Optimization (DPO), is stable, performant, and computationally lightweight, eliminating the need for sampling from the LM during fine-tuning or performing significant hyperparameter tuning. Our experiments show that DPO can fine-tune LMs to align with human preferences as well as or better than existing methods. Notably, fine-tuning with DPO exceeds PPO-based RLHF in ability to control sentiment of generations, and matches or improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.","url_abs":"https://arxiv.org/abs/2305.18290v3","url_pdf":"https://arxiv.org/pdf/2305.18290v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://bitbucket.org/paladinanalytics/direct-preference-optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/12kimih/hicupid","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/TU2021/DPO-VP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/ahmed-alllam/Direct-Preference-Optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/babelscape/alert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/davidkim205/komt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/eric-mitchell/direct-preference-optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/frankxwang/dpo-prefix-sharing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/huggingface/trl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/icip-cas/Word-Deed-Consistency-Test","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/imagination-research/lbt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/internlm/xtuner","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/kidist-amde/ddro","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/lucidrains/self-rewarding-lm-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/lujunru/sampo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/lvwerra/trl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/padlex/trl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/sauc-abadal/alt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/shibing624/medicalgpt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/sunzeyeah/rlhf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/tatsu-lab/alpaca_farm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/uppaal/detox-edit","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/wangclnlp/deepspeed-chat-extension","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/yifan-song793/eto","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/2024-MindSpore-1/Code3/tree/main/MindNLP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/OctopusMind/DPO","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"direct-preference-optimization-your-language","repo_url":"https://github.com/yangyucheng000/Paper-3/tree/main/dir-mindspore-main","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"preference-mapping","task_name":"Preference Mapping"},{"task_slug":"world-knowledge","task_name":"World Knowledge"},{"task_slug":"model","task_name":"model"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"dpo","method_name":"DPO"}],"datasets_introduced":[],"methods_introduced":[{"slug":"dpo","name":"DPO","full_name":"Direct Preference Optimization"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.18290","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.18290"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tatsu-lab/alpaca_farm","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eric-mitchell/direct-preference-optimization","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/imagination-research/lbt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/frankxwang/dpo-prefix-sharing","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/trl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TU2021/DPO-VP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/Paper-3/tree/main/dir-mindspore-main","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/padlex/trl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/self-rewarding-lm-pytorch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yifan-song793/eto","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/icip-cas/Word-Deed-Consistency-Test","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/internlm/xtuner","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunzeyeah/rlhf","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://bitbucket.org/paladinanalytics/direct-preference-optimization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/davidkim205/komt","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shibing624/medicalgpt","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/babelscape/alert","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lujunru/sampo","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OctopusMind/DPO","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lvwerra/trl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wangclnlp/deepspeed-chat-extension","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sauc-abadal/alt","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/12kimih/hicupid","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"ran_fixture":1,"unverified":25},"by_repo_kind":{"listed":{"samples":31,"ran":6,"repositories":7}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"be7453ec96f94038","entry":"DPO","repo":"OctopusMind/DPO","repo_kind":"listed","path":"dpo.py","file_url":"https://github.com/OctopusMind/DPO/blob/HEAD/dpo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"be7453ec96f94038"}},{"code_sha256_prefix":"d69bfa3e1e60a92b","entry":"PairWiseLoss","repo":"sunzeyeah/rlhf","repo_kind":"listed","path":"src/models/loss.py","file_url":"https://github.com/sunzeyeah/rlhf/blob/HEAD/src/models/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d69bfa3e1e60a92b"}},{"code_sha256_prefix":"0a800ad29eb472ac","entry":"Policy","repo":"sauc-abadal/alt","repo_kind":"listed","path":"alt/trainer/alt_trainer.py","file_url":"https://github.com/sauc-abadal/alt/blob/HEAD/alt/trainer/alt_trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a800ad29eb472ac"}},{"code_sha256_prefix":"8f1ce89b816a0734","entry":"convert_state_dict","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/sd_utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/sd_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f1ce89b816a0734"}},{"code_sha256_prefix":"c79211fc0225a27f","entry":"preference_loss","repo":"icip-cas/Word-Deed-Consistency-Test","repo_kind":"listed","path":"dpo/trainers.py","file_url":"https://github.com/icip-cas/Word-Deed-Consistency-Test/blob/HEAD/dpo/trainers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c79211fc0225a27f"}},{"code_sha256_prefix":"5bcf8e086423cf6b","entry":"top_k_top_p_filtering","repo":"padlex/trl","repo_kind":"listed","path":"trl/core.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/core.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5bcf8e086423cf6b"}},{"code_sha256_prefix":"68d8bd0d51b34185","entry":"ALTTrainer_noKL","repo":"sauc-abadal/alt","repo_kind":"listed","path":"alt/trainer/alt_trainer.py","file_url":"https://github.com/sauc-abadal/alt/blob/HEAD/alt/trainer/alt_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68d8bd0d51b34185"}},{"code_sha256_prefix":"88b1660f36f94aca","entry":"T5ForPretrain","repo":"kidist-amde/ddro","repo_kind":"listed","path":"src/pretrain/T5ForPretrain.py","file_url":"https://github.com/kidist-amde/ddro/blob/HEAD/src/pretrain/T5ForPretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88b1660f36f94aca"}},{"code_sha256_prefix":"9fc13a517d28b68e","entry":"T5ForPretrainDPO","repo":"kidist-amde/ddro","repo_kind":"listed","path":"src/pretrain/T5ForPretrain.py","file_url":"https://github.com/kidist-amde/ddro/blob/HEAD/src/pretrain/T5ForPretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9fc13a517d28b68e"}},{"code_sha256_prefix":"99d26fdcc1af5b62","entry":"average_precision","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/evaluate.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99d26fdcc1af5b62"}},{"code_sha256_prefix":"9169d5b42acf842a","entry":"compare_metrics","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/compare_models.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/compare_models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9169d5b42acf842a"}},{"code_sha256_prefix":"e491afec66bfd582","entry":"conversations_formatting_function","repo":"padlex/trl","repo_kind":"listed","path":"trl/extras/dataset_formatting.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/extras/dataset_formatting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e491afec66bfd582"}},{"code_sha256_prefix":"fa59c920caf016e6","entry":"convert_state_dict_to_diffusers","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/sd_utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/sd_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa59c920caf016e6"}},{"code_sha256_prefix":"c32b636f13ed9ee5","entry":"convert_to_scalar","repo":"padlex/trl","repo_kind":"listed","path":"trl/core.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c32b636f13ed9ee5"}},{"code_sha256_prefix":"a51ae45a1f6c0b5b","entry":"flatten_dict","repo":"padlex/trl","repo_kind":"listed","path":"trl/core.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a51ae45a1f6c0b5b"}},{"code_sha256_prefix":"2f5119732e8dbd63","entry":"get_all_parameters","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f5119732e8dbd63"}},{"code_sha256_prefix":"e9ac5f8dce946d2f","entry":"hit_at_k","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/evaluate.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9ac5f8dce946d2f"}},{"code_sha256_prefix":"875b816ac9842673","entry":"instructions_formatting_function","repo":"padlex/trl","repo_kind":"listed","path":"trl/extras/dataset_formatting.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/extras/dataset_formatting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"875b816ac9842673"}},{"code_sha256_prefix":"2a9ca02cb6e196ff","entry":"is_transformers_greater_than","repo":"padlex/trl","repo_kind":"listed","path":"trl/import_utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/import_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a9ca02cb6e196ff"}},{"code_sha256_prefix":"48685a350888e505","entry":"iter_params","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48685a350888e505"}},{"code_sha256_prefix":"0f3aeaf1bff007f2","entry":"load_checkpoint","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/train_ddro_encoder_decoder.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/train_ddro_encoder_decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f3aeaf1bff007f2"}},{"code_sha256_prefix":"4099bb8045c005b6","entry":"load_data","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/eval_ddro_docid_ranking.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/eval_ddro_docid_ranking.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4099bb8045c005b6"}},{"code_sha256_prefix":"01a7e0dc6f29f042","entry":"load_doc_offsets","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/train_ddro_encoder_decoder.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/train_ddro_encoder_decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"01a7e0dc6f29f042"}},{"code_sha256_prefix":"9c879833332cded9","entry":"load_encoded_docid","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/generate_demo.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/generate_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c879833332cded9"}},{"code_sha256_prefix":"1233495c1bf99438","entry":"load_metrics","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/compare_models.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/compare_models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1233495c1bf99438"}},{"code_sha256_prefix":"8c68bf8c060f4814","entry":"load_query_texts","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/train_ddro_encoder_decoder.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/train_ddro_encoder_decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c68bf8c060f4814"}},{"code_sha256_prefix":"19b1210f13924026","entry":"prepare_dataset","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/finetune_docTTTTTquery_NQ.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/finetune_docTTTTTquery_NQ.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19b1210f13924026"}},{"code_sha256_prefix":"a05d09685378cdb9","entry":"recall_at_k","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/utils/evaluate.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a05d09685378cdb9"}},{"code_sha256_prefix":"ade247252fd2d3db","entry":"scheduler_step","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/modeling_sd_base.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/modeling_sd_base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ade247252fd2d3db"}},{"code_sha256_prefix":"ccaedb0c5f904b0b","entry":"setup_chat_format","repo":"padlex/trl","repo_kind":"listed","path":"trl/models/utils.py","file_url":"https://github.com/padlex/trl/blob/HEAD/trl/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ccaedb0c5f904b0b"}},{"code_sha256_prefix":"bdad0f787751d773","entry":"split_dataset","repo":"kidist-amde/ddro-direct-document-relevance-optimization","repo_kind":"listed","path":"src/pretrain/finetune_docTTTTTquery_NQ.py","file_url":"https://github.com/kidist-amde/ddro-direct-document-relevance-optimization/blob/HEAD/src/pretrain/finetune_docTTTTTquery_NQ.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bdad0f787751d773"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}