{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seeing-eye-to-ai-human-alignment-via-gaze","title":"Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models","arxiv_id":"2410.01532","date":"2024-10-02","proceeding":null,"authors":["Angela Lopez-Cardona","Carlos Segura","Alexandros Karatzoglou","Sergi Abadal","Ioannis Arapakis"],"abstract":"Advancements in Natural Language Processing (NLP), have led to the emergence of Large Language Models (LLMs) such as GPT, Llama, Claude, and Gemini, which excel across a range of tasks but require extensive fine-tuning to align their outputs with human expectations. A widely used method for achieving this alignment is Reinforcement Learning from Human Feedback (RLHF), which, despite its success, faces challenges in accurately modelling human preferences. In this paper, we introduce GazeReward, a novel framework that integrates implicit feedback -- and specifically eye-tracking (ET) data -- into the Reward Model (RM). In addition, we explore how ET-based features can provide insights into user preferences. Through ablation studies we test our framework with different integration methods, LLMs, and ET generator models, demonstrating that our approach significantly improves the accuracy of the RM on established human preference datasets. This work advances the ongoing discussion on optimizing AI alignment with human values, exploring the potential of cognitive data for shaping future NLP research.","url_abs":"https://arxiv.org/abs/2410.01532v3","url_pdf":"https://arxiv.org/pdf/2410.01532v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seeing-eye-to-ai-human-alignment-via-gaze","repo_url":"https://github.com/telefonica-scientific-research/gaze_reward","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"LGPL-3.0"}}],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt","method_name":"GPT"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2410.01532","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01532"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/telefonica-scientific-research/gaze_reward","reach":{"status":"ok","spdx":"LGPL-3.0"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"16a28c41c81257e9","entry":"cosine_scheduler_with_min_lr_ratio","repo":"telefonica-scientific-research/gaze_reward","repo_kind":"official","path":"rlhf_rw/trainers/reward_trainer_general.py","file_url":"https://github.com/telefonica-scientific-research/gaze_reward/blob/HEAD/rlhf_rw/trainers/reward_trainer_general.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"16a28c41c81257e9"}},{"code_sha256_prefix":"71e22b388bcd4060","entry":"preprocess_data_reward","repo":"telefonica-scientific-research/gaze_reward","repo_kind":"official","path":"rlhf_rw/reward_utils/dataset_proceser_reward.py","file_url":"https://github.com/telefonica-scientific-research/gaze_reward/blob/HEAD/rlhf_rw/reward_utils/dataset_proceser_reward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"71e22b388bcd4060"}},{"code_sha256_prefix":"c3b979a557f3c20b","entry":"wandb_hp_space","repo":"telefonica-scientific-research/gaze_reward","repo_kind":"official","path":"rlhf_rw/trainers/reward_trainer_general.py","file_url":"https://github.com/telefonica-scientific-research/gaze_reward/blob/HEAD/rlhf_rw/trainers/reward_trainer_general.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c3b979a557f3c20b"}},{"code_sha256_prefix":"a55fa8fa35de83c8","entry":"custom_cosine_scheduler","repo":"telefonica-scientific-research/gaze_reward","repo_kind":"official","path":"rlhf_rw/trainers/reward_trainer.py","file_url":"https://github.com/telefonica-scientific-research/gaze_reward/blob/HEAD/rlhf_rw/trainers/reward_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"LGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a55fa8fa35de83c8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}