{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hippo-enhancing-the-table-understanding","title":"HIPPO: Enhancing the Table Understanding Capability of Large Language Models through Hybrid-Modal Preference Optimization","arxiv_id":"2502.17315","date":"2025-02-24","proceeding":null,"authors":["Zhenghao Liu","Haolan Wang","Xinze Li","Qiushi Xiong","Xiaocui Yang","Yu Gu","Yukun Yan","Qi Shi","Fangfang Li","Ge Yu","Maosong Sun"],"abstract":"Tabular data contains rich structural semantics and plays a crucial role in organizing and manipulating information. To better capture these structural semantics, this paper introduces the HybrId-modal Preference oPtimizatiOn (HIPPO) model, which represents tables using both text and image, and optimizes MLLMs to effectively learn more comprehensive table information from these multiple modalities. Specifically, HIPPO samples model responses from hybrid-modal table representations and designs a modality-consistent sampling strategy to enhance response diversity and mitigate modality bias during DPO training. Experimental results on table question answering and table fact verification tasks demonstrate the effectiveness of HIPPO, achieving a 4% improvement over various table reasoning models. Further analysis reveals that HIPPO not only enhances reasoning abilities based on unimodal table representations but also facilitates the extraction of crucial and distinct semantics from different modal representations. All data and codes are available at https://github.com/NEUIR/HIPPO.","url_abs":"https://arxiv.org/abs/2502.17315v1","url_pdf":"https://arxiv.org/pdf/2502.17315v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hippo-enhancing-the-table-understanding","repo_url":"https://github.com/neuir/hippo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"fact-verification","task_name":"Fact Verification"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"dpo","method_name":"DPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.17315","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.17315"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neuir/hippo","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c39977215832d2d1","entry":"extract_tqa_answer_list","repo":"neuir/hippo","repo_kind":"official","path":"src/generate_data/generate_dpo_data.py","file_url":"https://github.com/neuir/hippo/blob/HEAD/src/generate_data/generate_dpo_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c39977215832d2d1"}},{"code_sha256_prefix":"76ce4cc039285ed2","entry":"get_markdown_table","repo":"neuir/hippo","repo_kind":"official","path":"src/generate_data/image_generate_response.py","file_url":"https://github.com/neuir/hippo/blob/HEAD/src/generate_data/image_generate_response.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76ce4cc039285ed2"}},{"code_sha256_prefix":"1402245af4d59028","entry":"get_markdown_table_infotabs","repo":"neuir/hippo","repo_kind":"official","path":"src/generate_data/image_generate_response.py","file_url":"https://github.com/neuir/hippo/blob/HEAD/src/generate_data/image_generate_response.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1402245af4d59028"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}