{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mka-leveraging-cross-lingual-consensus-for","title":"MKA: Leveraging Cross-Lingual Consensus for Model Abstention","arxiv_id":"2503.23687","date":"2025-03-31","proceeding":null,"authors":["Sharad Duwal"],"abstract":"Reliability of LLMs is questionable even as they get better at more tasks. A wider adoption of LLMs is contingent on whether they are usably factual. And if they are not, on whether they can properly calibrate their confidence in their responses. This work focuses on utilizing the multilingual knowledge of an LLM to inform its decision to abstain or answer when prompted. We develop a multilingual pipeline to calibrate the model's confidence and let it abstain when uncertain. We run several multilingual models through the pipeline to profile them across different languages. We find that the performance of the pipeline varies by model and language, but that in general they benefit from it. This is evidenced by the accuracy improvement of $71.2\\%$ for Bengali over a baseline performance without the pipeline. Even a high-resource language like English sees a $15.5\\%$ improvement. These results hint at possible further improvements.","url_abs":"https://arxiv.org/abs/2503.23687v1","url_pdf":"https://arxiv.org/pdf/2503.23687v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mka-leveraging-cross-lingual-consensus-for","repo_url":"https://github.com/sharad461/MKA-hallucination","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"hint","method_name":"HINT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.23687","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.23687"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/sharad461/MKA-hallucination","reach":null}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d687733ae71b86f4","entry":"find_most_probable_answer","repo":"sharad461/MKA-hallucination","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/sharad461/MKA-hallucination/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d687733ae71b86f4"}},{"code_sha256_prefix":"a176812d6e477a25","entry":"get_similarity","repo":"sharad461/MKA-hallucination","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/sharad461/MKA-hallucination/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a176812d6e477a25"}},{"code_sha256_prefix":"15f8c77f22dc20f8","entry":"process_answers","repo":"sharad461/MKA-hallucination","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/sharad461/MKA-hallucination/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15f8c77f22dc20f8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}