{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/q-align-teaching-lmms-for-visual-scoring-via","title":"Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels","arxiv_id":"2312.17090","date":"2023-12-28","proceeding":null,"authors":["HaoNing Wu","ZiCheng Zhang","Weixia Zhang","Chaofeng Chen","Liang Liao","Chunyi Li","Yixuan Gao","Annan Wang","Erli Zhang","Wenxiu Sun","Qiong Yan","Xiongkuo Min","Guangtao Zhai","Weisi Lin"],"abstract":"The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents. While recent studies have demonstrated the exceptional potentials of large multi-modality models (LMMs) on a wide range of related fields, in this work, we explore how to teach them for visual rating aligned with human opinions. Observing that human raters only learn and judge discrete text-defined levels in subjective studies, we propose to emulate this subjective process and teach LMMs with text-defined rating levels instead of scores. The proposed Q-Align achieves state-of-the-art performance on image quality assessment (IQA), image aesthetic assessment (IAA), as well as video quality assessment (VQA) tasks under the original LMM structure. With the syllabus, we further unify the three tasks into one model, termed the OneAlign. In our experiments, we demonstrate the advantage of the discrete-level-based syllabus over direct-score-based variants for LMMs. Our code and the pre-trained weights are released at https://github.com/Q-Future/Q-Align.","url_abs":"https://arxiv.org/abs/2312.17090v1","url_pdf":"https://arxiv.org/pdf/2312.17090v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"q-align-teaching-lmms-for-visual-scoring-via","repo_url":"https://github.com/q-future/q-align","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"aesthetics-quality-assessment","task_name":"Aesthetics Quality Assessment"},{"task_slug":"image-quality-assessment","task_name":"Image Quality Assessment"},{"task_slug":"video-quality-assessment","task_name":"Video Quality Assessment"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/aesthetics-quality-assessment-on-aesthetic","task":"Aesthetics Quality Assessment","dataset":"Aesthetic Visual Analysis","model":"OneAlign","rank_in_archive_order":3,"of":3,"metrics":{"SRCC":"0.823"},"uses_additional_data":false},{"leaderboard":"/sota/image-quality-assessment-on-koniq-10k","task":"Image Quality Assessment","dataset":"KonIQ-10k","model":"OneAlign","rank_in_archive_order":2,"of":4,"metrics":{"PLCC":"0.952","SRCC":"0.941"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-live-fb-lsvq","task":"Video Quality Assessment","dataset":"LIVE-FB LSVQ","model":"OneAlign + FAST-VQA","rank_in_archive_order":1,"of":13,"metrics":{"PLCC":"0.900"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-live-fb-lsvq","task":"Video Quality Assessment","dataset":"LIVE-FB LSVQ","model":"OneAlign","rank_in_archive_order":3,"of":13,"metrics":{"PLCC":"0.886"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-msu-sr-qa-dataset","task":"Video Quality Assessment","dataset":"MSU SR-QA Dataset","model":"Q-Align (IQA)","rank_in_archive_order":2,"of":60,"metrics":{"KLCC":"0.61677","PLCC":"0.74116","SROCC":"0.75088","Type":"NR"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-msu-sr-qa-dataset","task":"Video Quality Assessment","dataset":"MSU SR-QA Dataset","model":"Q-Align (VQA)","rank_in_archive_order":3,"of":60,"metrics":{"KLCC":"0.58634","PLCC":"0.71121","SROCC":"0.71812","Type":"NR"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-msu-sr-qa-dataset","task":"Video Quality Assessment","dataset":"MSU SR-QA Dataset","model":"Q-Align (IAA)","rank_in_archive_order":36,"of":60,"metrics":{"KLCC":"0.42211","PLCC":"0.50055","SROCC":"0.51521","Type":"NR"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2312.17090","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.17090"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/q-future/q-align","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"72950fb8c1bfc7a8","entry":"load_video","repo":"q-future/q-align","repo_kind":"official","path":"q_align/evaluate/scorer.py","file_url":"https://github.com/q-future/q-align/blob/HEAD/q_align/evaluate/scorer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"72950fb8c1bfc7a8"}},{"code_sha256_prefix":"f0748f0bd53b18ae","entry":"wa5","repo":"q-future/q-align","repo_kind":"official","path":"q_align/evaluate/iaa_eval.py","file_url":"https://github.com/q-future/q-align/blob/HEAD/q_align/evaluate/iaa_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f0748f0bd53b18ae"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}