{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fast-vqa-efficient-end-to-end-video-quality","title":"FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling","arxiv_id":"2207.02595","date":"2022-07-06","proceeding":null,"authors":["HaoNing Wu","Chaofeng Chen","Jingwen Hou","Liang Liao","Annan Wang","Wenxiu Sun","Qiong Yan","Weisi Lin"],"abstract":"Current deep video quality assessment (VQA) methods are usually with high computational costs when evaluating high-resolution videos. This cost hinders them from learning better video-quality-related representations via end-to-end training. Existing approaches typically consider naive sampling to reduce the computational cost, such as resizing and cropping. However, they obviously corrupt quality-related information in videos and are thus not optimal for learning good representations for VQA. Therefore, there is an eager need to design a new quality-retained sampling scheme for VQA. In this paper, we propose Grid Mini-patch Sampling (GMS), which allows consideration of local quality by sampling patches at their raw resolution and covers global quality with contextual relations via mini-patches sampled in uniform grids. These mini-patches are spliced and aligned temporally, named as fragments. We further build the Fragment Attention Network (FANet) specially designed to accommodate fragments as inputs. Consisting of fragments and FANet, the proposed FrAgment Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and learns effective video-quality-related representations. It improves state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P high-resolution videos. The newly learned video-quality-related representations can also be transferred into smaller VQA datasets, boosting performance in these scenarios. Extensive experiments show that FAST-VQA has good performance on inputs of various resolutions while retaining high efficiency. We publish our code at https://github.com/timothyhtimothy/FAST-VQA.","url_abs":"https://arxiv.org/abs/2207.02595v1","url_pdf":"https://arxiv.org/pdf/2207.02595v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fast-vqa-efficient-end-to-end-video-quality","repo_url":"https://github.com/timothyhtimothy/fast-vqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"fast-vqa-efficient-end-to-end-video-quality","repo_url":"https://github.com/QualityAssessment/FAST-VQA-and-FasterVQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"fast-vqa-efficient-end-to-end-video-quality","repo_url":"https://github.com/VQAssessment/FAST-VQA-and-FasterVQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"fast-vqa-efficient-end-to-end-video-quality","repo_url":"https://github.com/timothyhtimothy/fast-vqa-and-fastervqa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"video-quality-assessment","task_name":"Video Quality Assessment"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-quality-assessment-on-konvid-1k","task":"Video Quality Assessment","dataset":"KoNViD-1k","model":"FAST-VQA (finetuned on KonViD-1k)","rank_in_archive_order":4,"of":21,"metrics":{"PLCC":"0.892"},"uses_additional_data":true},{"leaderboard":"/sota/video-quality-assessment-on-konvid-1k","task":"Video Quality Assessment","dataset":"KoNViD-1k","model":"FAST-VQA (trained on LSVQ only)","rank_in_archive_order":9,"of":21,"metrics":{"PLCC":"0.855"},"uses_additional_data":true},{"leaderboard":"/sota/video-quality-assessment-on-live-fb-lsvq","task":"Video Quality Assessment","dataset":"LIVE-FB LSVQ","model":"FAST-VQA","rank_in_archive_order":4,"of":13,"metrics":{"PLCC":"0.877"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-live-vqc","task":"Video Quality Assessment","dataset":"LIVE-VQC","model":"FAST-VQA (finetuned on LIVE-VQC)","rank_in_archive_order":4,"of":20,"metrics":{"PLCC":"0.862"},"uses_additional_data":true},{"leaderboard":"/sota/video-quality-assessment-on-live-vqc","task":"Video Quality Assessment","dataset":"LIVE-VQC","model":"FAST-VQA (trained on LSVQ only)","rank_in_archive_order":6,"of":20,"metrics":{"PLCC":"0.844"},"uses_additional_data":true},{"leaderboard":"/sota/video-quality-assessment-on-msu-video-quality","task":"Video Quality Assessment","dataset":"MSU NR VQA Database","model":"FAST-VQA","rank_in_archive_order":17,"of":21,"metrics":{"KLCC":"0.6498","PLCC":"0.8613","SRCC":"0.8308","Type":"NR"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-msu-video-quality","task":"Video Quality Assessment","dataset":"MSU NR VQA Database","model":"FASTER-VQA","rank_in_archive_order":18,"of":21,"metrics":{"KLCC":"0.5645","PLCC":"0.8087","SRCC":"0.7508","Type":"NR"},"uses_additional_data":false},{"leaderboard":"/sota/video-quality-assessment-on-youtube-ugc","task":"Video Quality Assessment","dataset":"YouTube-UGC","model":"FAST-VQA (finetuned on YouTube-UGC)","rank_in_archive_order":6,"of":17,"metrics":{"PLCC":"0.852"},"uses_additional_data":true},{"leaderboard":"/sota/video-quality-assessment-on-youtube-ugc","task":"Video Quality Assessment","dataset":"YouTube-UGC","model":"FAST-VQA (trained on LSVQ only)","rank_in_archive_order":16,"of":17,"metrics":{"PLCC":"0.748"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.02595","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.02595"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/timothyhtimothy/FAST-VQA","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"9313f05ec0453024","entry":"get_adaptive_window_size","repo":"timothyhtimothy/FAST-VQA","repo_kind":"official","path":"fastvqa/models/swin_backbone.py","file_url":"https://github.com/timothyhtimothy/FAST-VQA/blob/HEAD/fastvqa/models/swin_backbone.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9313f05ec0453024"}},{"code_sha256_prefix":"95127f2f4ab9c3fa","entry":"global_position_index","repo":"timothyhtimothy/FAST-VQA","repo_kind":"official","path":"fastvqa/models/swin_backbone.py","file_url":"https://github.com/timothyhtimothy/FAST-VQA/blob/HEAD/fastvqa/models/swin_backbone.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"95127f2f4ab9c3fa"}},{"code_sha256_prefix":"2058ed1a750032c1","entry":"BasicLayer","repo":"timothyhtimothy/FAST-VQA","repo_kind":"official","path":"fastvqa/models/swin_backbone.py","file_url":"https://github.com/timothyhtimothy/FAST-VQA/blob/HEAD/fastvqa/models/swin_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2058ed1a750032c1"}},{"code_sha256_prefix":"61ba8901cf0f6fe7","entry":"SwinTransformer3D","repo":"timothyhtimothy/FAST-VQA","repo_kind":"official","path":"fastvqa/models/swin_backbone.py","file_url":"https://github.com/timothyhtimothy/FAST-VQA/blob/HEAD/fastvqa/models/swin_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"61ba8901cf0f6fe7"}},{"code_sha256_prefix":"1bb1dfa32c562fac","entry":"SwinTransformerBlock3D","repo":"timothyhtimothy/FAST-VQA","repo_kind":"official","path":"fastvqa/models/swin_backbone.py","file_url":"https://github.com/timothyhtimothy/FAST-VQA/blob/HEAD/fastvqa/models/swin_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1bb1dfa32c562fac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}