{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/auto-encoding-score-distribution-regression","title":"Auto-Encoding Score Distribution Regression for Action Quality Assessment","arxiv_id":"2111.11029","date":"2021-11-22","proceeding":null,"authors":["Boyu Zhang","Jiayuan Chen","Yinfei Xu","HUI ZHANG","Xu Yang","Xin Geng"],"abstract":"The action quality assessment (AQA) of videos is a challenging vision task since the relation between videos and action scores is difficult to model. Thus, AQA has been widely studied in the literature. Traditionally, AQA is treated as a regression problem to learn the underlying mappings between videos and action scores. But previous methods ignored data uncertainty in AQA dataset. To address aleatoric uncertainty, we further develop a plug-and-play module Distribution Auto-Encoder (DAE). Specifically, it encodes videos into distributions and uses the reparameterization trick in variational auto-encoders (VAE) to sample scores, which establishes a more accurate mapping between videos and scores. Meanwhile, a likelihood loss is used to learn the uncertainty parameters. We plug our DAE approach into MUSDL and CoRe. Experimental results on public datasets demonstrate that our method achieves state-of-the-art on AQA-7, MTL-AQA, and JIGSAWS datasets. Our code is available at https://github.com/InfoX-SEU/DAE-AQA.","url_abs":"https://arxiv.org/abs/2111.11029v2","url_pdf":"https://arxiv.org/pdf/2111.11029v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"auto-encoding-score-distribution-regression","repo_url":"https://github.com/InfoX-SEU/DAE-AQA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"auto-encoding-score-distribution-regression","repo_url":"https://github.com/InfoX-SEU/DAE_AQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"auto-encoding-score-distribution-regression","repo_url":"https://github.com/luciferbobo/dae-aqa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-quality-assessment","task_name":"Action Quality Assessment"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-quality-assessment-on-aqa-7","task":"Action Quality Assessment","dataset":"AQA-7","model":"DAE-CoRe","rank_in_archive_order":1,"of":9,"metrics":{"Spearman Correlation":"85.20%"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-aqa-7","task":"Action Quality Assessment","dataset":"AQA-7","model":"DAE-MLP","rank_in_archive_order":3,"of":9,"metrics":{"Spearman Correlation":"82.58%"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-jigsaws","task":"Action Quality Assessment","dataset":"JIGSAWS","model":"DAE-CoRe","rank_in_archive_order":3,"of":5,"metrics":{"Spearman Correlation":"0.86"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-jigsaws","task":"Action Quality Assessment","dataset":"JIGSAWS","model":"DAE-MT","rank_in_archive_order":4,"of":5,"metrics":{"Spearman Correlation":"0.76"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-jigsaws","task":"Action Quality Assessment","dataset":"JIGSAWS","model":"DAE-MLP","rank_in_archive_order":5,"of":5,"metrics":{"Spearman Correlation":"0.72"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-mtl-aqa","task":"Action Quality Assessment","dataset":"MTL-AQA","model":"DAE-CoRe","rank_in_archive_order":4,"of":21,"metrics":{"Spearman Correlation":"95.89"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-mtl-aqa","task":"Action Quality Assessment","dataset":"MTL-AQA","model":"DAE-MT","rank_in_archive_order":7,"of":21,"metrics":{"Spearman Correlation":"94.52"},"uses_additional_data":false},{"leaderboard":"/sota/action-quality-assessment-on-mtl-aqa","task":"Action Quality Assessment","dataset":"MTL-AQA","model":"DAE-MLP","rank_in_archive_order":13,"of":21,"metrics":{"Spearman Correlation":"92.31"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.11029","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}