{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/label-ranker-self-aware-preference-for","title":"Label Ranker: Self-Aware Preference for Classification Label Position in Visual Masked Self-Supervised Pre-Trained Model","arxiv_id":null,"date":"2025-03-03","proceeding":"Preprints.org 2025 3","authors":["Peihao Xiang","Kaida Wu","Ou Bai"],"abstract":"This paper investigates the impact of randomly initialized unique encoding of classification label position on the visual masked self-supervised pre-trained model when fine-tuning downstream classification tasks. Our findings indicate that different random initializations lead to significant variations in fine-tuned results, even when using the same allocation strategy for classification datasets. The accuracy gap between these results suggests that the visual masked self-supervised pre-trained model has an inherent preference for classification label positions. To investigate this, we compare it with the non-self-supervised visual pre-trained model and hypothesize that the masked self-supervised model exhibits a self-aware bias toward certain label positions. To mitigate the instability caused by random encoding, we propose a classification label position ranking algorithm, Label Ranker. It is based on 1-D dimensionality reduction of feature maps using Linear Discriminant Analysis and position-rank encoding of them by unsupervised feature clustering using the similarity property of Euclidean distance. This algorithm ensures that label position encoding align with the model’s inherent preference. Extensive ablation experiments using ImageMAE and VideoMAE models on the CIFAR-100, UCF101, and HMDB51 classification datasets validate our approach. Results demonstrate that our method effectively stabilizes classification label position encoding, improving fine-tuned performance for visual masked self-supervised models.","url_abs":"https://www.preprints.org/manuscript/202503.0003/v1","url_pdf":"https://www.preprints.org/frontend/manuscript/a1bd3a89aef7da37c281038909f5fd6d/download_pub","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"label-ranker-self-aware-preference-for","repo_url":"https://github.com/Peihao-Xiang/Label-Ranker","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"dimensionality-reduction","task_name":"Dimensionality Reduction"},{"task_slug":"human-activity-recognition","task_name":"Human Activity Recognition"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"multi-label-learning","task_name":"Multi-Label Learning"},{"task_slug":null,"task_name":"Position"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"policy-similarity-metric","method_name":"Policy Similarity Metric"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"self-learning","method_name":"Self-Learning"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"},{"method_slug":"classifier-guidance","method_name":"classifier-guidance"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-activity-recognition-on-hmdb51","task":"Human Activity Recognition","dataset":"HMDB51","model":"Label-Ranker","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"61.18%"},"uses_additional_data":false},{"leaderboard":"/sota/human-activity-recognition-on-ucf-101","task":"Human Activity Recognition","dataset":"UCF 101","model":"Label-Ranker","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"89.50%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"Label-Ranker","rank_in_archive_order":210,"of":211,"metrics":{"Accuracy":"90.82%"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}