Papers › Label Ranker: Self-Aware Preference for Classification Label Position in Visual Masked...

Label Ranker: Self-Aware Preference for Classification Label Position in Visual Masked Self-Supervised Pre-Trained Model

3 Mar 2025Preprints.org 2025 3archive 2025-07-28

Peihao Xiang, Kaida Wu, Ou Bai

This paper investigates the impact of randomly initialized unique encoding of classification label position on the visual masked self-supervised pre-trained model when fine-tuning downstream classification tasks. Our findings indicate that different random initializations lead to significant variations in fine-tuned results, even when using the same allocation strategy for classification datasets. The accuracy gap between these results suggests that the visual masked self-supervised pre-trained model has an inherent preference for classification label positions. To investigate this, we compare it with the non-self-supervised visual pre-trained model and hypothesize that the masked self-supervised model exhibits a self-aware bias toward certain label positions. To mitigate the instability caused by random encoding, we propose a classification label position ranking algorithm, Label Ranker. It is based on 1-D dimensionality reduction of feature maps using Linear Discriminant Analysis and position-rank encoding of them by unsupervised feature clustering using the similarity property of Euclidean distance. This algorithm ensures that label position encoding align with the model’s inherent preference. Extensive ablation experiments using ImageMAE and VideoMAE models on the CIFAR-100, UCF101, and HMDB51 classification datasets validate our approach. Results demonstrate that our method effectively stabilizes classification label position encoding, improving fine-tuned performance for visual masked self-supervised models.

Paper PDF Code

Code

Peihao-Xiang/Label-Ranker tf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Classification Dimensionality Reduction Human Activity Recognition Image Classification Multi-Label Learning Semantic Similarity

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Human Activity Recognition	HMDB51	Label-Ranker	Accuracy	61.18%	#1 of 1	Archive leaderboard	report
Human Activity Recognition	UCF 101	Label-Ranker	Accuracy	89.50%	#1 of 1	Archive leaderboard	report
Image Classification	CIFAR-100	Label-Ranker	Accuracy	90.82%	#210 of 211	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

ALIGN Attention Cosine Annealing Dense Connections Layer Normalization Linear Layer Linear Warmup With Cosine Annealing Multi-Head Attention Policy Similarity Metric Residual Connection Self-Learning Softmax Vision Transformer classifier-guidance

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections