Papers › Multi-task Self-Supervised Visual Learning
Multi-task Self-Supervised Visual Learning
Carl Doersch, Andrew Zisserman
We investigate methods for combining multiple self-supervised tasks--i.e., supervised tasks where data can be collected without manual labeling--in order to train a single visual representation. First, we provide an apples-to-apples comparison of four different self-supervised tasks using the very deep ResNet-101 architecture. We then combine tasks to jointly train a network. We also explore lasso regularization to encourage the network to factorize the information in its representation, and methods for "harmonizing" network inputs in order to learn a more unified representation. We evaluate all methods on ImageNet classification, PASCAL VOC detection, and NYU depth prediction. Our results show that deeper networks work better, and that combining tasks--even via a naive multi-head architecture--always improves performance. Our best joint network nearly matches the PASCAL performance of a model pre-trained on ImageNet classification, and matches the ImageNet network on NYU depth prediction.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Self-Supervised Image Classification | ImageNet | Colorisation (improved) (ResNet-101) | Number of Params | 44M | #139 of 144 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | Colorisation (improved) (ResNet-101) | Top 1 Accuracy | 39.6 | #139 of 144 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | Colorisation (improved) (ResNet-101) | Top 5 Accuracy | 62.5 | #139 of 144 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | Multi-task SSL (ResNet-101) | Number of Params | 44M | #144 of 144 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | Multi-task SSL (ResNet-101) | Top 5 Accuracy | 70.2 | #144 of 144 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections