Papers › Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection

Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection

1 Dec 2018NeurIPS 2018 12archive 2025-07-28

Taylor Mordan, Nicolas Thome, Gilles Henaff, Matthieu Cord

Multi-Task Learning (MTL) is appealing for deep learning regularization. In this paper, we tackle a specific MTL context denoted as primary MTL, where the ultimate goal is to improve the performance of a given primary task by leveraging several other auxiliary tasks. Our main methodological contribution is to introduce ROCK, a new generic multi-modal fusion block for deep learning tailored to the primary MTL context. ROCK architecture is based on a residual connection, which makes forward prediction explicitly impacted by the intermediate auxiliary representations. The auxiliary predictor's architecture is also specifically designed to our primary MTL context, by incorporating intensive pooling operators for maximizing complementarity of intermediate representations. Extensive experiments on NYUv2 dataset (object detection with scene classification, depth prediction, and surface normal estimation as auxiliary tasks) validate the relevance of the approach and its superiority to flat MTL approaches. Our method outperforms state-of-the-art object detection models on NYUv2 dataset by a large margin, and is also able to handle large-scale heterogeneous inputs (real and synthetic images) with missing annotation modalities.

Paper PDF Code

Code

vita-epfl/rock-pytorch pytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Depth Estimation Depth Prediction Multi-Task Learning Object Detection Scene Classification Surface Normal Estimation object-detection

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Global Average Pooling Kaiming Initialization Max Pooling Non Maximum Suppression ReLU Residual Block Residual Connection SSD

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections