Papers › An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask...

An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems

25 May 2022arXiv:2205.12755archive 2025-07-28

Andrea Gesmundo, Jeff Dean

Multitask learning assumes that models capable of learning from multiple tasks can achieve better quality and efficiency via knowledge transfer, a key feature of human learning. Though, state of the art ML models rely on high customization for each task and leverage size and data scale rather than scaling the number of tasks. Also, continual learning, that adds the temporal aspect to multitask, is often focused to the study of common pitfalls such as catastrophic forgetting instead of being studied at a large scale as a critical component to build the next generation artificial intelligence.We propose an evolutionary method capable of generating large scale multitask models that support the dynamic addition of new tasks. The generated multitask models are sparsely activated and integrates a task-based routing that guarantees bounded compute cost and fewer added parameters per task as the model expands.The proposed method relies on a knowledge compartmentalization technique to achieve immunity against catastrophic forgetting and other common pitfalls such as gradient interference and negative transfer. We demonstrate empirically that the proposed method can jointly solve and achieve competitive results on 69public image classification tasks, for example improving the state of the art on a competitive benchmark such as cifar10 by achieving a 15% relative error reduction compared to the best model trained on public data.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

google-research/google-research officialjaxSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Efficient Neural Architecture Search via Parameter Sharing (arXiv:1802.03268; that paper's own run record: 18 ran (of which 10 constructed an object rather than computing a result; 12 with no instrument failure: 1 honoured, 0 violated, 11 with no contract checked; 6 where Syntology's instrument failed) · 20 unverified), Milking CowMask for Semi-Supervised Image Classification (arXiv:2003.12022; that paper's own run record: 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result), Neural Additive Models: Interpretable Machine Learning with Neural Nets (arXiv:2004.13912; that paper's own run record: 26 ran (of which 14 constructed an object rather than computing a result; 26 with no instrument failure: 5 honoured, 0 violated, 21 with no contract checked; 0 where Syntology's instrument failed) · 7 unverified) and 21 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Continual LearningFine-Grained Image ClassificationImage ClassificationTransfer Learningimage-classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Fine-Grained Image Classification Caltech-101 µ2Net (ViT-L/16) Top-1 Error Rate 7% #8 of 18 Archive leaderboard report
Fine-Grained Image Classification Oxford 102 Flowers µ2Net (ViT-L/16) Accuracy 99.61% #3 of 25 Archive leaderboard report
Fine-Grained Image Classification Oxford-IIIT Pets µ2Net (ViT-L/16) Accuracy 95.3 #4 of 19 Archive leaderboard report
Fine-Grained Image Classification SUN397 µ2Net (ViT-L/16) Accuracy 84.8 #1 of 5 Archive leaderboard report
Image Classification CIFAR-10 µ2Net (ViT-L/16) Percentage correct 99.49 #3 of 265 Archive leaderboard report
Image Classification CIFAR-100 µ2Net (ViT-L/16) Percentage correct 94.95 #3 of 211 Archive leaderboard report
Image Classification DTD µ2Net (ViT-L/16) Accuracy 81.0 #5 of 11 Archive leaderboard report
Image Classification EMNIST-Digits µ2Net (ViT-L/16) Accuracy (%) 99.82 #2 of 7 Archive leaderboard report
Image Classification EuroSAT µ2Net (ViT-L/16) Accuracy (%) 99.2 #4 of 15 Archive leaderboard report
Image Classification ImageNet µ2Net (ViT-L/16) Top 1 Accuracy 86.74% #125 of 1060 Archive leaderboard report
Image Classification KMNIST µ2Net (ViT-L/16) Accuracy 98.68 #1 of 1 Archive leaderboard report
Image Classification MNIST µ2Net (ViT-L/16) Accuracy 99.75 #64 of 81 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections