Papers › Example-based Hypernetworks for Out-of-Distribution Generalization

Example-based Hypernetworks for Out-of-Distribution Generalization

27 Mar 2022arXiv:2203.14276archive 2025-07-28

Tomer Volk, Eyal Ben-David, Ohad Amosy, Gal Chechik, Roi Reichart

As Natural Language Processing (NLP) algorithms continually achieve new milestones, out-of-distribution generalization remains a significant challenge. This paper addresses the issue of multi-source adaptation for unfamiliar domains: We leverage labeled data from multiple source domains to generalize to unknown target domains at training. Our innovative framework employs example-based Hypernetwork adaptation: a T5 encoder-decoder initially generates a unique signature from an input example, embedding it within the source domains' semantic space. This signature is subsequently utilized by a Hypernetwork to generate the task classifier's weights. We evaluated our method across two tasks - sentiment classification and natural language inference - in 29 adaptation scenarios, where it outpaced established algorithms. In an advanced version, the signature also enriches the input example's representation. We also compare our finetuned architecture to few-shot GPT-3, demonstrating its effectiveness in essential use cases. To our knowledge, this marks the first application of Hypernetworks to the adaptation for unknown domains.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

tomervolk/hyper-pada officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Decoder Domain Adaptation Natural Language Inference Out-of-Distribution Generalization Sentiment Analysis Sentiment Classification

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Adafactor Attention Attention Dropout BPE Dense Connections Dropout Gated Linear Unit HyperNetwork Inverse Square Root Schedule Layer Normalization Linear Layer Multi-Head Attention Residual Connection SentencePiece Softmax T5

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections