Papers › An Extended Study of Human-like Behavior under Adversarial Training

An Extended Study of Human-like Behavior under Adversarial Training

22 Mar 2023arXiv:2303.12669archive 2025-07-28

Paul Gavrikov, Janis Keuper, Margret Keuper

Neural networks have a number of shortcomings. Amongst the severest ones is the sensitivity to distribution shifts which allows models to be easily fooled into wrong predictions by small perturbations to inputs that are often imperceivable to humans and do not have to carry semantic meaning. Adversarial training poses a partial solution to address this issue by training models on worst-case perturbations. Yet, recent work has also pointed out that the reasoning in neural networks is different from humans. Humans identify objects by shape, while neural nets mainly employ texture cues. Exemplarily, a model trained on photographs will likely fail to generalize to datasets containing sketches. Interestingly, it was also shown that adversarial training seems to favorably increase the shift toward shape bias. In this work, we revisit this observation and provide an extensive analysis of this effect on various architectures, the common ℓ₂- and ℓ_∞-training, and Transformer-based models. Further, we provide a possible explanation for this phenomenon from a frequency perspective.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

paulgavrikov/adversarial_training_vs_humans officialmentioned on GitHubCC-BY-SA-4.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 ConvolutionAbsolute Position EncodingsAdamAttentionAverage PoolingBPEBatch NormalizationBottleneck Residual BlockConvolutionDense ConnectionsDropoutGlobal Average PoolingKaiming InitializationLabel SmoothingLayer NormalizationLinear LayerMax PoolingMulti-Head AttentionPosition-Wise Feed-Forward LayerReLUResidual BlockResidual ConnectionSoftmaxTransformerWide Residual BlockWideResNetfail

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections