</> Code with Papers by Syntology

Papers › SAG-ViT: A Scale-Aware, High-Fidelity Patching Approach with Graph Attention for...

SAG-ViT: A Scale-Aware, High-Fidelity Patching Approach with Graph Attention for Vision Transformers

14 Nov 2024arXiv:2411.09420archive 2025-07-28

Shravan Venkatraman, Jaskaran Singh Walia, Joe Dhanith P R

Vision Transformers (ViTs) have redefined image classification by leveraging self-attention to capture complex patterns and long-range dependencies between image patches. However, a key challenge for ViTs is efficiently incorporating multi-scale feature representations, which is inherent in convolutional neural networks (CNNs) through their hierarchical structure. Graph transformers have made strides in addressing this by leveraging graph-based modeling, but they often lose or insufficiently represent spatial hierarchies, especially since redundant or less relevant areas dilute the image's contextual representation. To bridge this gap, we propose SAG-ViT, a Scale-Aware Graph Attention ViT that integrates multi-scale feature capabilities of CNNs, representational power of ViTs, graph-attended patching to enable richer contextual representation. Using EfficientNetV2 as a backbone, the model extracts multi-scale feature maps, dividing them into patches to preserve richer semantic information compared to directly patching the input images. The patches are structured into a graph using spatial and feature similarities, where a Graph Attention Network (GAT) refines the node embeddings. This refined graph representation is then processed by a Transformer encoder, capturing long-range dependencies and complex interactions. We evaluate SAG-ViT on benchmark datasets across various domains, validating its effectiveness in advancing image classification tasks. Our code and weights are available at https://github.com/shravan-18/SAG-ViT.

PaperPDFCode

In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

shravan-18/SAG-ViT officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Graph AttentionImage Classificationimage-classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Classification CIFAR-10 SAG-ViT F1 95.74 #265 of 265 Archive leaderboard report
Image Classification GTSRB SAG-ViT F1 99.58 #1 of 1 Archive leaderboard report
Image Classification NCT-CRC-HE-100K SAG-ViT F1 98.61 #1 of 1 Archive leaderboard report
Image Classification RESISC45 SAG-ViT F1 95.49 #19 of 20 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAbsolute Position EncodingsAdamAttentionAverage PoolingBPEBatch NormalizationConvolutionDense ConnectionsDepthwise ConvolutionDepthwise Separable ConvolutionDropoutEfficientNetEfficientNetV2Inverted Residual BlockLabel SmoothingLayer NormalizationLinear LayerMulti-Head AttentionPatchingPointwise ConvolutionPosition-Wise Feed-Forward LayerRMSPropReLUResidual ConnectionSigmoid ActivationSoftmaxSqueeze-and-Excitation BlockTransformerVision Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections