{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spectral-convolutional-transformer","title":"Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysis","arxiv_id":"2403.18063","date":"2024-03-26","proceeding":null,"authors":["Badri N. Patro","Suhas Ranganath","Vinay P. Namboodiri","Vijay S. Agneeswaran"],"abstract":"Transformers have revolutionized image modeling tasks with adaptations like DeIT, Swin, SVT, Biformer, STVit, and FDVIT. However, these models often face challenges with inductive bias and high quadratic complexity, making them less efficient for high-resolution images. State space models (SSMs) such as Mamba, V-Mamba, ViM, and SiMBA offer an alternative to handle high resolution images in computer vision tasks. These SSMs encounter two major issues. First, they become unstable when scaled to large network sizes. Second, although they efficiently capture global information in images, they inherently struggle with handling local information. To address these challenges, we introduce Heracles, a novel SSM that integrates a local SSM, a global SSM, and an attention-based token interaction module. Heracles leverages a Hartely kernel-based state space model for global image information, a localized convolutional network for local details, and attention mechanisms in deeper layers for token interactions. Our extensive experiments demonstrate that Heracles-C-small achieves state-of-the-art performance on the ImageNet dataset with 84.5\\% top-1 accuracy. Heracles-C-Large and Heracles-C-Huge further improve accuracy to 85.9\\% and 86.4\\%, respectively. Additionally, Heracles excels in transfer learning tasks on datasets such as CIFAR-10, CIFAR-100, Oxford Flowers, and Stanford Cars, and in instance segmentation on the MSCOCO dataset. Heracles also proves its versatility by achieving state-of-the-art results on seven time-series datasets, showcasing its ability to generalize across domains with spectral data, capturing both local and global information. The project page is available at this link.\\url{https://github.com/badripatro/heracles}","url_abs":"https://arxiv.org/abs/2403.18063v2","url_pdf":"https://arxiv.org/pdf/2403.18063v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spectral-convolutional-transformer","repo_url":"https://github.com/badripatro/heracles","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"spectral-convolutional-transformer","repo_url":"https://github.com/badripatro/sct","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"state-space-models","task_name":"State Space Models"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"cpvt","method_name":"CPVT"},{"method_slug":"conditional-positional-encoding","method_name":"Conditional Positional Encoding"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"cvt","method_name":"CvT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pvt","method_name":"PVT"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"positional-encoding-generator","method_name":"Positional Encoding Generator"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatial-reduction-attention","method_name":"Spatial-Reduction Attention"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.18063","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.18063"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/badripatro/sct","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/badripatro/heracles","reach":{"status":"ok"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"bbff8d1aecb5fc69","entry":"build_transform","repo":"badripatro/heracles","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/badripatro/heracles/blob/HEAD/datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bbff8d1aecb5fc69"}},{"code_sha256_prefix":"6cc5ed3b37c68d9d","entry":"pil_loader","repo":"badripatro/heracles","repo_kind":"official","path":"mcloader/mcloader.py","file_url":"https://github.com/badripatro/heracles/blob/HEAD/mcloader/mcloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6cc5ed3b37c68d9d"}},{"code_sha256_prefix":"d84653ce829572b8","entry":"rand_bbox","repo":"badripatro/heracles","repo_kind":"official","path":"heracles.py","file_url":"https://github.com/badripatro/heracles/blob/HEAD/heracles.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d84653ce829572b8"}},{"code_sha256_prefix":"63dfd023b306bac7","entry":"setup_logger","repo":"badripatro/heracles","repo_kind":"official","path":"utils.py","file_url":"https://github.com/badripatro/heracles/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63dfd023b306bac7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}