{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-19849","title":"HexFormer: Hyperbolic Vision Transformer with Exponential Map Aggregation","arxiv_id":"2601.19849","date":"2026-01-27","proceeding":null,"authors":["Haya Alyoussef","Ahmad Bdeir","Diego Coello de Portugal Mecke","Tom Hanika","Niels Landwehr","Lars Schmidt-Thieme"],"abstract":"Data across modalities such as images, text, and graphs often contains hierarchical and relational structures, which are challenging to model within Euclidean geometry. Hyperbolic geometry provides a natural framework for representing such structures. Building on this property, this work introduces HexFormer, a hyperbolic vision transformer for image classification that incorporates exponential map aggregation within its attention mechanism. Two designs are explored: a hyperbolic ViT (HexFormer) and a hybrid variant (HexFormer-Hybrid) that combines a hyperbolic encoder with an Euclidean linear classification head. HexFormer incorporates a novel attention mechanism based on exponential map aggregation, which yields more accurate and stable aggregated representations than standard centroid based averaging, showing that simpler approaches retain competitive merit. Experiments across multiple datasets demonstrate consistent performance improvements over Euclidean baselines and prior hyperbolic ViTs, with the hybrid variant achieving the strongest overall results. Additionally, this study provides an analysis of gradient stability in hyperbolic transformers. The results reveal that hyperbolic models exhibit more stable gradients and reduced sensitivity to warmup strategies compared to Euclidean architectures, highlighting their robustness and efficiency in training. Overall, these findings indicate that hyperbolic geometry can enhance vision transformer architectures by improving gradient stability and accuracy. In addition, relatively simple mechanisms such as exponential map aggregation can provide strong practical benefits.","url_abs":"https://arxiv.org/abs/2601.19849","url_pdf":"https://arxiv.org/pdf/2601.19849","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2601.19849","atlas_url":"https://app.syntology.ai/?focus=2601.19849","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.19849"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/HayaAlyoussef/HexFormer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98350b92a936748b","entry":"insert_docs","repo":"HayaAlyoussef/HexFormer","repo_kind":"found_in_text","path":"lib/geoopt/docutils.py","file_url":"https://github.com/HayaAlyoussef/HexFormer/blob/HEAD/lib/geoopt/docutils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98350b92a936748b"}},{"code_sha256_prefix":"19896c4283eab1ec","entry":"make_tuple","repo":"HayaAlyoussef/HexFormer","repo_kind":"found_in_text","path":"lib/geoopt/utils.py","file_url":"https://github.com/HayaAlyoussef/HexFormer/blob/HEAD/lib/geoopt/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19896c4283eab1ec"}},{"code_sha256_prefix":"d4e7ce8697742d11","entry":"prod","repo":"HayaAlyoussef/HexFormer","repo_kind":"found_in_text","path":"lib/geoopt/utils.py","file_url":"https://github.com/HayaAlyoussef/HexFormer/blob/HEAD/lib/geoopt/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4e7ce8697742d11"}},{"code_sha256_prefix":"47af2deb6b9eac9c","entry":"strip_tuple","repo":"HayaAlyoussef/HexFormer","repo_kind":"found_in_text","path":"lib/geoopt/utils.py","file_url":"https://github.com/HayaAlyoussef/HexFormer/blob/HEAD/lib/geoopt/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47af2deb6b9eac9c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}