{"url":"/method/nest","slug":"nest","name":"NesT","full_name":"NesT","full_name_withheld":false,"description_markdown":"**NesT** stacks canonical transformer layers to conduct local self-attention on every image block independently, and then \"nests\" them hierarchically. Coupling of processed information between spatially adjacent blocks is achieved through a proposed block aggregation between every two hierarchies. The overall hierarchical structure can be determined by two key hyper-parameters: patch size $S × S$ and number of block hierarchies $T_d$. All blocks inside each hierarchy share one set of parameters. Given input of image, each image is linearly projected to an embedding. All embeddings are partitioned to blocks and flattened to generate final input. Each transformer layers is composed of a multi-head self attention (MSA) layer followed by a feed-forward fully-connected network (FFN) with skip-connection and Layer normalization. Positional embeddings are added to encode spatial information before feeding into the block. Lastly, a nested hierarchy with block aggregation is built -- every four spatially connected blocks are merged into one.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding","paper":"/paper/aggregating-nested-transformers","first_author":"Zizhao Zhang","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/aggregating-nested-transformers"},"source":{"url":"https://arxiv.org/abs/2105.12723v4","title":"Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Vision Transformers","url":"/methods/category/vision-transformers","pwc_aliases":["vision-transformer"]}],"n_papers_tagged":37,"archive_num_papers":37,"papers_newest_first":[{"paper":"/paper/nestquant-post-training-integer-nesting","title":"NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN","date":"2025-06-22","arxiv_id":"2506.17870","n_code_links":1,"syntology":null},{"paper":null,"title":"Mixed-feedback oscillations in the foraging dynamics of arboreal turtle ants","date":"2025-04-15","arxiv_id":"2504.11551","n_code_links":0,"syntology":null},{"paper":null,"title":"Experimental Validation of Distributed Dispatching of Multiple Active Distribution Networks Using the ADMM","date":"2025-03-19","arxiv_id":"2503.15356","n_code_links":0,"syntology":null},{"paper":null,"title":"Ant Nest Detection Using Underground P-Band TomoSAR","date":"2024-12-16","arxiv_id":"2412.11865","n_code_links":0,"syntology":null},{"paper":null,"title":"NEST: A Neuromodulated Small-world Hypergraph Trajectory Prediction Model for Autonomous Driving","date":"2024-12-16","arxiv_id":"2412.11682","n_code_links":0,"syntology":null},{"paper":null,"title":"Argentine ants regulate traffic flow with stopped individuals","date":"2024-12-09","arxiv_id":"2412.06587","n_code_links":0,"syntology":null},{"paper":null,"title":"Revisiting the Many Instruments Problem using Random Matrix Theory","date":"2024-08-16","arxiv_id":"2408.08580","n_code_links":0,"syntology":null},{"paper":null,"title":"Performance Prediction of Hub-Based Swarms","date":"2024-08-09","arxiv_id":"2408.04822","n_code_links":0,"syntology":null},{"paper":null,"title":"Soli-enabled Noncontact Heart Rate Detection for Sleep and Meditation Tracking","date":"2024-07-08","arxiv_id":"2407.06458","n_code_links":0,"syntology":null},{"paper":null,"title":"NeST: Neural Stress Tensor Tomography by leveraging 3D Photoelasticity","date":"2024-06-14","arxiv_id":"2406.10212","n_code_links":0,"syntology":null},{"paper":null,"title":"Nearest Neighbor Speculative Decoding for LLM Generation and Attribution","date":"2024-05-29","arxiv_id":"2405.19325","n_code_links":0,"syntology":null},{"paper":null,"title":"Direct learning of home vector direction for insect-inspired robot navigation","date":"2024-05-06","arxiv_id":"2405.03827","n_code_links":0,"syntology":null},{"paper":"/paper/looper-a-learned-automatic-code-optimizer-for","title":"LOOPer: A Learned Automatic Code Optimizer For Polyhedral Compilers","date":"2024-03-18","arxiv_id":"2403.11522","n_code_links":1,"syntology":null},{"paper":"/paper/drattack-prompt-decomposition-and","title":"DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers","date":"2024-02-25","arxiv_id":"2402.16914","n_code_links":1,"syntology":{"ran":4,"of":7,"unverified":3,"pointer_only":0}},{"paper":null,"title":"Norm Enforcement with a Soft Touch: Faster Emergence, Happier Agents","date":"2024-01-29","arxiv_id":"2401.16461","n_code_links":0,"syntology":null},{"paper":null,"title":"Timing decisions as the next frontier for collective intelligence","date":"2023-12-01","arxiv_id":"2312.02187","n_code_links":0,"syntology":null},{"paper":"/paper/insplad-a-dataset-and-benchmark-for-power","title":"InsPLAD: A Dataset and Benchmark for Power Line Asset Inspection in UAV Images","date":"2023-11-02","arxiv_id":"2311.01619","n_code_links":1,"syntology":null},{"paper":null,"title":"Robustify and Tighten the Lee Bounds: A Sample Selection Model under Stochastic Monotonicity and Symmetry Assumptions","date":"2023-11-01","arxiv_id":"2311.00439","n_code_links":0,"syntology":null},{"paper":null,"title":"Interpreting TSLS Estimators in Information Provision Experiments","date":"2023-09-09","arxiv_id":"2309.04793","n_code_links":0,"syntology":null},{"paper":null,"title":"Finite population effects on optimal communication for social foragers","date":"2023-08-01","arxiv_id":"2308.00298","n_code_links":0,"syntology":null},{"paper":"/paper/inference-for-relative-sparsity","title":"Inference for relative sparsity","date":"2023-06-25","arxiv_id":"2306.14297","n_code_links":1,"syntology":null},{"paper":null,"title":"Deep Huber quantile regression networks","date":"2023-06-17","arxiv_id":"2306.10306","n_code_links":0,"syntology":null},{"paper":"/paper/superneuro-a-fast-and-scalable-simulator-for","title":"SuperNeuro: A Fast and Scalable Simulator for Neuromorphic Computing","date":"2023-05-04","arxiv_id":"2305.02510","n_code_links":1,"syntology":null},{"paper":null,"title":"Physics Symbolic Learner for Discovering Ground-Motion Models Via NGA-West2 Database","date":"2023-03-23","arxiv_id":"2303.14179","n_code_links":0,"syntology":null},{"paper":"/paper/neuron-structure-modeling-for-generalizable","title":"Neuron Structure Modeling for Generalizable Remote Physiological Measurement","date":"2023-03-10","arxiv_id":"2303.05955","n_code_links":1,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":2}},{"paper":"/paper/emergent-communication-enhances-foraging","title":"Emergent communication enhances foraging behaviour in evolved swarms controlled by Spiking Neural Networks","date":"2022-12-16","arxiv_id":"2212.08484","n_code_links":1,"syntology":null},{"paper":"/paper/uncovering-the-dynamics-of-the-wealth","title":"Uncovering the Dynamics of the Wealth Distribution","date":"2022-11-24","arxiv_id":"2211.15509","n_code_links":1,"syntology":null},{"paper":null,"title":"Interpreting Instrumental Variable Estimands with Unobserved Treatment Heterogeneity: The Effects of College Education","date":"2022-11-23","arxiv_id":"2211.13132","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-organizing nest migration dynamics synthesis for ant colony systems","date":"2022-10-08","arxiv_id":"2210.03975","n_code_links":0,"syntology":null},{"paper":"/paper/tensorir-an-abstraction-for-automatic","title":"TensorIR: An Abstraction for Automatic Tensorized Program Optimization","date":"2022-07-09","arxiv_id":"2207.04296","n_code_links":2,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/regression-1","name":"regression","papers":3},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/prediction","name":"Prediction","papers":2},{"task":"/task/anatomy","name":"Anatomy","papers":1},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/audio-denoising","name":"Audio Denoising","papers":1},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":1},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1},{"task":"/task/benchmarking","name":"Benchmarking","papers":1},{"task":"/task/decision-making","name":"Decision Making","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/domain-generalization","name":"Domain Generalization","papers":1},{"task":"/task/event-detection","name":"Event Detection","papers":1},{"task":"/task/evolutionary-algorithms","name":"Evolutionary Algorithms","papers":1},{"task":null,"name":"GPU","papers":1},{"task":"/task/image-generation","name":"Image Generation","papers":1},{"task":"/task/in-context-learning","name":"In-Context Learning","papers":1}],"tasks_shown":20,"n_tasks":35,"usage_by_year":[{"year":"2021","papers":5},{"year":"2022","papers":7},{"year":"2023","papers":10},{"year":"2024","papers":12},{"year":"2025","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/nest"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}