{"url":"/method/ghost-module","slug":"ghost-module","name":"Ghost Module","full_name":"Ghost Module","full_name_withheld":false,"description_markdown":"A **Ghost Module** is an image block for convolutional neural network that aims to generate more features by using fewer parameters. Specifically, an ordinary convolutional layer in deep neural networks is split into two parts. The first part involves ordinary convolutions but their total number is controlled. Given the intrinsic feature maps from the first part, a series of simple linear operations are applied for generating more feature maps. \r\n\r\nGiven the widely existing redundancy in intermediate feature maps calculated by mainstream CNNs, ghost modules aim to reduce them. In practice, given the input data $X\\in\\mathbb{R}^{c\\times h\\times w}$, where $c$ is the number of input channels and $h$ and $w$ are the height and width of the input data, respectively,  the operation of an arbitrary convolutional layer for producing $n$ feature maps can be formulated as\r\n\r\n$$\r\nY = X*f+b,\r\n$$\r\n\r\nwhere $*$ is the [convolution](https://paperswithcode.com/method/convolution) operation, $b$ is the bias term, $Y\\in\\mathbb{R}^{h'\\times w'\\times n}$ is the output feature map with $n$ channels, and $f\\in\\mathbb{R}^{c\\times k\\times k \\times n}$ is the convolution filters in this layer. In addition, $h'$ and $w'$ are the height and width of the output data, and $k\\times k$ is the kernel size of convolution filters $f$, respectively. During this convolution procedure, the required number of FLOPs can be calculated as $n\\cdot h'\\cdot w'\\cdot c\\cdot k\\cdot k$, which is often as large as hundreds of thousands since the number of filters $n$ and the channel number $c$ are generally very large (e.g. 256 or 512).\r\n\r\nHere, the number of parameters (in $f$ and $b$) to be optimized is explicitly determined by the dimensions of input and output feature maps. The output feature maps of convolutional layers often contain much redundancy, and some of them could be similar with each other. We point out that it is unnecessary to generate these redundant feature maps one by one with large number of FLOPs and parameters. Suppose that the output feature maps are *ghosts* of a handful of intrinsic feature maps with some cheap transformations. These intrinsic feature maps are often of smaller size and produced by ordinary convolution filters. Specifically, $m$ intrinsic feature maps $Y'\\in\\mathbb{R}^{h'\\times w'\\times m}$ are generated using a primary convolution:\r\n\r\n$$\r\nY' = X*f',\r\n$$\r\n\r\nwhere $f'\\in\\mathbb{R}^{c\\times k\\times k \\times m}$ is the utilized filters, $m\\leq n$ and the bias term is omitted for simplicity. The hyper-parameters such as filter size, stride, padding, are the same as those in the ordinary convolution to keep the spatial size (ie $h'$ and $w'$) of the output feature maps consistent. To further obtain the desired $n$ feature maps, we apply a series of cheap linear operations on each intrinsic feature in $Y'$ to generate $s$ ghost features according to the following function:\r\n\r\n$$\r\ny_{ij} = \\Phi_{i,j}(y'_i),\\quad \\forall\\; i = 1,...,m,\\;\\; j = 1,...,s,\r\n$$\r\n\r\nwhere $y'\\_i$ is the $i$-th intrinsic feature map in $Y'$, $\\Phi\\_{i,j}$ in the above function is the $j$-th (except the last one) linear operation for generating the $j$-th ghost feature map $y_{ij}$, that is to say, $y'\\_i$ can have one or more ghost feature maps $\\{y\\_{ij}\\}\\_{j=1}^{s}$. The last $\\Phi\\_{i,s}$ is the identity mapping for preserving the intrinsic feature maps. we can obtain $n=m\\cdot s$ feature maps $Y=[y\\_{11},y\\_{12},\\cdots,y\\_{ms}]$ as the output data of a Ghost module. Note that the linear operations $\\Phi$ operate on each channel whose computational cost is much less than the ordinary convolution. In practice, there could be several different linear operations in a Ghost module, eg $3\\times 3$ and $5\\times5$ linear kernels, which will be analyzed in the experiment part.","description_state":"present","introduced_year":null,"introduced_by":{"title":"GhostNet: More Features from Cheap Operations","paper":"/paper/ghostnet-more-features-from-cheap-operations","first_author":"Kai Han","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/ghostnet-more-features-from-cheap-operations"},"source":{"url":"https://arxiv.org/abs/1911.11907v2","title":"GhostNet: More Features from Cheap Operations","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/huawei-noah/ghostnet/blob/e8c449210c833996e14ffcdd3946ecdbb7098fd1/pytorch/ghostnet.py#L76","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Image Model Blocks","url":"/methods/category/image-model-blocks","pwc_aliases":[]}],"n_papers_tagged":30,"archive_num_papers":30,"papers_newest_first":[{"paper":null,"title":"GRNN:Recurrent Neural Network based on Ghost Features for Video Super-Resolution","date":"2025-05-14","arxiv_id":"2505.10577","n_code_links":0,"syntology":null},{"paper":"/paper/cross-video-identity-correlating-for-person","title":"Cross-video Identity Correlating for Person Re-identification Pre-training","date":"2024-09-27","arxiv_id":"2409.18569","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}},{"paper":"/paper/a-lightweight-insulator-defect-detection","title":"A Lightweight Insulator Defect Detection Model Based on Drone Images","date":"2024-08-26","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/idd-yolov5-a-lightweight-insulator-defect","title":"IDD-YOLOv5: A Lightweight Insulator Defect Real-time Detection Algorithm","date":"2024-08-19","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"A lightweight YOLOv5-FFM model for occlusion pedestrian detection","date":"2024-08-13","arxiv_id":"2408.06633","n_code_links":0,"syntology":null},{"paper":"/paper/liteyolo-id-a-lightweight-object-detection","title":"LiteYOLO-ID: A Lightweight Object Detection Network for Insulator Defect Detection","date":"2024-06-24","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Multimodal Emotion Recognition based on Facial Expressions, Speech, and EEG","date":"2024-06-11","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Ghost-Stereo: GhostNet-based Cost Volume Enhancement and Aggregation for Stereo Matching Networks","date":"2024-05-23","arxiv_id":"2405.14520","n_code_links":0,"syntology":null},{"paper":null,"title":"GRAN: Ghost Residual Attention Network for Single Image Super Resolution","date":"2023-02-28","arxiv_id":"2302.14557","n_code_links":0,"syntology":null},{"paper":null,"title":"Short-Term Memory Convolutions","date":"2023-02-08","arxiv_id":"2302.04331","n_code_links":0,"syntology":null},{"paper":"/paper/ghostnetv2-enhance-cheap-operation-with-long","title":"GhostNetV2: Enhance Cheap Operation with Long-Range Attention","date":"2022-11-23","arxiv_id":"2211.12905","n_code_links":12,"syntology":{"ran":8,"of":10,"unverified":2,"pointer_only":10}},{"paper":"/paper/repghost-a-hardware-efficient-ghost-module","title":"RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization","date":"2022-11-11","arxiv_id":"2211.06088","n_code_links":3,"syntology":{"ran":5,"of":17,"unverified":12,"pointer_only":0}},{"paper":"/paper/network-amplification-with-efficient-macs","title":"Network Amplification With Efficient MACs Allocation","date":"2022-07-01","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":null,"title":"YOLOv5s-GTB: light-weighted and improved YOLOv5s for bridge crack detection","date":"2022-06-03","arxiv_id":"2206.01498","n_code_links":0,"syntology":null},{"paper":"/paper/mocovit-mobile-convolutional-vision","title":"MoCoViT: Mobile Convolutional Vision Transformer","date":"2022-05-25","arxiv_id":"2205.12635","n_code_links":1,"syntology":null},{"paper":"/paper/triplenet-a-low-computing-power-platform-of","title":"Efficient Convolutional Neural Networks on Raspberry Pi for Image Classification","date":"2022-04-02","arxiv_id":"2204.00943","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":0}},{"paper":"/paper/threshnet-an-efficient-densenet-using","title":"ThreshNet: An Efficient DenseNet Using Threshold Mechanism to Reduce Connections","date":"2022-01-09","arxiv_id":"2201.03013","n_code_links":1,"syntology":null},{"paper":"/paper/gpu-net-lightweight-u-net-with-more-diverse","title":"GPU-Net: Lightweight U-Net with more diverse features","date":"2022-01-07","arxiv_id":"2201.02656","n_code_links":1,"syntology":null},{"paper":null,"title":"Ghost-dil-NetVLAD: A Lightweight Neural Network for Visual Place Recognition","date":"2021-12-22","arxiv_id":"2112.11679","n_code_links":0,"syntology":null},{"paper":"/paper/ghostshiftaddnet-more-features-from-energy","title":"GhostShiftAddNet: More Features from Energy-Efficient Operations","date":"2021-09-20","arxiv_id":"2109.09495","n_code_links":3,"syntology":null},{"paper":null,"title":"GhostBERT: Generate More Features with Cheap Operations for BERT","date":"2021-08-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/greedy-network-enlarging","title":"Greedy Network Enlarging","date":"2021-07-31","arxiv_id":"2108.00177","n_code_links":1,"syntology":null},{"paper":null,"title":"AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition","date":"2021-02-10","arxiv_id":"2102.05775","n_code_links":0,"syntology":null},{"paper":"/paper/tokens-to-token-vit-training-vision","title":"Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet","date":"2021-01-28","arxiv_id":"2101.11986","n_code_links":13,"syntology":{"ran":21,"of":26,"unverified":5,"pointer_only":8}},{"paper":"/paper/ghostsr-learning-ghost-features-for-efficient","title":"GhostSR: Learning Ghost Features for Efficient Image Super-Resolution","date":"2021-01-21","arxiv_id":"2101.08525","n_code_links":4,"syntology":null},{"paper":null,"title":"A Multi-task Joint Framework for Real-time Person Search","date":"2020-12-11","arxiv_id":"2012.06418","n_code_links":0,"syntology":null},{"paper":null,"title":"Real-time Semantic Segmentation with Context Aggregation Network","date":"2020-11-02","arxiv_id":"2011.00993","n_code_links":0,"syntology":null},{"paper":"/paper/model-rubik-s-cube-twisting-resolution-depth","title":"Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets","date":"2020-10-28","arxiv_id":"2010.14819","n_code_links":9,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":1}},{"paper":"/paper/hs-resnet-hierarchical-split-block-on","title":"HS-ResNet: Hierarchical-Split Block on Convolutional Neural Network","date":"2020-10-15","arxiv_id":"2010.07621","n_code_links":2,"syntology":null},{"paper":"/paper/ghostnet-more-features-from-cheap-operations","title":"GhostNet: More Features from Cheap Operations","date":"2019-11-27","arxiv_id":"1911.11907","n_code_links":33,"syntology":{"ran":6,"of":23,"unverified":17,"pointer_only":4}}],"papers_shown":30,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":8},{"task":"/task/object-detection","name":"Object Detection","papers":5},{"task":"/task/image-classification","name":"image-classification","papers":5},{"task":"/task/defect-detection","name":"Defect Detection","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/insulator-defect-detection","name":"Insulator Defect Detection","papers":3},{"task":"/task/segmentation","name":"Segmentation","papers":3},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":3},{"task":"/task/super-resolution","name":"Super-Resolution","papers":3},{"task":"/task/object-detection-1","name":"object-detection","papers":3},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":2},{"task":"/task/image-super-resolution","name":"Image Super-Resolution","papers":2},{"task":"/task/model-compression","name":"Model Compression","papers":2},{"task":"/task/small-object-detection","name":"Small Object Detection","papers":2},{"task":"/task/acoustic-scene-classification","name":"Acoustic Scene Classification","papers":1},{"task":"/task/action-recognition-in-videos","name":"Action Recognition","papers":1},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":1},{"task":null,"name":"CPU","papers":1},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1}],"tasks_shown":20,"n_tasks":45,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":4},{"year":"2021","papers":7},{"year":"2022","papers":8},{"year":"2023","papers":2},{"year":"2024","papers":7},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/ghost-module"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}