</> Code with Papers by Syntology

Methods › Computer Vision › Image Feature Extractors › Non-Local Operation

Non-Local Operation

181 papers tagged archive 2025-07-28

Introduced by Xiaolong Wang et al. in Non-local Neural Networks

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

A Non-Local Operation is a component for capturing long-range dependencies with deep neural networks. It is a generalization of the classical non-local mean operation in computer vision. Intuitively a non-local operation computes the response at a position as a weighted sum of the features at all positions in the input feature maps. The set of positions can be in space, time, or spacetime, implying that these operations are applicable for image, sequence, and video problems.

Following the non-local mean operation, a generic non-local operation for deep neural networks is defined as:

𝕪ᵢ = 1/(𝒞(𝕩))∑_(∀j)f(𝕩ᵢ, 𝕩ⱼ)g(𝕩ⱼ)

Here i is the index of an output position (in space, time, or spacetime) whose response is to be computed and j is the index that enumerates all possible positions. x is the input signal (image, sequence, video; often their features) and y is the output signal of the same size as x. A pairwise function f computes a scalar (representing relationship such as affinity) between i and all j. The unary function g computes a representation of the input signal at the position j. The response is normalized by a factor C(x).

The non-local behavior is due to the fact that all positions (∀j) are considered in the operation. As a comparison, a convolutional operation sums up the weighted input in a local neighborhood (e.g., i − 1 ≤j ≤i + 1 in a 1D case with kernel size 3), and a recurrent operation at time i is often based only on the current and the latest time steps (e.g., j = i or i − 1).

The non-local operation is also different from a fully-connected (fc) layer. The equation above computes responses based on relationships between different locations, whereas fc uses learned weights. In other words, the relationship between xⱼ and xᵢ is not a function of the input data in fc, unlike in nonlocal layers. Furthermore, the formulation in the equation above supports inputs of variable sizes, and maintains the corresponding size in the output. On the contrary, an fc layer requires a fixed-size input/output and loses positional correspondence (e.g., that from xᵢ to yᵢ at the position i).

A non-local operation is a flexible building block and can be easily used together with convolutional/recurrent layers. It can be added into the earlier part of deep neural networks, unlike fc layers that are often used in the end. This allows us to build a richer hierarchy that combines both non-local and local information.

In terms of parameterisation, we usually parameterise g as a linear embedding of the form g(xⱼ) = W_g𝕩ⱼ , where W_g is a weight matrix to be learned. This is implemented as, e.g., 1×1 convolution in space or 1×1×1 convolution in spacetime. For f we use an affinity function, a list of which can be found here.

PaperSourceSee Code · tea1528/Non-Local-NN-Pytorch

Papers archive 2025-07-28

30 shown of 181, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 197 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Image Generation37
Conditional Image Generation15
Object Detection14
Generative Adversarial Network12
Semantic Segmentation12
Instance Segmentation10
object-detection10
Segmentation9
reinforcement-learning9
Super-Resolution8
Multi-agent Reinforcement Learning7
Reinforcement Learning7
Action Recognition6
Data Augmentation6
Denoising6
Image Classification6
Reinforcement Learning (RL)6
Transfer Learning6
image-classification6
Decision Making5

Usage over time archive 2025-07-28

Papers per year tagged with Non-Local Operation: 2017 to 2025, peak 45 45 0 2017: 1 paper 2017 2018: 5 papers 2018 2019: 20 papers 2019 2020: 45 papers 2020 2021: 26 papers 2021 2022: 36 papers 2022 2023: 41 papers 2023 2024: 6 papers 2024 2025: 1 paper 2025
Papers per year the archive tags with this method, by the paper's archive date (181 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Image Feature Extractors

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections