Methods › General › Regularization › Orthogonal Regularization
Orthogonal Regularization
Introduced by Andrew Brock et al. in Neural Photo Editing with Introspective Adversarial Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Orthogonal Regularization is a regularization technique for convolutional neural networks, introduced with generative modelling as the task in mind. Orthogonality is argued to be a desirable quality in ConvNet filters, partially because multiplication by an orthogonal matrix leaves the norm of the original matrix unchanged. This property is valuable in deep or recurrent networks, where repeated matrix multiplication can result in signals vanishing or exploding. To try to maintain orthogonality throughout training, Orthogonal Regularization encourages weights to be orthogonal by pushing them towards the nearest orthogonal manifold. The objective function is augmented with the cost:
ℒₒᵣₜₕₒ = ∑(|WWᵀ − I|)
Where ∑ indicates a sum across all filter banks, W is a filter bank, and I is the identity matrix
Papers archive 2025-07-28
27 shown of 27, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Stochastic Orthogonal Regularization for deep projective priors 19 May 2025 · 0 repositories · arXiv:2505.13078
-
Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization 1 Nov 2024 · 1 repository · arXiv:2411.00392Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
-
Principal Orthogonal Latent Components Analysis (POLCA Net) 9 Oct 2024 · 1 repository · arXiv:2410.07289
-
Convolutional Neural Network Compression Based on Low-Rank Decomposition 29 Aug 2024 · 0 repositories · arXiv:2408.16289
-
Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN 13 Aug 2024 · 1 repository · arXiv:2408.06761
-
Efficient Pareto Manifold Learning with Low-Rank Structure 30 Jul 2024 · 0 repositories · arXiv:2407.20734
-
Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalization 21 Jul 2024 · 1 repository · arXiv:2407.15085Syntology ran 11 of 13 samples · 2 unverified
-
Energizing Federated Learning via Filter-Aware Attention 18 Nov 2023 · 0 repositories · arXiv:2311.12049
-
Free Lunch for Gait Recognition: A Novel Relation Descriptor 22 Aug 2023 · 0 repositories · arXiv:2308.11487
-
Inferior Alveolar Nerve Segmentation in CBCT images using Connectivity-Based Selective Re-training 18 Aug 2023 · 1 repository · arXiv:2308.09298
-
Revisiting Orthogonality Regularization: A Study for Convolutional Neural Networks in Image Classification 23 Jun 2022 · 1 repository
-
ProFairRec: Provider Fairness-aware News Recommendation 10 Apr 2022 · 1 repository · arXiv:2204.04724
-
Dreaming To Prune Image Deraining Networks 1 Jan 2022 · 0 repositories
-
Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning 18 Oct 2021 · 0 repositories · arXiv:2110.09374
-
Connecting Low-Loss Subspace for Personalized Federated Learning 16 Sep 2021 · 1 repository · arXiv:2109.07628Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
-
Arabic aspect based sentiment analysis using bidirectional GRU based models 23 Jan 2021 · 0 repositories · arXiv:2101.10539
-
Towards Disentangling Latent Space for Unsupervised Semantic Face Editing 5 Nov 2020 · 1 repository · arXiv:2011.02638
-
A Spectral Energy Distance for Parallel Speech Synthesis 3 Aug 2020 · 2 repositories · arXiv:2008.01160Syntology ran 0 of 3 samples · 3 unverified
-
Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved MobileNets 30 Mar 2020 · 1 repository · arXiv:2003.13549
-
Transformation-based Adversarial Video Prediction on Large-Scale Data 9 Mar 2020 · 0 repositories · arXiv:2003.04035
-
Efficient Model for Image Classification With Regularization Tricks 1 Feb 2020 · 1 repository
-
High Fidelity Speech Synthesis with Adversarial Networks 25 Sep 2019 · 3 repositories · arXiv:1909.11646Syntology ran 2 of 5 samples · 3 unverified · 4 pointer-only (licence)
-
Adversarial Video Generation on Complex Datasets 15 Jul 2019 · 1 repository · arXiv:1907.06571
-
OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization 26 May 2019 · 1 repository · arXiv:1905.10836
-
CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis 27 Dec 2018 · 0 repositories · arXiv:1812.10735
-
Large Scale GAN Training for High Fidelity Natural Image Synthesis 28 Sep 2018 · 35 repositories · arXiv:1809.11096Syntology ran 14 of 41 samples · 27 unverified · 4 pointer-only (licence)
-
Neural Photo Editing with Introspective Adversarial Networks 22 Sep 2016 · 2 repositories · arXiv:1609.07093Syntology ran 2 of 2 samples · 0 unverified
Tasks archive 2025-07-28
20 shown of 52 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections