{"url":"/method/off-diagonal-orthogonal-regularization","slug":"off-diagonal-orthogonal-regularization","name":"Off-Diagonal Orthogonal Regularization","full_name":"Off-Diagonal Orthogonal Regularization","full_name_withheld":false,"description_markdown":"**Off-Diagonal Orthogonal Regularization** is a modified form of [orthogonal regularization](https://paperswithcode.com/method/orthogonal-regularization) originally used in [BigGAN](https://paperswithcode.com/method/biggan). The original orthogonal regularization is known to be limiting so the authors explore several variants designed to relax the constraint while still imparting the desired smoothness to the models. They opt for a modification where they remove diagonal terms from the regularization, and aim to minimize the pairwise cosine similarity between filters but does not constrain their norm:\r\n\r\n$$ R\\_{\\beta}\\left(W\\right) = \\beta|| W^{T}W \\odot \\left(\\mathbf{1}-I\\right) ||^{2}\\_{F} $$\r\n\r\nwhere $\\mathbf{1}$ denotes a matrix with all elements set to 1. The authors sweep $\\beta$ values and select $10^{−4}$.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Large Scale GAN Training for High Fidelity Natural Image Synthesis","paper":"/paper/large-scale-gan-training-for-high-fidelity","first_author":"Andrew Brock","n_authors":3,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/large-scale-gan-training-for-high-fidelity"},"source":{"url":"http://arxiv.org/abs/1809.11096v2","title":"Large Scale GAN Training for High Fidelity Natural Image Synthesis","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/ajbrock/BigGAN-PyTorch/blob/b70f16c4a879b2d5d5d7bcb73794424aef5eec1f/train_fns.py#L51","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":132,"archive_num_papers":132,"papers_newest_first":[{"paper":null,"title":"ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks","date":"2024-11-06","arxiv_id":"2411.03999","n_code_links":0,"syntology":null},{"paper":"/paper/unsupervised-panoptic-interpretation-of","title":"Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization","date":"2024-10-27","arxiv_id":"2410.20573","n_code_links":1,"syntology":null},{"paper":null,"title":"On quantifying and improving realism of images generated with diffusion","date":"2023-09-26","arxiv_id":"2309.14756","n_code_links":0,"syntology":null},{"paper":null,"title":"Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/a-strategic-framework-for-optimal-decisions","title":"A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory","date":"2023-07-27","arxiv_id":"2307.14732","n_code_links":2,"syntology":null},{"paper":"/paper/pyrus-base-an-open-source-python-framework","title":"Pyrus Base: An Open Source Python Framework for the RoboCup 2D Soccer Simulation","date":"2023-07-22","arxiv_id":"2307.16875","n_code_links":1,"syntology":null},{"paper":"/paper/diffusion-models-beat-gans-on-image","title":"Diffusion Models Beat GANs on Image Classification","date":"2023-07-17","arxiv_id":"2307.08702","n_code_links":1,"syntology":null},{"paper":null,"title":"Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs","date":"2023-06-28","arxiv_id":"2306.15903","n_code_links":0,"syntology":null},{"paper":null,"title":"Rosetta Neurons: Mining the Common Units in a Model Zoo","date":"2023-06-15","arxiv_id":"2306.09346","n_code_links":0,"syntology":null},{"paper":null,"title":"Toward more accurate and generalizable brain deformation estimators for traumatic brain injury detection with unsupervised domain adaptation","date":"2023-06-08","arxiv_id":"2306.05255","n_code_links":0,"syntology":null},{"paper":null,"title":"FOOCTTS: Generating Arabic Speech with Acoustic Environment for Football Commentator","date":"2023-06-07","arxiv_id":"2306.07936","n_code_links":0,"syntology":null},{"paper":null,"title":"Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning","date":"2023-05-29","arxiv_id":"2305.17886","n_code_links":0,"syntology":null},{"paper":"/paper/is-centralized-training-with-decentralized","title":"Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?","date":"2023-05-27","arxiv_id":"2305.17352","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":1}},{"paper":null,"title":"Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations","date":"2023-05-22","arxiv_id":"2305.13030","n_code_links":0,"syntology":null},{"paper":"/paper/an-empirical-study-on-google-research","title":"An Empirical Study on Google Research Football Multi-agent Scenarios","date":"2023-05-16","arxiv_id":"2305.09458","n_code_links":1,"syntology":null},{"paper":"/paper/the-muse-2023-multimodal-sentiment-analysis","title":"The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisation","date":"2023-05-05","arxiv_id":"2305.03369","n_code_links":1,"syntology":null},{"paper":"/paper/sportsmot-a-large-multi-object-tracking","title":"SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes","date":"2023-04-11","arxiv_id":"2304.05170","n_code_links":1,"syntology":null},{"paper":"/paper/vars-video-assistant-referee-system-for","title":"VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views","date":"2023-04-10","arxiv_id":"2304.04617","n_code_links":1,"syntology":null},{"paper":null,"title":"Towards Active Learning for Action Spotting in Association Football Videos","date":"2023-04-09","arxiv_id":"2304.04220","n_code_links":0,"syntology":null},{"paper":null,"title":"Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football","date":"2023-03-25","arxiv_id":"2303.15471","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Generative Multi-Agent Imitation Model as a Computational Benchmark for Evaluating Human Performance in Complex Interactive Tasks: A Case Study in Football","date":"2023-03-23","arxiv_id":"2303.13323","n_code_links":0,"syntology":null},{"paper":"/paper/high-fidelity-image-synthesis-with-deep-vaes","title":"High Fidelity Image Synthesis With Deep VAEs In Latent Space","date":"2023-03-23","arxiv_id":"2303.13714","n_code_links":2,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":2}},{"paper":"/paper/breaking-common-sense-whoops-a-vision-and","title":"Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images","date":"2023-03-13","arxiv_id":"2303.07274","n_code_links":0,"syntology":null},{"paper":"/paper/nfl-career-success-as-predicted-by-nfl","title":"NFL Career Success as Predicted by NFL Scouting Combine","date":"2023-03-10","arxiv_id":"2303.05774","n_code_links":1,"syntology":null},{"paper":"/paper/ultra-high-resolution-detector-simulation","title":"Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning","date":"2023-03-07","arxiv_id":"2303.08046","n_code_links":1,"syntology":null},{"paper":null,"title":"Identification of pattern mining algorithm for rugby league players positional groups separation based on movement patterns","date":"2023-02-25","arxiv_id":"2302.14058","n_code_links":0,"syntology":null},{"paper":"/paper/transformer-based-neural-marked-spatio","title":"Transformer-Based Neural Marked Spatio Temporal Point Process Model for Football Match Events Analysis","date":"2023-02-18","arxiv_id":"2302.09276","n_code_links":1,"syntology":null},{"paper":"/paper/tizero-mastering-multi-agent-football-with","title":"TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play","date":"2023-02-15","arxiv_id":"2302.07515","n_code_links":1,"syntology":null},{"paper":null,"title":"Inferring Player Location in Sports Matches: Multi-Agent Spatial Imputation from Limited Observations","date":"2023-02-13","arxiv_id":"2302.06569","n_code_links":0,"syntology":null},{"paper":"/paper/order-matters-agent-by-agent-policy","title":"Order Matters: Agent-by-agent Policy Optimization","date":"2023-02-13","arxiv_id":"2302.06205","n_code_links":1,"syntology":{"ran":2,"of":4,"unverified":2,"pointer_only":0}}],"papers_shown":30,"tasks":[{"task":"/task/image-generation","name":"Image Generation","papers":37},{"task":"/task/conditional-image-generation","name":"Conditional Image Generation","papers":15},{"task":null,"name":"Generative Adversarial Network","papers":11},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":9},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":7},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":7},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":6},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":6},{"task":"/task/super-resolution","name":"Super-Resolution","papers":6},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":6},{"task":"/task/decision-making","name":"Decision Making","papers":5},{"task":"/task/unconditional-image-generation","name":"Unconditional Image Generation","papers":5},{"task":"/task/attribute","name":"Attribute","papers":4},{"task":"/task/clustering","name":"Clustering","papers":4},{"task":"/task/denoising","name":"Denoising","papers":4},{"task":"/task/diversity","name":"Diversity","papers":4},{"task":"/task/object","name":"Object","papers":4},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":4},{"task":"/task/benchmarking","name":"Benchmarking","papers":3},{"task":"/task/decoder","name":"Decoder","papers":3}],"tasks_shown":20,"n_tasks":141,"usage_by_year":[{"year":"2018","papers":2},{"year":"2019","papers":10},{"year":"2020","papers":28},{"year":"2021","papers":20},{"year":"2022","papers":33},{"year":"2023","papers":37},{"year":"2024","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/off-diagonal-orthogonal-regularization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}