{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-a-deeper-understanding-of-adversarial","title":"On Output Activation Functions for Adversarial Losses: A Theoretical Analysis via Variational Divergence Minimization and An Empirical Study on MNIST Classification","arxiv_id":"1901.08753","date":"2019-01-25","proceeding":null,"authors":["Hao-Wen Dong","Yi-Hsuan Yang"],"abstract":"Recent years have seen adversarial losses been applied to many fields. Their applications extend beyond the originally proposed generative modeling to conditional generative and discriminative settings. While prior work has proposed various output activation functions and regularization approaches, some open questions still remain unanswered. In this paper, we aim to study the following two research questions: 1) What types of output activation functions form a well-behaved adversarial loss? 2) How different combinations of output activation functions and regularization approaches perform empirically against one another? To answer the first question, we adopt the perspective of variational divergence minimization and consider an adversarial loss well-behaved if it behaves as a divergence-like measure between the data and model distributions. Using a generalized formulation for adversarial losses, we derive the necessary and sufficient conditions of a well-behaved adversarial loss. Our analysis reveals a large class of theoretically valid adversarial losses. For the second question, we propose a simple comparative framework for adversarial losses using discriminative adversarial networks. The proposed framework allows us to efficiently evaluate adversarial losses using a standard evaluation metric such as the classification accuracy. With the proposed framework, we evaluate a comprehensive set of 168 combinations of twelve output activation functions and fourteen regularization approaches on the handwritten digit classification problem to decouple their effects. Our empirical findings suggest that there is no single winning combination of output activation functions and regularization approaches across all settings. Our theoretical and empirical results may together serve as a reference for choosing or designing adversarial losses in future research.","url_abs":"https://arxiv.org/abs/1901.08753v3","url_pdf":"https://arxiv.org/pdf/1901.08753v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-a-deeper-understanding-of-adversarial","repo_url":"https://github.com/salu133445/dan","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1901.08753","atlas_url":"https://app.syntology.ai/?focus=1901.08753","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.08753"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salu133445/dan","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":16},"by_repo_kind":{"official":{"samples":16,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"39a7b44290d7f7d6","entry":"classic_gan_losses","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/losses.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39a7b44290d7f7d6"}},{"code_sha256_prefix":"532ce6c3a951d80b","entry":"convconcat","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/presets/ops.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/presets/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"532ce6c3a951d80b"}},{"code_sha256_prefix":"7ebc66c169767bcf","entry":"get_adv_losses","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/losses.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ebc66c169767bcf"}},{"code_sha256_prefix":"5da276d0f1cfbea3","entry":"get_l2_norm","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/model.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5da276d0f1cfbea3"}},{"code_sha256_prefix":"6c50317414bf1d88","entry":"get_normalization","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/presets/ops.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/presets/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c50317414bf1d88"}},{"code_sha256_prefix":"99d72493ce9b6a6c","entry":"load_component","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/utils.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99d72493ce9b6a6c"}},{"code_sha256_prefix":"437ca4ed58b40c6c","entry":"load_data","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/data.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"437ca4ed58b40c6c"}},{"code_sha256_prefix":"d734bf997af678c8","entry":"load_data_from_npy","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/data.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d734bf997af678c8"}},{"code_sha256_prefix":"88a4709aadaa1849","entry":"load_params","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/utils.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88a4709aadaa1849"}},{"code_sha256_prefix":"e9b277604f711d0d","entry":"load_yaml","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/utils.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9b277604f711d0d"}},{"code_sha256_prefix":"d2616d28a605aeb3","entry":"nonsaturating_gan_losses","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/losses.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2616d28a605aeb3"}},{"code_sha256_prefix":"4e288115d85d7bcc","entry":"set_data_shape","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/data.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e288115d85d7bcc"}},{"code_sha256_prefix":"54aa94d3da1df967","entry":"shift","repo":"salu133445/dan","repo_kind":"official","path":"src/process_data.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/process_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54aa94d3da1df967"}},{"code_sha256_prefix":"d9d55b6590a04aee","entry":"sn_conv2d","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/presets/spectral_norm.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/presets/spectral_norm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9d55b6590a04aee"}},{"code_sha256_prefix":"1f9327121f6048f9","entry":"sn_dense","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/presets/spectral_norm.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/presets/spectral_norm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f9327121f6048f9"}},{"code_sha256_prefix":"95cafc39c0a7381c","entry":"spectral_normalize","repo":"salu133445/dan","repo_kind":"official","path":"src/dan/presets/spectral_norm.py","file_url":"https://github.com/salu133445/dan/blob/HEAD/src/dan/presets/spectral_norm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95cafc39c0a7381c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}