{"url":"/method/self-adversarial-negative-sampling","slug":"self-adversarial-negative-sampling","name":"Self-Adversarial Negative Sampling","full_name":"Self-Adversarial Negative Sampling","full_name_withheld":false,"description_markdown":"**Self-Adversarial Negative Sampling** is a negative sampling technique used for methods like [word embeddings](https://paperswithcode.com/methods/category/word-embeddings) and [knowledge graph embeddings](https://paperswithcode.com/methods/category/graph-embeddings). The traditional negative sampling loss from word2vec for optimizing distance-based models be written as:\r\n\r\n$$ L = −\\log\\sigma\\left(\\gamma − d\\_{r}\\left(\\mathbf{h}, \\mathbf{t}\\right)\\right) − \\sum^{n}\\_{i=1}\\frac{1}{k}\\log\\sigma\\left(d\\_{r}\\left(\\mathbf{h}^{'}\\_{i}, \\mathbf{t}^{'}\\_{i}\\right) - \\gamma\\right) $$\r\n\r\nwhere $\\gamma$ is a fixed margin, $\\sigma$ is the sigmoid function, and $\\left(\\mathbf{h}^{'}\\_{i}, r, \\mathbf{t}^{'}\\_{i}\\right)$ is the $i$-th negative triplet. \r\n\r\nThe negative sampling loss samples the negative triplets in a uniform way. Such a uniform negative sampling suffers the problem of inefficiency since many samples are obviously false as training goes on, which does not provide any meaningful information. Therefore, the authors propose an approach called self-adversarial negative sampling, which samples negative triples according to the current embedding model. Specifically, we sample negative triples from the following distribution:\r\n\r\n$$ p\\left(h^{'}\\_{j}, r, t^{'}\\_{j} | \\text{set}\\left(h\\_{i}, r\\_{i}, t\\_{i} \\right) \\right) = \\frac{\\exp\\alpha{f}\\_{r}\\left(\\mathbf{h}^{'}\\_{j}, \\mathbf{t}^{'}\\_{j}\\right)}{\\sum\\_{i=1}\\exp\\alpha{f}\\_{r}\\left(\\mathbf{h}^{'}\\_{i}, \\mathbf{t}^{'}\\_{i}\\right)} $$\r\n\r\nwhere $\\alpha$ is the temperature of sampling. Moreover, since the sampling procedure may be costly, the authors treat the above probability as the weight of the negative sample. Therefore, the final negative sampling loss with self-adversarial training takes the following form:\r\n\r\n$$ L = −\\log\\sigma\\left(\\gamma − d\\_{r}\\left(\\mathbf{h}, \\mathbf{t}\\right)\\right) − \\sum^{n}\\_{i=1}p\\left(h^{'}\\_{i}, r, t^{'}\\_{i}\\right)\\log\\sigma\\left(d\\_{r}\\left(\\mathbf{h}^{'}\\_{i}, \\mathbf{t}^{'}\\_{i}\\right) - \\gamma\\right) $$","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1902.10197v1","title":"RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Negative Sampling","url":"/methods/category/negative-sampling","pwc_aliases":[]}],"n_papers_tagged":30,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Is Architectural Complexity Overrated? Competitive and Interpretable Knowledge Graph Completion with RelatE","date":"2025-05-25","arxiv_id":"2505.18971","n_code_links":0,"syntology":null},{"paper":"/paper/sparsetransx-efficient-training-of","title":"SparseTransX: Efficient Training of Translation-Based Knowledge Graph Embeddings Using Sparse Matrix Operations","date":"2025-02-24","arxiv_id":"2502.16949","n_code_links":2,"syntology":null},{"paper":"/paper/unified-interpretation-of-smoothing-methods","title":"Unified Interpretation of Smoothing Methods for Negative Sampling Loss Functions in Knowledge Graph Embedding","date":"2024-07-05","arxiv_id":"2407.04251","n_code_links":1,"syntology":null},{"paper":"/paper/pac-bayesian-generalization-bounds-for-2","title":"PAC-Bayesian Generalization Bounds for Knowledge Graph Representation Learning","date":"2024-05-10","arxiv_id":"2405.06418","n_code_links":1,"syntology":{"ran":3,"of":4,"unverified":1,"pointer_only":4}},{"paper":null,"title":"Edge-Enabled Anomaly Detection and Information Completion for Social Network Knowledge Graphs","date":"2024-01-13","arxiv_id":"2401.07022","n_code_links":0,"syntology":null},{"paper":"/paper/block-diagonal-orthogonal-relation-and-matrix","title":"Block-Diagonal Orthogonal Relation and Matrix Entity for Knowledge Graph Embedding","date":"2024-01-11","arxiv_id":"2401.05967","n_code_links":1,"syntology":null},{"paper":null,"title":"Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?","date":"2023-11-15","arxiv_id":"2311.09109","n_code_links":0,"syntology":null},{"paper":"/paper/model-based-subsampling-for-knowledge-graph","title":"Model-based Subsampling for Knowledge Graph Completion","date":"2023-09-17","arxiv_id":"2309.09296","n_code_links":1,"syntology":null},{"paper":"/paper/bess-balanced-entity-sampling-and-sharing-for","title":"BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion","date":"2022-11-22","arxiv_id":"2211.12281","n_code_links":2,"syntology":{"ran":0,"of":9,"unverified":9,"pointer_only":0}},{"paper":"/paper/self-improving-safety-performance-of","title":"Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms","date":"2022-10-29","arxiv_id":"2210.16575","n_code_links":2,"syntology":null},{"paper":null,"title":"CompoundE: Knowledge Graph Embedding with Translation, Rotation and Scaling Compound Operations","date":"2022-07-12","arxiv_id":"2207.05324","n_code_links":0,"syntology":null},{"paper":"/paper/comprehensive-analysis-of-negative-sampling","title":"Comprehensive Analysis of Negative Sampling in Knowledge Graph Representation Learning","date":"2022-06-21","arxiv_id":"2206.10140","n_code_links":1,"syntology":null},{"paper":"/paper/learning-to-borrow-relation-representation-1","title":"Learning to Borrow -- Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion","date":"2022-04-27","arxiv_id":"2204.13097","n_code_links":1,"syntology":null},{"paper":null,"title":"SpaceE: Knowledge Graph Embedding by Relational Linear Transformation in the Entity Space","date":"2022-04-21","arxiv_id":"2204.10245","n_code_links":0,"syntology":null},{"paper":"/paper/a-unified-framework-for-rank-based-evaluation","title":"A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs","date":"2022-03-14","arxiv_id":"2203.07544","n_code_links":2,"syntology":{"ran":0,"of":5,"unverified":5,"pointer_only":0}},{"paper":"/paper/simkgc-simple-contrastive-knowledge-graph","title":"SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models","date":"2022-03-04","arxiv_id":"2203.02167","n_code_links":2,"syntology":{"ran":3,"of":6,"unverified":3,"pointer_only":3}},{"paper":null,"title":"Learning to Borrow– Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion","date":"2022-01-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"CORE: A Knowledge Graph Entity Type Prediction Method via Complex Space Regression and Embedding","date":"2021-12-19","arxiv_id":"2112.10067","n_code_links":0,"syntology":null},{"paper":null,"title":"Mixture-of-Graphs: Zero-shot Relational Learning for Knowledge Graph by Fusing Ontology and Textual Experts","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/qubite-qubit-embedding-for-knowledge-graph","title":"QubitE: Qubit Embedding for Knowledge Graph Completion","date":"2021-11-16","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Towards Robust Knowledge Graph Embedding via Multi-task Reinforcement Learning","date":"2021-11-11","arxiv_id":"2111.06103","n_code_links":0,"syntology":null},{"paper":null,"title":"Why does Negative Sampling not Work Well? Analysis of Convexity in Negative Sampling","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"NOTE: Solution for KDD-CUP 2021 WikiKG90M-LSC","date":"2021-07-05","arxiv_id":"2107.01892","n_code_links":0,"syntology":null},{"paper":"/paper/quatde-dynamic-quaternion-embedding-for","title":"QuatDE: Dynamic Quaternion Embedding for Knowledge Graph Completion","date":"2021-05-19","arxiv_id":"2105.09002","n_code_links":1,"syntology":null},{"paper":"/paper/dense-an-enhanced-non-abelian-group","title":"DensE: An Enhanced Non-commutative Representation for Knowledge Graph Embedding with Adaptive Semantic Hierarchy","date":"2020-08-11","arxiv_id":"2008.04548","n_code_links":1,"syntology":null},{"paper":"/paper/convolutional-complex-knowledge-graph","title":"Convolutional Complex Knowledge Graph Embeddings","date":"2020-08-07","arxiv_id":"2008.03130","n_code_links":2,"syntology":null},{"paper":"/paper/orthogonal-relation-transforms-with-graph","title":"Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding","date":"2019-11-09","arxiv_id":"1911.04910","n_code_links":0,"syntology":null},{"paper":null,"title":"TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction","date":"2019-10-01","arxiv_id":"1910.00702","n_code_links":0,"syntology":null},{"paper":null,"title":"Group Representation Theory for Knowledge Graph Embedding","date":"2019-09-11","arxiv_id":"1909.05100","n_code_links":0,"syntology":null},{"paper":"/paper/rotate-knowledge-graph-embedding-by","title":"RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space","date":"2019-02-26","arxiv_id":"1902.10197","n_code_links":10,"syntology":{"ran":0,"of":5,"unverified":5,"pointer_only":0}}],"papers_shown":30,"tasks":[{"task":"/task/graph-embedding","name":"Graph Embedding","papers":20},{"task":"/task/knowledge-graph-embedding","name":"Knowledge Graph Embedding","papers":19},{"task":"/task/knowledge-graphs","name":"Knowledge Graphs","papers":14},{"task":"/task/link-prediction","name":"Link Prediction","papers":13},{"task":"/task/knowledge-graph-completion","name":"Knowledge Graph Completion","papers":12},{"task":null,"name":"Relation","papers":11},{"task":"/task/entity-embeddings","name":"Entity Embeddings","papers":4},{"task":"/task/knowledge-graph-embeddings","name":"Knowledge Graph Embeddings","papers":4},{"task":"/task/translation","name":"Translation","papers":4},{"task":"/task/representation-learning","name":"Representation Learning","papers":3},{"task":null,"name":"Triplet","papers":3},{"task":"/task/benchmarking","name":"Benchmarking","papers":2},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":2},{"task":"/task/decoder","name":"Decoder","papers":2},{"task":null,"name":"GPU","papers":2},{"task":"/task/graph-representation-learning","name":"Graph Representation Learning","papers":2},{"task":"/task/prediction","name":"Prediction","papers":2},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":2},{"task":"/task/relational-reasoning","name":"Relational Reasoning","papers":2},{"task":"/task/sentence","name":"Sentence","papers":2}],"tasks_shown":20,"n_tasks":51,"usage_by_year":[{"year":"2019","papers":4},{"year":"2020","papers":2},{"year":"2021","papers":7},{"year":"2022","papers":9},{"year":"2023","papers":2},{"year":"2024","papers":4},{"year":"2025","papers":2}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/self-adversarial-negative-sampling"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}