Papers › Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems

Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems

12 Oct 2020EMNLP 2021 11arXiv:2010.05740archive 2025-07-28

Yanjie Gou, Yinjie Lei, Lingqiao Liu, Yong Dai, Chunxu Shen

Incorporating knowledge bases (KB) into end-to-end task-oriented dialogue systems is challenging, since it requires to properly represent the entity of KB, which is associated with its KB context and dialogue context. The existing works represent the entity with only perceiving a part of its KB context, which can lead to the less effective representation due to the information loss, and adversely favor KB reasoning and response generation. To tackle this issue, we explore to fully contextualize the entity representation by dynamically perceiving all the relevant entities} and dialogue history. To achieve this, we propose a COntext-aware Memory Enhanced Transformer framework (COMET), which treats the KB as a sequence and leverages a novel Memory Mask to enforce the entity to only focus on its relevant entities and dialogue history, while avoiding the distraction from the irrelevant entities. Through extensive experiments, we show that our COMET framework can achieve superior performance over the state of the arts.

Paper PDF Conference PDF

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Response Generation Task-Oriented Dialogue Systems

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Task-Oriented Dialogue Systems	KVRET	COMET	BLEU	17.3	#2 of 10	Archive leaderboard	report
Task-Oriented Dialogue Systems	KVRET	COMET	Entity F1	63.6	#2 of 10	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Dynamic Memory Network GRU Label Smoothing Layer Normalization Linear Layer Memory Network Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections