Knowledge graph DST : a case study of rich representations for dialogue-state tracking

Abstract

The flow of a task-oriented dialogue between a virtual assistant and a human user is dependent on user intents, i.e. the user’s goals. Unlike open dialogue systems, a user may have a precise goal such as booking a restaurant or calling 911, and as such, require precise -as well as succinct- responses to address the user’s requests. Dialogue State Tracking (DST) is a task that attempts to track this information by estimating the user goals with slot-values, e.g. (Slot=Restaurant, Value=Chinese) based on what the user has said. By keeping in mind the user’s goals, the virtual assistant can access an external knowledge base to generate the appropriate succinct response. Although current solutions can accurately generate states (63%), the traditional slot-value annotations are not sufficiently expressive, thus limiting an agent’s goal estimation abilities in multi-domain and multi-task dialogues. In this investigation, we propose Knowledge Graph-DST, which uses an RDF-inspired schema to represent the dialogue state. This directed graph contains richer annotations composed of triples (subject, predicate, object) that include additional knowledge ranging from what information should the system be looking for, what has been found, and what actions have been taken so far. By combining both a long variant of the encoder-decoder transformer T5 (long-tglobal) and the rich representations in our pipeline, Knowledge Graph DST improves over the state of the art with a 68% joint goal accuracy (JGA), albeit by heavily increasing training and annotation costs. Because of this, we also explore whether light fine-tuning methods such as prefix-tuning, infused adapters by inhibiting and amplifying inner activation -also known as IA3 -, and low-rank adaptation of large language models (LoRA) can yield gains and reduce overall computational costs. Our analysis shows that only LoRA can match full adaption in the DST task. Finally, we complete this project with a twofold investigation. First, an analysis of the relevance of the rich cumulative graph states in DST using gradient input saliency with our custom implementation of the explainable transformer library ecco. And, second, an environmental cost analysis to determine whether the performance and costs increase are justifiable trade-offs.