Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Confidence measure for speech-to-concept end-to-end spoken language understanding

Abstract : Recent studies have led to the introduction of Speech-to-Concept End-to-End (E2E) neural architectures for Spoken Language Understanding (SLU) that reach state of the art performance. In this work, we propose a way to compute confidence measures on semantic concepts recognized by a Speech-to-Text E2E SLU system. We investigate the use of the hidden representations of our CTC-based SLU system to train an external simple classifier. We experiment two kinds of external simple classifiers to analyze subsequences of hidden representations involved in recognized semantic concepts. The first external classifier is based on a MLP while the second one is based on a bLSTM neural network. We compare them to a baseline confidence measure computed directly from the softmax outputs of the E2E system. On the French challenging MEDIA corpus, when the confidence measure is used to reject, experiments show that using an external BLSTM significantly out-performs the other approaches in terms of precision/recall. To evaluate the additional information provided by this confidence measure, we compute the value of Normalised Cross-Entropy (NCE). Reaching a value equal to 0.288, we show that our best proposed confidence measure brings relevant information about the reliability of a recognized concept.
Complete list of metadatas

Cited literature [20 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02940028
Contributor : Antoine Caubrière <>
Submitted on : Monday, September 21, 2020 - 3:23:27 PM
Last modification on : Thursday, September 24, 2020 - 3:07:23 AM

File

IS20(2).pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02940028, version 1

Citation

Antoine Caubrière, Yannick Estève, Antoine Laurent, Emmanuel Morin. Confidence measure for speech-to-concept end-to-end spoken language understanding. 2020. ⟨hal-02940028⟩

Share

Metrics

Record views

47

Files downloads

50