testing simon

In the context of speech recognition, I use the pseudonym “Ralf Herzog”. In the future, I will use my real name “Kai Schott”, too. E.g. instead of Ralf's Latin IPA FLAC files, I use the expression Schott's Latin IPA FLAC files. Maybe I will name the next version of my German PLS dictionary “Schott’s German dictionary” (instead of Ralf's German dictionary).

Take a look into german-dictionary.xml:

<lexeme role="Substantiv">
<grapheme>Loveparade</grapheme>
<phoneme>lavpɛʀɛɪ̯t</phoneme>
</lexeme>

When a German person speaks the word ‘Loveparade’, how would he probably pronounce the ‘r’? According to the Wikipedia, there are nine different ‘r’ phonemes available:

Alveolar trill [r] · Alveolar approximant [ɹ] · Alveolar tap [ɾ] · Alveolar lateral flap [ɺ] · Retroflex approximant [ɻ] · Retroflex flap [ɽ] · Uvular trill [ʀ] · Voiced uvular fricative [ʁ] · Labialized [ʋ]

Should I replace the ʀ by a different ‘r’-phoneme?

I totally agree with this statement:

“Where other ASR attempts focused on either understanding words semantically (what does this word mean?) or on word bigram and trigram patterns (which words are most likely to come next?), both techniques you described, the HMM approach at the phoneme level was far more successful.”

This is why I am developing lots of PLS dictionaries. The PLS dictionaries work mainly on the phoneme level. When you train long words (words with lots of phonemes) you get better recognition results than with short words. What does this mean for the English and for the German language?

1. For the English language, you need “word bigram and trigram patterns” to get good recognition results. This means that you need a grammar.

2. For the German language, “the HMM approach at the phoneme level” is sufficient if you train only long words (e.g. “zusammengesetzte Wörter”). If you want to train short words for the German language (e.g. the German words zwei or zwölf or zurück), you should train them in “word bigram and trigram patterns” (like in the English language).

I want to train several thousand long German words (without a specific grammar; only the terminal category “Unknown” will be used) because I believe that “the HMM approach at the phoneme level” is the most promising way to start. Later, short German words can be added. For them, I need “word bigram and trigram patterns”.

English vocabulary doesn’t contain a lot of long words. This means that good recognition results for the English language require “word bigram and trigram patterns”. I find this task more difficult than “the HMM approach at the phoneme level”.

In short:
a. Train several thousand long German words (“the HMM approach at the phoneme level”).
b. Train short German words (“word bigram and trigram patterns”).
c. Train English words (“word bigram and trigram patterns”).

This is a good idea: the user could

“go through a 10 minute training procedure to adapt the static model to his voice. This adaption data could then be sent to voxforge to improve the general model.”

Yes, the user should have the chance to help with the improvement of speech models by submitting his own recordings via simon to Voxforge.

“It all comes down to the community and very much depends upon their participation.”

Yes. I agree. We need more participants who can build plug-ins on their own. By the way, if you want people to participate, offer them export functionality. Maybe this is possible with the simon scenario concept. The user should be able to export everything he has created with simon (wav files, prompts, hmmdefs, tiedlist, macros, scenario).

“If the speech model is working, I am sure I could implement a basic dictation function in simon in a couple of monts if not weeks.”

What is the problem with the German speech model? Personally, I don’t know how to build a speech model with sam. I have recorded about 10.000 German FLAC files, and I would record more if I knew that this would improve the speech model. Unfortunately, I don’t know how I could process them.

By the way, if you want good results, read the SAMPA phonemes when you record your voice with simon. You should avoid to omit any phoneme when speaking. Speak intuitively.

simon should offer a dictation function that inserts automatically a space-bar after each word.

Let’s take a look at this file (ISO-8859-1):

  <word>
   <name>Abbrechen</name>
   <pronunciation>gls a p b R E C @ n</pronunciation>
   <terminal>SimpelKommando</terminal>
  </word>
  <word>
   <name>Acht</name>
   <pronunciation>a x t</pronunciation>
   <terminal>Nummer</terminal>
  </word>

Obviously, simon does now import the glottal stop. The word Abbrechen is with glottal stop. The word Acht is without glottal stop. More info:

Introduced the “gls” phoneme for a glotal stop (“?” in SAMPA).

Great. This should improve the recognition rate when you use Ralf's German dictionary for training. As far as I know, the German BOMP dictionary does have glottal stops, too.

The glottal stop is a difficult phoneme. I think that it is a good decision if simon supports the glottal stop.

The abbreviation gls is better than kn.

Additional thought: Maybe it is possible to develop a simon scenario (XML file) with an .xsl style-sheet? You all know that I am primarily interested in dictation, so a simon scenario with let’s say 1.000 German words would be an interesting start. This should be possible, but I don’t want to invest too much time. So I am looking for a way to get quick results.

My dream is an XML-based framework:
- SSML prompts that link to FLAC audio files;
- Ralf's German dictionary (PLS);
- an .xsl style-sheet that accesses e.g. 99 SSML prompts for training (source file), and that chooses the corresponding words from the PLS dictionary (source file);
- the object file could be a simon scenario.

The command in the Ubuntu terminal would look as follows:

$ saxonb-xslt -ext:on -s:'http://script.blau.in/german/35/prompts.xml' -xsl:'create-simon-scenario.xsl' -o:simon-scenario.xml

What do you think? It is possible to access files that are located in the Internet directly from the Ubuntu terminal, and process them.