1. Get a list of file names:

ls /home/ubuntu/Documents/201009/combine-0.2.1/wav-all > /home/ubuntu/Documents/201009/combine-0.2.1/dir-wav-all

2. Compare the list of file names with Ralf’s German dictionary:

cat /home/ubuntu/Documents/201008/german-0.2.1/object/german-0.2.2.xml | \
saxonb-xslt -ext:on -s:- -xsl:/home/ubuntu/Documents/201009/combine-0.2.1/compare.xsl

3. Delete old simon configuration files:

rm -r /home/ubuntu/.kde/share/apps/simon
rm -r /home/ubuntu/.kde/share/apps/simond
rm /home/ubuntu/.kde/share/config/sscrc
rm /home/ubuntu/.kde/share/config/speechmodelmanagementrc
rm /home/ubuntu/.kde/share/config/simonsoundrc
rm /home/ubuntu/.kde/share/config/simonscenariosrc
rm /home/ubuntu/.kde/share/config/simonrecognitionrc
rm /home/ubuntu/.kde/share/config/simonrc
rm /home/ubuntu/.kde/share/config/simoncommandrc
rm /home/ubuntu/.kde/share/config/samrc
rm /home/ubuntu/.kde/share/config/ksimondrc

4. Import Ralf's German dictionary into simon. Before I do that, I go to simon > Manage scenarios > New, and create the scenario “de-20100914″. I will export this scenario, and then use it with the “serialize scenarios” function in sam.

5. simon > Add grammar: Zahlwort Adjektiv Substantiv Verb
Add command: dictation plug-in

6. I want to use sam for the creation of the speech model. More than 300000 words should be in the speech model that I want to create.

7. Manage scenarios: Export scenario “de-20100914″

8. A few steps later, the following error message (sam) appears:

Generation of the transcription failed. Please check if you have correctly specified the paths to mkphones0.led and mkphons1.led. (/usr/share/kde4/apps/simon/scripts/mkphones0.led, /usr/share/kde4/apps/simon/scripts/mkphones1.led)

And this is the corresponding build log:

Preperation
Generating wordlist…
Creating monophones…
“/usr/local/bin/HDMan” -A -D -T 1 -m -w “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//wlist” -g “/usr/share/kde4/apps/simon/scripts/global.ded” -n “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//monophones1″ -i “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict” “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//lexicon”
/usr/local/bin/HDMan -A -D -T 1 -m -w /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//wlist -g /usr/share/kde4/apps/simon/scripts/global.ded -n /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//monophones1 -i /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//lexicon

No HTK Configuration Parameters Set

Output dictionary /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict opened
Source dictionary /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//lexicon opened
Dictionary /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict created – 26168 words processed, 2 missing

No HTK Configuration Parameters Set

Generating Master Label File…
“/usr/local/bin/HLEd” -A -D -T 1 -l “*” -d “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict” -i “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//phones0.mlf” “/usr/share/kde4/apps/simon/scripts/mkphones0.led” “/tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//words.mlf”
/usr/local/bin/HLEd -A -D -T 1 -l * -d /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//dict -i /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//phones0.mlf /usr/share/kde4/apps/simon/scripts/mkphones0.led /tmp/kde-ubuntuJPP9Sy/sam/internalsamuser/compile//words.mlf

No HTK Configuration Parameters Set

Editing file: fiːɐ̯.lab

ERROR [+6550] LoadHTKList: Label Name Expected
FATAL ERROR – Terminating program /usr/local/bin/HLEd

I don’t know what went wrong.

9. I will release Ralf’s German speech model 0.1.7 later, not now.

Edit September 17, 2010: 10. Download Ralf's German speech model 0.1.7.

This will be a test series. I can’t guarantee that it will work out. But I think that my concept should work. At least, I hope so.

One note to the simon developers: Could you please modify the dictation plug-in so that it inserts a space after each word? That would be fine because I would like to produce one or several videos, and I don’t want to hit the space bar after each word that has been recognized.

I have prepared the special style-sheet compare.xsl for this test series. This style-sheet compares Ralf's German dictionary (source file #1) with Ralf's German IPA FLAC files (section xaa) (source file #2), and outputs two result documents: A small PLS dictionary that I will import as active dictionary into simon (result file #1), and a prompts file in HTK compatible format (result file #2) so that I can use the simon import training data function.

The style-sheet is invoked in the Ubuntu terminal:

$ saxonb-xslt -ext:on -s:'/home/ubuntu/Documents/201006/german-0.1.9.5/german-0.1.9.8.xml' -xsl:'/home/ubuntu/Documents/201007/ipa-prompts/compare.xsl' -o:'/home/ubuntu/Documents/201006/audacity/xaa-folder/dummy.xml'

I hope that it will work as expected.

If you are a German native speaker, and your voice is similar to my voice (similar dialect, male speaker), you can try the following:

1. Download Ralf’s German scenario, and import it into simon via manage scenarios.
2. Download hmmdefs.bz2, and extract it.
3. Take a look at the picture. Select the option Static model. Then load the path to hmmdefs.
4. Download tiedlist, and set the path to this file.
5. Download macros, and set the path.
6. Download stats, and set the path.

7. I didn’t test whether it works. I will test it later. If it works, you should be able to dictate the words that I am dictating in the Video: Recognize 200 German words. This means that with Ralf's German speech model you should be able to dictate 200 different German words.

8. You don’t need to install HTK. This means that you only need to install simon on your computer.

9. The terminology is as follows:
Ralf’s German speech model = Ralf’s German scenario + Ralf’s German base model
Ralf’s German base model = hmmdefs.bz2 + tiedlist + macros + stats

As the server compiled the model the following error occurred:
Could not adapt the base model. Please check your paths to HERest (/usr/local/bin/HERest).

Here is what I am doing right now: I want to take a closer look at the error message:

Preperation
Generating wordlist…
Creating monophones…
“/usr/local/bin/HDMan” -A -D -T 1 -m -w “/tmp/kde-ubuntu/simond/default/compile//wlist” -g “/usr/share/kde4/apps/simon/scripts/global.ded” -n “/tmp/kde-ubuntu/simond/default/compile//monophones1” -i “/tmp/kde-ubuntu/simond/default/compile//dict” “/tmp/kde-ubuntu/simond/default/compile//lexicon”
[...]
No HTK Configuration Parameters Set [<- How could I solve this issue?]

[...]

“/usr/local/bin/HLEd” -A -D -T 1 -l “*” -d “/tmp/kde-ubuntu/simond/default/compile//dict” -i “/tmp/kde-ubuntu/simond/default/compile//phones0.mlf” “/usr/share/kde4/apps/simon/scripts/mkphones0.led” “/tmp/kde-ubuntu/simond/default/compile//words.mlf”
Generating Master Label File…
[...]

No HTK Configuration Parameters Set

Editing file: angehören_S1_2010-07-12_19-54-08.0.lab
11 adds; 1 deletes; 0 replaces; 0 changes; 0 merges

No HTK Configuration Parameters Set

“/usr/local/bin/HLEd” -A -D -T 1 -l “*” -d “/tmp/kde-ubuntu/simond/default/compile//dict” -i “/tmp/kde-ubuntu/simond/default/compile//phones1.mlf” “/usr/share/kde4/apps/simon/scripts/mkphones1.led” “/tmp/kde-ubuntu/simond/default/compile//words.mlf”
[...]
No HTK Configuration Parameters Set

Editing file: angehören_S1_2010-07-12_19-54-08.0.lab
11 adds; 0 deletes; 0 replaces; 0 changes; 0 merges

No HTK Configuration Parameters Set

Coding audio files…
“/usr/local/bin/HCopy” -A -D -T 1 -C “/home/ubuntu/.kde/share/apps/simond/models/default/src/2010-07-12_19-54-31/wav_config” -S “/tmp/kde-ubuntu/simond/default/compile//codetrain.scp”
[...]

/home/ubuntu/.kde/share/apps/simond/models/default/samples//angehören_S1_2010-07-12_19-54-08.0.wav
-> /tmp/kde-ubuntu/simond/default/compile//mfcs/angehören_S1_2010-07-12_19-54-08.0.mfc
[...]
“/usr/local/bin/HVite” -A -D -T 1 -l “*” -o SWT -b SENT-END -C “/usr/share/kde4/apps/simon/scripts/config” -H “/home/ubuntu/.kde/share/apps/simond/models/default/active/basemacros” -H “/home/ubuntu/.kde/share/apps/simond/models/default/active/basehmmdefs” -i “/tmp/kde-ubuntu/simond/default/compile//adaptPhones.mlf” -m -t 250.0 150.0 1000.0 -y lab -a -I “/tmp/kde-ubuntu/simond/default/compile//words.mlf” -S “/tmp/kde-ubuntu/simond/default/compile//train.scp” “/tmp/kde-ubuntu/simond/default/compile//dict” “/home/ubuntu/.kde/share/apps/simond/models/default/active/basetiedlist”
Re-aligninging to base model…
[...]

Read 2697 physical / 9729 logical HMMs
Label file will be used to align each file
Aligning File: /tmp/kde-ubuntu/simond/default/compile//mfcs/angehören_S1_2010-07-12_19-54-08.0.mfc
Created lattice with 3 nodes / 2 arcs from label file
SENT-END ANGEHÖREN SENT-END == [476 frames] -59.0359 [Ac=-28101.1 LM=0.0] (Act=5.4)
[...]
“/usr/local/bin/HHEd” -H “/home/ubuntu/.kde/share/apps/simond/models/default/active/basemacros” -H “/home/ubuntu/.kde/share/apps/simond/models/default/active/basehmmdefs” -M “/tmp/kde-ubuntu/simond/default/compile//classes/” “/tmp/kde-ubuntu/simond/default/compile//regtree.hed” “/home/ubuntu/.kde/share/apps/simond/models/default/active/basetiedlist”
Generating regtree.hed…
Creating regression class tree…
Performing static adaption…
“/usr/local/bin/HERest” -C “/usr/share/kde4/apps/simon/scripts/config” -C “/usr/share/kde4/apps/simon/scripts/config.global” -I “/tmp/kde-ubuntu/simond/default/compile//adaptPhones.mlf” -S “/tmp/kde-ubuntu/simond/default/compile//aligned.scp” -H “/tmp/kde-ubuntu/simond/default/compile/classes/basemacros” -u a -J “/tmp/kde-ubuntu/simond/default/compile//classes” -K “/tmp/kde-ubuntu/simond/default/compile//xforms” mllr1 -H “/tmp/kde-ubuntu/simond/default/compile//classes/basehmmdefs” “/home/ubuntu/.kde/share/apps/simond/models/default/active/basetiedlist”
ERROR [+5010] InitSource: Cannot open source file /tmp/kde-ubuntu/simond/default/compile/classes/basemacros
ERROR [+7010] LoadAllMacros: Can’t open file
ERROR [+5010] InitSource: Cannot open source file /tmp/kde-ubuntu/simond/default/compile//classes/basehmmdefs
ERROR [+7010] LoadAllMacros: Can’t open file
ERROR [+7050] LoadHMMSet: Macro name expected
ERROR [+2321] Initialise: LoadHMMSet failed
FATAL ERROR – Terminating program /usr/local/bin/HERest

I would like to adapt the German simon Voxforge base model. But obviously, there is a problem with HERest.

1. Take a look at this install advice.

2. Remove -m32 in configure.ac.

3. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ ./configure

4. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ make

5. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ sudo make install

6. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ sudo apt-get install autoconf

7. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ autoconf

8. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ ./configure

9. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ make all

10. ubuntu@ubuntu-desktop:~/Documents/201005/htk$ sudo make install

11. Control if HTK has been installed by typing HVite into the terminal. HTK seems to be succesfully installed.

<lexeme role="Substantiv">
<grapheme>Immobilieneuphorie</grapheme>
<phoneme>ʔɪmoːbiːliːənɔɪ̯foːʀiː</phoneme>
</lexeme>

When you import the dictionary into simon, this word is displayed as follows:

The SAMPA transcription gls I m o: b i: l i: @ n OI f o: R i: contains the glottal stop. Does this improve the recognition result?

Take a look into the Wikipedia:

“In the northern varieties, [ʔ] occurs before word stems with initial vowel. It is not considered a phoneme, but an optional boundary mark of word stems.”

So maybe the glottal stop is not a phoneme but a boundary mark? Maybe it would be better to create a new IPA “phoneme” /ʔɪ/ – or to be more specific: /ʔ͡ɪ/? This “phoneme” doesn’t exist, but if the glottal stop is just a boundary mark, I think about a solution for this issue.

Are there any test results available? I don’t know how HTK does handle the glottal stop. Maybe it is a good decision to treat the glottal stop as phoneme. Maybe it would be better to introduce “new” phonemes? There could be a “normal” /ɪ/ phoneme, and a “new” /ʔ͡ɪ/ phoneme.

I am just thinking out loud. It is great that simon imports the glottal stop. I would like to know: does this improve the recognition results? I haven’t tested it.

1. Get spelling dictionary, license is GPLv2.

2. Convert mk_MK.dic from cp1251 to UTF-8 via Ubuntu terminal:

am3msi@am3msi-desktop:~/Documents/201004/macedonian-dictionary$ iconv -f cp1251 -t UTF-8 macedonian-utf8.dic

3. Convert mk_MK.aff

am3msi@am3msi-desktop:~/Documents/201004/macedonian-dictionary$ iconv -f cp1251 -t UTF-8 macedonian-utf8.aff

4. Changing the first line of the file macedonian-utf8.aff from SET microsoft-cp1251 to SET UTF-8 with gedit.

5. The unmunch command wouldn’t bring additional Macedonian words.

6. Adding <lexicon> tags to the file macedonian-utf8.dic (<lexicon> at the beginning of the file; </lexicon> at the end of the file).

7. Generating .xml file:

am3msi@am3msi-desktop:~/Documents/201004/macedonian-dictionary$ saxonb-xslt -ext:on -s:macedonian-utf8.dic -xsl:'http://spirit.blau.in/simon/files/2010/04/create-xml-file.xsl' -o:macedonian.xml

8. Thinking about the style-sheet improve-macedonian-dictionary.xsl. This style-sheet should contain the language tag mk. The grapheme-to-phoneme conversion should follow this table. Great:

“Macedonian orthography is consistent and phonemic in practice, an approximation of the principle of one grapheme per phoneme. A principle represented by Adelung’s saying, “write as you speak and read as it is written” („пишувај како што зборуваш и читај како што е напишано“).”

This means that Ralf's Macedonian dictionary shouldn’t be too bad.

9. Generating <phoneme> elements via Ubuntu terminal:

am3msi@am3msi-desktop:~/Documents/201004/macedonian-dictionary$ saxonb-xslt -ext:on -s:macedonian.xml -xsl:'improve-macedonian-dictionary.xsl' -o:macedonian-dictionary.xml

10. Download Ralf's Macedonian dictionary, and import it into simon. Take a look at the Shadow Vocabulary:

Word column: Macedonian words
Pronunciation column: Corresponding SAMPA pronunciation

11. It should be possible to train the word геометриска:

At the moment, I am not able to record this word because my sound configuration has to be adjusted. But the important thing is: You should be able to get some initial recognition results with Ralf's Macedonian dictionary. It would be nice if somebody used Ralf's Macedonian dictionary for training of some Macedonian words. I would like to know whether the Macedonian/Cyrillic alphabet causes issues with HTK or not. My guess is that it works, but I haven’t tested it.

1. Get the simon “Tastatur” scenario.
2. Applications > Universal Access > simon
3. simon > Settings > Configure simon… > Model Settings > General > Static model
4. Download voxforge_de_simon-2010-04-21_2.tar.bz2, and extract it.
5. Setting the path to the files hmmdefs, macros, stats, and tiedlist. Press the Apply button. Press OK.

6. Manage scenarios > Import > Download. I press the Close button because I don’t want to download the scenario directly with simon. I want to use the already downloaded “Tastatur” scenario. So I use Import > Import from file. I choose the file /home/am3msi/Documents/201004/tastatur-scenario/123547-keyboard.scenario. simon displays an error message:

Could not load scenario.

OK, I am using simon 0.2.91. This is not the current development version.

7. I am trying Import > Download > Tastatur [press Install button]. Press Close button. Again, an error message appears:

Could not init scenario “123547-keyboard.scenario”

8. I should get kdelibs 4.4.0. But this requires a partial upgrade of my Ubuntu 9.10 system. I want to wait with the upgrade until Ubuntu 10.04 has been released. This will be next week. I don’t want to break my Ubuntu system which is working quite fine. This means that at the moment I can’t test the current simon development version.

  <word>
   <name>Abbrechen</name>
   <pronunciation>gls a p b R E C @ n</pronunciation>
   <terminal>SimpelKommando</terminal>
  </word>
  <word>
   <name>Acht</name>
   <pronunciation>a x t</pronunciation>
   <terminal>Nummer</terminal>
  </word>

Obviously, simon does now import the glottal stop. The word Abbrechen is with glottal stop. The word Acht is without glottal stop. More info:

Introduced the “gls” phoneme for a glotal stop (“?” in SAMPA).

Great. This should improve the recognition rate when you use Ralf's German dictionary for training. As far as I know, the German BOMP dictionary does have glottal stops, too.

The glottal stop is a difficult phoneme. I think that it is a good decision if simon supports the glottal stop.

The abbreviation gls is better than kn.

Additional thought: Maybe it is possible to develop a simon scenario (XML file) with an .xsl style-sheet? You all know that I am primarily interested in dictation, so a simon scenario with let’s say 1.000 German words would be an interesting start. This should be possible, but I don’t want to invest too much time. So I am looking for a way to get quick results.

My dream is an XML-based framework:
- SSML prompts that link to FLAC audio files;
- Ralf's German dictionary (PLS);
- an .xsl style-sheet that accesses e.g. 99 SSML prompts for training (source file), and that chooses the corresponding words from the PLS dictionary (source file);
- the object file could be a simon scenario.

The command in the Ubuntu terminal would look as follows:

$ saxonb-xslt -ext:on -s:'http://script.blau.in/german/35/prompts.xml' -xsl:'create-simon-scenario.xsl' -o:simon-scenario.xml

What do you think? It is possible to access files that are located in the Internet directly from the Ubuntu terminal, and process them.

Let’s take a look into simon > Settings > Configure simon... > Model Settings:

It should be possible to load exactly these four files (hmmdefs, macros, stats, and tiedlist) into simon.

Questions:
1. Is it necessary to import Ralf's German dictionary, too?
2. Is it necessary to define a grammar?
3. Does recognition work out of the box when I load the files hmmdefs, macros, stats, and tiedlist into simon?