I want to export the following information: Filename, Expected result, Actual result, Recognition rate (below 50%). The resulting document should be a simple text file (or XML file or whatever). Is this possible with the current Export test result function of simon?

Take a closer look at the right column: The word DIVERGIERST has a recognition rate of 100%. The word DIN has a recognition rate of 11.9569%. You can see that the sorting is not correct.

Btw, it would be interesting to export the recognition results into a text file. What about adding an Export test results button?

/home/ubuntu/.kde/share/apps/simon/model/hmmdefs
/home/ubuntu/.kde/share/apps/simon/model/tiedlist
/home/ubuntu/.kde/share/apps/simon/model/model.dict
/home/ubuntu/.kde/share/apps/simon/model/model.dfa
/home/ubuntu/Documents/201008/wav-all
/home/ubuntu/Documents/201006/audacity/xea-folder/wav-xea
/home/ubuntu/Documents/201008/sam-0.1.6/lexicon
/home/ubuntu/Documents/201008/sam-0.1.6/model.grammar
/home/ubuntu/Documents/201008/sam-0.1.6/simple.voca
/home/ubuntu/Documents/201008/prompts-wav.txt
/home/ubuntu/Documents/201006/audacity/xea-folder/prompts-xea
/usr/share/kde4/apps/simon/model/tree1.hed
/usr/share/kde4/apps/simon/model/wav_config
16000
/usr/share/kde4/apps/simond/default.jconf
2

Question: what does the 2 in the last line mean?

By the way, I used the Serialize scenarios button (input file:///home/ubuntu/Documents/201007/german-speech-model-0.1.6/scenario-0.1.6.xml) to generate the following files:
file:///home/ubuntu/Documents/201008/sam-0.1.6/lexicon
file:///home/ubuntu/Documents/201008/sam-0.1.6/model.grammar
file:///home/ubuntu/Documents/201008/sam-0.1.6/simple.voca

I think that it is a good idea to test a subset (e.g. xea) against the superset (xaa+xab+xac+…+xpw).

“You can sort the files by each column simply by clicking on the column header. This way it is very easy to find bad samples by sorting by recognition rate.”

This function worked partially on my computer (when using a previous development version of sam). I hope that this issue will be fixed. Some words were sorted beginning from lowest recognition rate to the highest recognition rate, but not all.

What about a function that displays all words that are below a specific recognition rate (e.g. “display all words that are below 90% recognition rate”)? I need a tool that helps me to sort out the bad audio files as efficiently as possible. I hope that a future version of sam will do the job.

(a) HMM-definitions
(b) Tiedlist
(c) Dict
(d) DFA

When I am looking into simon, I can see the following options:

(a) Hmm Definition
(b) Tiedlist
(c) Macros
(d) Stats

I find it confusing:
- (a) and (b) obviously the same: sam can output the (a) Hmm Definition; simon can use the (a) Hmm Definition as input. sam can output (b) Tiedlist; simon can use the (b) Tiedlist that sam produced as input. So far so good.
- but what is with (c) and (d)? Is (c) Dict the same as (c) Macros? Is (d) DFA the same as (d) Stats?

Why does sam produce (c) Dict and (d) DFA as output files? Is it possible to use these sam output files as simon input files?

Some clarification would be helpful.

$ cd Documents/201001/speech2text
$ git pull origin master
$ ./build_ubuntu.sh

It is working.

I would like to know how sam > Input & output files > Serialize scenarios | Serialize prompts is working. Is there a tutorial available?

My question is: how can I import the model that I have built/tested with sam into simon?

simon offers to Manage scenarios: Import and Export are offered. I tried the Export button. This created an XML file:

<!DOCTYPE scenario>
<scenario version=”1″ icon=”simon” name=”General” lastModified=”2010-02-03T14:06:50″>
<simonCompatibility>
<minimumVersion>
<version>0.2.82</version>
</minimumVersion>
<maximumVersion/>
</simonCompatibility>
<authors>
<author>
<name>Anybody</name>
<contact>no@mail</contact>
</author>
</authors>
<licence>GPL</licence>
<vocabulary/>
<grammar>
<structure>Unknown</structure>
</grammar>
<actions/>
<trainingtexts/>
</scenario>

I would like to be able to import my sam speech model into simon. How can I do this?

What can I import into simon?
- I can press the Import Dictionary button to import an active dictionary and/or a shadow dictionary.
- I can switch to the Grammar tab, press the Import button. I didn’t try this function yet. I am not too interested because currently I don’t need a grammar function (200 German words could be recognized without a grammar – only the terminal Unknown was used; 1000 German words should be possible without grammar, I hope). Of course, if I want to restore my German speech model (53 MB), it is necessary to restore the grammar, too. So this Import [Grammar] button might be useful.
- In the Training tab, I can press the Import Trainingsdata button. This should import the prompts file and the corresponding wav files (stored in the training.data folder).

When I take a look at sam > Static model, I can see fields for Base macros and Base stats. Where are these files from my own German 200 words speech model located? When I download the English acoustic model from Voxforge (HTK_AcousticModel-2010-02-03_16kHz_16bit_MFCC_O_D.tgz), I can see the following files:

1. macros – this file is probably usable with sam > Input & output files > Static model > Base macros.
2. stats – probably usable with sam > Input & output files > Static model > Base stats.

When I build my own German speech model with sam, where are these files – macros and stats – located? Where can I find them? I assume that I need them if I want to restore my speech model for the usage with simon, but I am not sure.

This is what I want: Restore my 200 German words speech model (my current problem). Then I want to add more words to this speech model. I am planning to add about 10 words per day on average to my German speech model. It should grow continously. And if something goes wrong, I want to be able to restore from my backup file because I don’t want to begin again and again from scratch.

The Manage scenarios > Import and Export buttons might be of help in the future.

I already said it earlier, and I say it again because it is important: the user doesn’t want to loose his own work (= wav recordings that were made with simon). It should be possible to backup (=export) and to restore (=import) all files that are necessary to build a working speech model.

For me, it is OK to specify each specific path like it is possible with sam. But in the end, it has to work with simon.

I want to fine tune my German speech model with sam. Especially, I want to sort out wav files that have a low recognition rate with sam. After I have fine-tuned my German speech model with sam, I want to use it with simon (=import it into simon). How is this possible?

I am using the following path for the jconf file (I had to look into this blog post): /usr/share/kde4/apps/simond/default.jconf

This is the current content of the file /home/am3msi/Documents/201001/model/20100126-try-to-restore-german.sam:

/home/am3msi/Documents/201001/model/hmmdefs
/home/am3msi/Documents/201001/model/tiedlist
/home/am3msi/Documents/201001/model/model.dict
/home/am3msi/Documents/201001/model/model.dfa
/home/am3msi/Documents/201001/model/training.data/
/usr/share/kde4/apps/simond/default.jconf
/home/am3msi/Documents/201001/model/lexicon
/home/am3msi/Documents/201001/model/model.grammar
/home/am3msi/Documents/201001/model/model.voca
/home/am3msi/Documents/201001/model/prompts
/home/am3msi/Documents/201001/model/training.data/
/home/am3msi/Documents/201001/model/tree1.hed
/home/am3msi/Documents/201001/model/wav_config
16000
/home/am3msi/Documents/201001/model/prompts

Now, I click the Build model button.

1. I pressed the Build model button.
2. The Build log indicates that it worked out. Great. I assume that the previously existing files
/home/am3msi/Documents/201001/model/hmmdefs
/home/am3msi/Documents/201001/model/tiedlist
/home/am3msi/Documents/201001/model/model.dict
/home/am3msi/Documents/201001/model/model.dfa
have been replaced by new ones (probably with the identical content).

Now I want to test the model. So I press the Test model button. sam displays an error message:

Couldn’t open prompts file for reading: /home/am3msi/Documents/201001/model/training.data/

Why is that? What went wrong? Let’s take a look at the paths to the test files:
/home/am3msi/Documents/201001/model/training.data/
/usr/share/kde4/apps/simond/default.jconf
/home/am3msi/Documents/201001/model/prompts

The paths are correct. I am trying the following: I copy the prompts file to the training.data folder. But this didn’t solve my problem.

My guess is that there is a bug with sam. At least, it is possible to build a speech model with sam (from my German backup files). That is a good start. That means that my German wav recordings, my dictionary, my prompts aren’t lost.

My next step will be to take a closer look at simon. I will try to use my German backup files with simon. They worked with sam (only the Test model function failed, but the Build model function obviously worked). And I hope that they will work with simon, too.

1. I tested sam with Sample.jconf, and it seemed to work. But to be honest: Isn’t this just a dummy file? Every line begins with an # (number sign).

2. adin.jconf is very short, and everything is commented out.

3. default.jconf is probably the correct choice because some lines are valid, e.g.:

[...]
-h hmmdefs
[...]
-hlist tiedlist
[...]
-penalty1 5.0 # first pass
-penalty2 20.0 # second pass
[...]

Location of this file: /usr/share/kde4/apps/simond/default.jconf
I think that I will use sam with this file.

B. I assume that sam > Build model is the same like simon > Synchronize because the button is the same (green circular arrow).

This means that I can use simon for recording new words, and synchronize the speech model. In my last video, all words (more than 200 words) were recognized correctly. How can I eliminate wav files that are not so good? I need an efficient way to fix future problems. sam seems to fill the gap. With simon, I can record words, and synchronize. With sam, I can find out which words are below 100% confidence score. These words can be edited with sam (very nice feature).

So, simon is good for the main work. Fixing bad wav files can be done with sam. E.g. I found a wav file with a confidence score of about 89%. I edited this wav file with sam (sam offers the option to re-record a wav file). I will see whether this improves the confidence score after rebuilding (sam > Build model), and testing the model (sam > Test model).

C. It seems that sam is working without ksimond.

D. I don’t know yet how to handle wav files with multiple words. With sam, the confidence score of these wav files is 0 %. I added with simon the grammar structure “Unknown Unknown Unknown Unknown“, but it still doesn’t work out. But now, I found one sample which had a confidence score of 50%:

Result 6 of 10
=====================
Sentence: verlangsamende verlangsamendem verlangsamenden verlangsamendes
SAMPA:
Raw SAMPA:
Average Confidence: 50
Confidence Scores: 1.72309e-12 100 4.8642e-05 100

So I can say: it seems to work when I am adding the sentence structure “Unknown Unknown Unknown Unknown“.

E. The word deutschfeindlichen has a confidence score of 90.0133 %:

Result 1 of 3
=====================
Sentence: deutschfeindlichen
SAMPA:
Raw SAMPA:
Average Confidence: 90.0133
Confidence Scores: 90.0133

Result 2 of 3
=====================
Sentence: deutschfeindlichem
SAMPA:
Raw SAMPA:
Average Confidence: 9.98674
Confidence Scores: 9.98674

Result 3 of 3
=====================
Sentence: deutschfeindliche
SAMPA:
Raw SAMPA:
Average Confidence: 8.29019e-17
Confidence Scores: 8.29019e-17

So the alternatives deutschfeindlichem and deutschfeindliche have a much lower confidence score. This is fine. I can see how good it works internally. Because in the video (see link above), you can see 100 % perfect results. But internally, it is just about 90 % for deutschfeindlichen.

Here is another example Wortbreite:

Result 1 of 2
=====================
Sentence: Wortbreite
SAMPA:
Raw SAMPA:
Average Confidence: 0.0408209
Confidence Scores: 0.0408209

Result 2 of 2
=====================
Sentence: Wortbreiten
SAMPA:
Raw SAMPA:
Average Confidence: 99.9592
Confidence Scores: 99.9592

It should have recognized Wortbreite, but it recognized with more than 99 % confidence score the word Wortbreiten. So this match is wrong.

F. Conclusion: I hope you got an impression of sam. sam seems to be a great tool for speech model development.

By the way, currently I am building an additional German pronunciation dictionary that someday will contain the missing words from my German audio files. When I have the missing words in the dictionary, I hope that the testing process with sam will give me good results.