Posts Tagged ‘saxonb-xslt’

Ralf’s German dictionary 0.1.9.7

Thursday, June 17th, 2010

This article explains how I am preparing Ralf's German dictionary 0.1.9.7. At moment, I am focusing on the creation of new <phoneme> elements:

1. Add the following rule to improve-german.xsl:

<xsl:when test="ends-with(lower-case(../grapheme), 'ier') and
ends-with(., 'iːʀ') and
not(ends-with(../grapheme, 'eier'))"><xsl:value-of select="replace($sierra, 'iːʀ', 'iːɐ̯')"/></xsl:when>

2. Add this rule:

<xsl:when test="ends-with(../grapheme, 'gen') and
not(ends-with(../grapheme, 'ngen'))"><xsl:value-of select="replace($sierra, 'gən', 'gŋ̩')"/></xsl:when>

3. Invoke the following instruction via Ubuntu terminal because I want to test whether improve-german.xsl produces the desired results:

ubuntu@ubuntu-desktop:~$ saxonb-xslt -ext:on -s:'/home/ubuntu/Documents/201006/german-0.1.9.5/german-0.1.9.6.xml' -xsl:'/home/ubuntu/Documents/201005/german-0.1.9.4/improve-german.xsl' -o:'/home/ubuntu/Documents/201006/german-0.1.9.5/german-0.1.9.7.xml'

4. Add the following rule:

<xsl:when test="contains(../grapheme, 'gens') and
not(ends-with(../grapheme, 'ngens'))"><xsl:value-of select="replace($sierra, 'gəns', 'gŋ̩s')"/></xsl:when>

What does this rule create? Here is an example:
Source dictionary:

<lexeme role="Substantiv">
<grapheme>Volkswagens</grapheme>
<phoneme>fɔlksvagəns</phoneme>
</lexeme>

Object dictionary:

<lexeme role="Substantiv">
<grapheme>Volkswagens</grapheme>
<phoneme>fɔlksvagəns</phoneme>
<phoneme>fɔlksvagŋ̩s</phoneme>
</lexeme>

You can see that I am using XSLT to produce additional <phoneme> elements.

5. Add this rule:

<xsl:when test="ends-with(../grapheme, 'bens')"><xsl:value-of select="replace($sierra, 'bəns', 'bm̩s')"/></xsl:when>

Example: Source dictionary:

<lexeme role="Substantiv">
<grapheme>Schneetreibens</grapheme>
<phoneme>ʃneːtʀaɪ̯bəns</phoneme>
</lexeme>

Object dictionary:

<lexeme role="Substantiv">
<grapheme>Schneetreibens</grapheme>
<phoneme>ʃneːtʀaɪ̯bəns</phoneme>
<phoneme>ʃneːtʀaɪ̯bm̩s</phoneme>
</lexeme>

6. Add the following rule:

<xsl:when test="ends-with(../grapheme, 'ben')"><xsl:value-of select="replace($sierra, 'bən', 'bm̩')"/></xsl:when>

Example from the source dictionary:

<lexeme role="Verb">
<grapheme>ausgeben</grapheme>
<phoneme>ʔaʊ̯sgeːbən</phoneme>
</lexeme>

Target dictionary:

<lexeme role="Verb">
<grapheme>ausgeben</grapheme>
<phoneme>ʔaʊ̯sgeːbən</phoneme>
<phoneme>ʔaʊ̯sgeːbm̩</phoneme>
</lexeme>

With this rule, I added about 1400 <phoneme> elements. It would be too much work to do this manually. Thanks to saxonb-xslt I can work efficiently and precisely.

7. If you have suggestions for improvements of Ralf's German dictionary, please tell me. At the moment, the dictionary contains 384067 <lexeme> elements. I don’t want to add more words to the dictionary at the moment. I want to improve the phoneme quality. And there are a lot of <grapheme> elements that have more than one possible pronunciation. This is my current focus to add a <phoneme> element where it seems to be appropriate. If you are missing something, please tell me.

(more…)

Clear button; improve phoneme

Wednesday, February 3rd, 2010

1. This is what I did a couple of minutes ago:

$ cd Documents/201001/speech2text
$ git pull origin master
$ ./build_ubuntu.sh

After starting simon, I can see that a Clear button is available:

clear

It should now be possible to delete the active dictionary.

2. Let me make an additional remark about the phonemes of Ralf's German dictionary. The phonemes

S t E n d e: k E m pf @
S t E n d e: O R g a n i: z a ts I o: n

are not optimal. e: indicates a long vowel. Instead, there should be the short vowel @. When you watch the video 200 German words, then you can listen how I pronounce these words. I pronounce them with e: (long e) instead of @ (short e). Such problems occur often in Ralf's German dictionary.

How do I modify Ralf's German dictionary? I use the Ubuntu terminal:

$ saxonb-xslt -ext:on -s:german-dictionary-0.1.7.xml -xsl:modify-german-dictionary.xsl -o:prepare-0.1.8.xml

To avoid a special java heap space error, I run VisualVM in the background. The result is that it is possible to modify Ralf's German dictionary with the XSLT style-sheet.

Why am I telling you these details that are not directly related to simon? Because you need a pronunciation dictionary if you want to use simon for dictation. And it is necessary to improve the quality of the phonemes that are contained in Ralf's German dictionary.

3. It is possible to modify phonemes with simon using the Edit Word button. My approach with saxonb-xslt is necessary for dictionary development because this way it is possible to modify lots of <lexeme> elements.

4. I removed the active and the shadow dictionary using the Clear button. What about an Export dictionary button? If someone edits the dictionary with simon (Edit Word button), he may want to export the dictionary.

Import of prompts 01 failed

Tuesday, October 13th, 2009

This is what I am currently doing: I checked out revision 1040.

I want to import the prompts 01 into simon. First, I had to convert the 40 flac files into wav files with the following command:

liberty@liberty-desktop:~/200910/editing-ralfherzog/01$ for f in *.flac; do sox "$f" -t wav -r 16000 -s -c 1 "subfolder/${f%.flac}.wav"; done

Then I transformed the file http://script.blau.in/german/01/prompts.xml into (almost) HTK compatible format with the following command:

liberty@liberty-desktop:~/200910/editing-ralfherzog/01$ saxonb-xslt -ext:on -o:PROMPTS01 -xsl:transform-ssml-prompts.xsl -s:prompts.xml

The stylesheet transform-ssml-prompts.xsl has the following content:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<!-- 20091013; license: GPL -->
<xsl:output method="text"/>
<xsl:template match="speak">
<xsl:for-each select="audio">
<xsl:value-of select="replace(@src, 'flac','wav')"/>
<xsl:text> </xsl:text>
<xsl:value-of select="."/><xsl:text>
</xsl:text>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>

I think that I found an error. I forgot to capitalize the prompts with the XPath expression upper-case(). I will have to correct the stylesheet. Probably that is the reason why the Import Trainingsdata function didn’t work out.

Creating Ralf’s German dictionary

Tuesday, September 22nd, 2009

To get an impression how I create the German PLS dictionary, watch the video (19.2 MB, WMV):

[20100101: video removed]

Currently, I am preparing a new version of Ralf’s German dictionary. The dictionary should be 100% simon compatible (version 0.1 contains some minor mistakes).

This is what I did yesterday:
1. I created more than 80.000 pronunciations with eSpeak from a set of 300.000 words. Not all words were transcribed, I don’t know what went wrong.
2. Then I created an XSLT stylesheet to transform the eSpeak phoneset into IPA with saxonb-xslt.
3. The result was that I had a list of the phonemes, but the graphemes are missing. What can I do? I decided to start dictating the missing graphemes with DNS 9.5. You can see the dictation process in the video.