This should come in handy for text classification using WEKA (and hopefully save your time).

During my M.Phil. course, one of the tasks was to build a basic QA system and extend it however we liked. We used the TREC 8 dataset for evaluations. While building the system, I evaluated how current search engines (read Google) performed on this task. For this, I just queried the exact question and used the summaries of the top five results as answers. Evaluating at that time (2008), I got a Mean Reciprocal Rank (MRR) score of 0.212 over 198 questions. 156 questions had no answers found in top 5 responses.

This term, I am demonstrating for the same task. Demonstrators are usually PhD students who provide help and guidance to junior students. For pure geek fun and lack of better things to do while taking a break, I decided to quickly jolt down a JavaScript (read  JQuery ) based QA system. This time,  the resulting MRR score over 198 questions was 0.384 while only 79 questions had no answers found in top 5 responses.

The results show clearly that during the last two years, Google has significantly improved on answering NLP queries. In fact (IIRC), my baseline system back in 2008 (based on RMRS based matching of sentences from the top 100 documents returned by an IR system) could only achieve an MRR score of approximately 0.290, showing that the current results are much better than that baseline. I hope this decade sees some more developments/improvements in QA systems and I can ask a system What do you get if you multiply six by nine?

I’ve always said there was something fundamentally wrong with the universe. ~Arthur Dent

1. a good mic (recommended but not necessary)
2. a quite room
3. any software which can cut and save wav files. I use Praat.
4. the console version of Praat, called PraatCon. This will be used to extract the formants from wav files.
5. about 30 mins.

Once you have all these things, here is what you do for recording the phone sounds

1. Start Praat. You will see two windows. You can close the other window but keep the one titled Praat Objects open.
2. In the menu, select New/Record mono sound. Select sampling frequency to 16000 Hz, press Record and start speaking in the mic. When you are through, press Stop. Speak all the phones you want in a single go but with silence in between. In the screenshot below, AA (as in father) is spoken multiple times.
3. Press Save to list and then Close. You should now an entry in the Objects listbox. Make sure the entry is select and then press Edit.
4. Select the phone for which you want to extract the formants by pressing the mouse button on the start of the phone and dragging it to the point right before it finishes. Please note that we will be using the mean formant value for a phone so it is essential that only the middle part of the phone is selected to ensure accuracy and no other noise or silence is present. The selected area will be highlighted.
5. From the File menu, select Write selected sound to WAV file, enter the name of the wav file and save.
6. Repeat step 4 and 5 for all the phones you want to save.

That’s it for the recording part. Alternately, if you already have a collection of sound files with a single phone, you can use that. You can download one such collection here. The zip file has 40 files where 2 speakers have spoken AA (as in father) and II (as in been) 10 times each.

Click here to download script and sound files.

Although you can use the GUI based Praat to find out the formant values for each phone, we want to automate the task. To do so, we will use the console version of Praat called PraatCon, and run a script to extract the formants. The small Praat script written below takes the name of a phone file as argument and outputs the first two formats F1 and F2 to the console.

form Display mean F1 and F2
   sentence filename
endform
filename_noext$ = replace$ (filename$, ".wav", "", 0)

Read from file... 'filename$'
select Sound 'filename_noext$'
To Formant (burg)... 0 5 5500 0.025 50
f1 = Get mean... 1 0 0 Hertz
f2 = Get mean... 2 0 0 Hertz
clearinfo
print 'filename$' 'f1''tab$''f2''newline$'

This script along with a small batch file (run.bat) which runs this script for all wav files present in the current directory, can be downloaded with all the sound files by clicking on the link below. To get this running, extract all files in a directory and place your downloaded copy of PraatCon in the same directory. The just click on the run.bat icon and watch the magic!

Click here to download script and sound files.

The sound files in the archive give this nice little graph when their F1 and F2 values are plotted as an XY scatter graph shown below. All this was done to construct a linearly separable dataset which can be given to students for training a perceptron to distinguish between two phones. Any suggestion/comments are welcome.

Click here to download script and sound files.

I made a small 10 min demo to simulate the classic river crossing puzzle for teaching state space search in my AI class. I did exercise my right to be creative by changing the characters a bit. Jerry, Tom and Spike have to cross a river, going from the right side to left, in a boat. While you are in the boat, they behave. But as soon as you leave Tom with Jerry or Tom with Spike on one side WITHOUT the boat, they start fighting. Being a peace loving person, you don’t want that. The goal is to move them across the river with no fights. (as you may have noticed, there is no eating/killing involved in my version)

Here, we can represent each object by a Boolean variable and draw the states accordingly. The picture above can be represented by 1 0 1 0 where the left most 1 represents that the boat is on the left bank and the zero after that tells that Jerry is on right one. Similarly the next 1 and 0 denote Tom on the left bank and Spike on the right one respectively.Here is a link to the executable in case anyone wants to play it or give it to a kid to figure out. Drop me an email if you want the source code (C#). The executable requires an installation of the Microsoft .NET framework 3.5.

Download River Crossing Puzzle

Microsoft .NET framework 3.5

While preparing the first few lectures, I realized I needed a simulation of the 8 Queens Problem using Breadth First Search and Depth First Search. While there are many solutions on the Internet, I couldn’t find any program which lets me step through the algorithm. Besides, it had been a long time since  I did any programming in Windows.Forms so my code deprived fingers needed a sink.

So here’s my version of the 8 queens problem solver. Each board state is represented by a string. The string represent the columns starting from the left most column and the number in that position tells the row in which a queen is present.  The screenshot above thus represents the state 03142. The state 0314 would represent an empty column 4. You can download the compiled binary by clicking on the link below. I will try to upload the source once the assignment deadline has passed. It was developed in C# so you might have to install Microsoft .NET framework 3.5 to run this program. Comments/Suggestions/Bug Reports are welcome.

Download the 8 Queen Problem Solver