Chaoticity » OpenWare

Making a copy of WEKA Instances

awais — Mon, 12 Apr 2010 21:54:09 +0000

This ‘thing’ took about 30 minutes to figure out. According to the WEKA documentation, if you add a new Instance to an existing Instances object, String values are not transferred ! In case you are working on copying a dataset with a string attribute, you need to transfer the string manually. The code segment below copies the i^th instance from source to dest where the first attribute (at index 0) is a string attribute.

1
2
3

dest.add(source.instance(i));
dest.instance(dest.numInstances()-1)
.setValue(0,source.instance(i).toString(0));

This should come in handy for text classification using WEKA (and hopefully save your time).

Online English to Urdu Translator

awais — Sat, 23 Jan 2010 19:39:33 +0000

While all the online English to Urdu translators that I have seen don’t really work that well (read suck), if we make use the overlapping vocabulary and grammar of Hindi and Urdu along with using Google’s translation API, things come out pretty decent (as mentioned in my previous post). Here’s a small 15 min first cut script which just uses English to Hindi translation and then transliterates from Hindi to Urdu. Feel free to use the code and do ping me if you improve something. This works as a Hindi to Urdu transliterator as well.

How are you?
آپ کیسی ہیں؟

(Thanks to عزت مآب جناب آغا علی رضا قزلباش رحمتہ اللہ علیہ who graciously sent me his term report on Hindi to Urdu transliteration, from where I’ve copied (and modified) the character mapping.)

What do you tweet about? : A shell script for getting most frequent words for twitter

awais — Fri, 18 Dec 2009 20:29:51 +0000

There are a lot of web apps around which report your twitter stats. But at times, it’s better to do things yourself. I haven’t done any fun coding for ages now so last night, I finally got around to making a small program to gather twitter word statistics. The fun part was to do everything using unix tools. Here’s a small script file which displays the 10 most used words in the tweets for any twitter id. I have only tested it under cygwin so this is probably the best place to say “USE AT YOUR OWN RISK”.

Here’s how it works.

downloads all status information in a directory
extracts the status message lines
does some regex magic and filters stop words like the, a, an etc. ( haven’t seen this done earlier anywhere but the join command comes in handy for processing stopwords)
displays the top 10 most frequent words (and emoticons)

Twitter assigns a limit to the number of messages that you can download (3200). Also, the twitter id timeline has to be public for this script to work. All you need to do is download the script file and stop word list, keep them in the same directory, run it with the twitter id in the command line and you’ll get the list of words with the frequency at the start of each line. For example,

$ ./tword.sh barackobama
161 watch
119 live
92 http://mybarackobamacom/livestream
81 health
63 reform
55 today
52 rally
48 #hc09
47 &
38 vote

The script takes time to complete so be patient. As you may have noticed, there are still html tags inside. You can remove them by piping in any html2text program. There’s a small perl script in the zipfile which does this processing. The output now brings in a new word “change”. You will, however, need to pipe this in the script after installing HTML::Entities though CPAN.

$ ./tword.sh barackobama
161 watch
119 live
92 http://mybarackobamacom/livestream
83 health
68 change
63 reform
55 today
55 rally
48 #hc09
39 vote

My list toppers as good, , time, day, twitter, read, hope, back, :p and make. I wonder if this makes me a happy person

JabRef and Google Scholar

awais — Sat, 14 Nov 2009 16:57:18 +0000

I can’t seem to find any way to import the bib entries provided by google scholar to JabRef directly. You can enable the Import into BibTex link from the preferences but it streams the bib file as text/plain which opens up in the browser. You can save it and import it but that wastes a lot of clicks. The easiest option is to copy-paste all the text into a new JabRef entry (Ctrl+N). The default settings leave the double curly braces in the title (to preserve case) which can be removed by enabling the Remove double braces… checkbox in the File tab of Options/Preferences. This works for JabRef 2.5.

Are you interested in using computers in the classrooms?

awais — Fri, 31 Jul 2009 16:02:44 +0000

A friend of mine is carrying out research in classroom based e-assessment in developing countries such as Pakistan. The aim of the research is to assist primary school teachers with computer software that:

· Is aligned with the particular subject curriculum they follow in their schools.

· Provides pupils with challenges and interactive short quizzes and tests to take after completing a topic taught by the teacher in the classroom.

· Provides students with immediate and diagnostic feedback on their performance on each challenge or test they attempt.

· Helps teachers in identifying the individual pupils needing help in certain conceptual areas of the curriculum, in managing the overall classroom portfolio, and in assuring better teaching and learning within the socio-cultural context of their educational system.

You can help in two ways!

1. The project is still in the stage where available literature on formative e-assessment is being critically reviewed and any bright ideas relevant to the topic are welcome.

2. One very important aspect of the research is related to the kind of tools and technologies that should be utilized in developing the software product. This is done keeping into consideration the fact that the use of expensive tools, technologies and infrastructure does not help much in sustaining any change in the educational systems of developing countries. Therefore, Moodle (www.moodle.org) and other such open source learning management systems are being considered for the initial version of this project.

We need someone who can evaluate both Moodle and OpenMark, tell whether the functionality of diagnostic assessment used in OpenMark can also be integrated in Moodle, and how. Monetary remuneration is available for this activity.

The Open University (www.open.ac.uk) in the UK is one of the pioneers in establishing distance learning (also e-learning) programs for higher education. They are currently using OpenMark (https://openmark.dev.java.net/), their own open source Computer Assisted Assessment (CAA) system, as well as Moodle to develop formative assessment tests for their students enrolled in distance learning programs (http://labspace.open.ac.uk/course/view.php?id=3484&topic=all). You will find some documentation on this at http://labspace.open.ac.uk/course/view.php?id=3484&topic=all, and at http://labspace.open.ac.uk/mod/resource/view.php?id=381989&direct=1. After that we would need help in taking on the development of our own software from there.

If you know anyone who is interested, please leave a comment or drop an email to awais {at} chaoticity.com

Solving the 8 Queens Problem

awais — Fri, 19 Jun 2009 20:30:21 +0000

I will be teaching a course in Artificial Intelligence this summer. For some parts of the course, I plan to show implemented demos along with the course material. Looking from the students’ point of view, such tools not only decrease the learning time while in the class room, but also give enough material to play around at home in case you want to repeat what was taught. (I can empathise because I was once, and will be again (hopefully soon), a student).

While preparing the first few lectures, I realized I needed a simulation of the 8 Queens Problem using Breadth First Search and Depth First Search. While there are many solutions on the Internet, I couldn’t find any program which lets me step through the algorithm. Besides, it had been a long time since I did any programming in Windows.Forms so my code deprived fingers needed a sink.

So here’s my version of the 8 queens problem solver. Each board state is represented by a string. The string represent the columns starting from the left most column and the number in that position tells the row in which a queen is present. The screenshot above thus represents the state 03142. The state 0314 would represent an empty column 4. You can download the compiled binary by clicking on the link below. I will try to upload the source once the assignment deadline has passed. It was developed in C# so you might have to install Microsoft .NET framework 3.5 to run this program. Comments/Suggestions/Bug Reports are welcome.

Download the 8 Queen Problem Solver

OpenWare Education: BLOSSOMS project at MIT

awais — Wed, 21 Jan 2009 20:23:06 +0000

There was a talk today by Dr. Richard Larson at FAST-NU on an initiative by LINC called BLOSSOMS (Blended Learning Open Source Science or Math Studies). It focuses on developing open ware modules for high school students which can be recorded and shared in classrooms. The creative twist is to create a ‘blend’ between the video lectures and an in class teacher. Participants are to be selected from US (MIT), Jordan and Pakistan.

Its open for students as well as teachers and virtually anyone can create content to teach high school students. Some of the related videos can be viewed from MIT TechTV. If anyone is interested, he/she/it should drop Dr. Larson an email.