Chaoticity » chaos

Where Iqbal lived in Cambridge

awais — Tue, 06 Jul 2010 20:09:38 +0000

During his undergrad (if you can call it that), Iqbal read at Trinity College, Cambridge. By current definition of the phrase, he was a ‘mature student’. He stayed at 17 Portugal Place.At that time, the house might have been college-owned, but I can’t confirm that.

It’s a smallish house with a narrow street on one side and a wider one on the other. The wider street opens up in Jesus Green, a large ground. The house is a 5 minute walk away from the river Cam.

I wish I could go and live there for some time, just to check if creativity is influenced by proximity to greatness, even if it’s time-shifted. Without going into any more details, here are some pictures.

observing infinities

awais — Thu, 01 Jul 2010 21:39:55 +0000

Making a copy of WEKA Instances

awais — Mon, 12 Apr 2010 21:54:09 +0000

This ‘thing’ took about 30 minutes to figure out. According to the WEKA documentation, if you add a new Instance to an existing Instances object, String values are not transferred ! In case you are working on copying a dataset with a string attribute, you need to transfer the string manually. The code segment below copies the i^th instance from source to dest where the first attribute (at index 0) is a string attribute.

1
2
3

dest.add(source.instance(i));
dest.instance(dest.numInstances()-1)
.setValue(0,source.instance(i).toString(0));

This should come in handy for text classification using WEKA (and hopefully save your time).

Google as a Question Answering System

awais — Sat, 06 Feb 2010 04:34:58 +0000

A Question Answering (QA) system is an Information Retrieval system which gives the answer to a question posed in natural language. For example, if you ask it Who wrote Hamlet?, it should answer Shakespeare. A few years ago (don’t ask me how many), search engines did not focus on language queries. Recently [sic], Google has started incorporating some NLP (Natural Language Processing) in their results. You can try it out by typing the same question in the search box yourself ( or clicking here ).

During my M.Phil. course, one of the tasks was to build a basic QA system and extend it however we liked. We used the TREC 8 dataset for evaluations. While building the system, I evaluated how current search engines (read Google) performed on this task. For this, I just queried the exact question and used the summaries of the top five results as answers. Evaluating at that time (2008), I got a Mean Reciprocal Rank (MRR) score of 0.212 over 198 questions. 156 questions had no answers found in top 5 responses.

This term, I am demonstrating for the same task. Demonstrators are usually PhD students who provide help and guidance to junior students. For pure geek fun and lack of better things to do while taking a break, I decided to quickly jolt down a JavaScript (read JQuery ) based QA system. This time, the resulting MRR score over 198 questions was 0.384 while only 79 questions had no answers found in top 5 responses.

The results show clearly that during the last two years, Google has significantly improved on answering NLP queries. In fact (IIRC), my baseline system back in 2008 (based on RMRS based matching of sentences from the top 100 documents returned by an IR system) could only achieve an MRR score of approximately 0.290, showing that the current results are much better than that baseline. I hope this decade sees some more developments/improvements in QA systems and I can ask a system What do you get if you multiply six by nine?

I’ve always said there was something fundamentally wrong with the universe. ~Arthur Dent

Visualizing Citation Networks

awais — Thu, 04 Feb 2010 04:59:07 +0000

For techies: I’ve been working on citation networks lately. You can visualize such a network as a graph. In this graph, the nodes represent publications (papers,articles etc) and the edges represent citations between them. The graph above was produced using the GraphViz. The data is from the ACL Anthology Network which contains publications from the publicly available ACL Anthology.

For non-techies: Oooooo! pretty picture!

A Typical Day of Research (and why I hate Depth First Search )

awais — Fri, 29 Jan 2010 03:11:23 +0000

old fog

awais — Tue, 19 Jan 2010 00:50:34 +0000

کھڑکی سے جھانکتی ہے کسے بار بار دُھند

Custom Resolution in Remote Desktop

awais — Tue, 05 Jan 2010 15:06:49 +0000

I have a 1920×1080 desktop at work but when I use remote desktop to connect to home, it automatically resizes to my compact 1024*768 desktop. Most programs don’t seem to have a problem but I was working on Weka KnowledgeFlow and one of my flows, originally designed on the higher resolution, never showed a horizontal scroll. It might just be a java thing. In short, I had to look for a method to remote using a higher resolution than that of the local machine. Luckily, you can specify a custom resolution for the RDC using a command line switch ( more here ). The command line below gave be enough space to fix the flow. I hope this helps someone out there.

mstsc /w:1280 /h:1024

The picture above is my office machine when i was trying a horizontal flip. It works when you have many consoles open but the bottom part gets for browsing/coding, it’s not that great.

What do you tweet about? : A shell script for getting most frequent words for twitter

awais — Fri, 18 Dec 2009 20:29:51 +0000

There are a lot of web apps around which report your twitter stats. But at times, it’s better to do things yourself. I haven’t done any fun coding for ages now so last night, I finally got around to making a small program to gather twitter word statistics. The fun part was to do everything using unix tools. Here’s a small script file which displays the 10 most used words in the tweets for any twitter id. I have only tested it under cygwin so this is probably the best place to say “USE AT YOUR OWN RISK”.

Here’s how it works.

downloads all status information in a directory
extracts the status message lines
does some regex magic and filters stop words like the, a, an etc. ( haven’t seen this done earlier anywhere but the join command comes in handy for processing stopwords)
displays the top 10 most frequent words (and emoticons)

Twitter assigns a limit to the number of messages that you can download (3200). Also, the twitter id timeline has to be public for this script to work. All you need to do is download the script file and stop word list, keep them in the same directory, run it with the twitter id in the command line and you’ll get the list of words with the frequency at the start of each line. For example,

$ ./tword.sh barackobama
161 watch
119 live
92 http://mybarackobamacom/livestream
81 health
63 reform
55 today
52 rally
48 #hc09
47 &
38 vote

The script takes time to complete so be patient. As you may have noticed, there are still html tags inside. You can remove them by piping in any html2text program. There’s a small perl script in the zipfile which does this processing. The output now brings in a new word “change”. You will, however, need to pipe this in the script after installing HTML::Entities though CPAN.

$ ./tword.sh barackobama
161 watch
119 live
92 http://mybarackobamacom/livestream
83 health
68 change
63 reform
55 today
55 rally
48 #hc09
39 vote

My list toppers as good, , time, day, twitter, read, hope, back, :p and make. I wonder if this makes me a happy person

certainty

awais — Tue, 24 Nov 2009 15:59:34 +0000