I was trying to trace the source of the quote “Any sufficiently advanced financial instrument is indistinguishable from fraud.”. If you do a quoted google search on a custom date range, an interesting problem can be seen.

The results contain pages originally published in 2005 but re-indexed recently. While re-indexing, the current tweets of the author were visible to the crawler and got indexed along with the original article. This makes it seem like the quoted text was mentioned first in 2005 where as originally it’s only a recent meme.

One way to avoid this might focus on identifying dynamic widgets like twitter/news/weather feeds and eliminating them from the index.  The HTTP Header (pasted below) lists the last-updated date which probably means that google is either getting the date from the first time it indexed the post or from the URL itself. Whatever the case is, it’s an interesting problem to distinguish between the ‘original’ content and other dynamically added elements on a page.

HTTP/1.1 200 OK
Server: nginx Date: Tue, 23 Nov 2010 17:50:20 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Vary: Cookie, Accept-Encoding
X-hacker: If you're reading this, you should visit automattic.com/jobs and apply to join the fun, mention this header. X-Pingback:

P.S. An interesting way to advertise. You have read this header anyway so you might want to apply for the job :)

P.P.S. On second thought, it’s not much of a ‘challenge’ per se. It’s just an interesting problem.

Is google (finally) stemming Urdu? The last time I checked, there were doing something like a transliteration based search but in the screenshot below, you can see that searching for the phrase ان پڑھ چٹا shows some stemming is being used. Does anyone know anything?  Oh, and while I’m on this topic, I would also like to know why is it called چٹا ان پڑھ ?

A Question Answering (QA) system is an Information Retrieval system which gives the answer to a question posed in natural language. For example, if you ask it Who wrote Hamlet?, it should answer Shakespeare. A few years ago (don’t ask me how many), search engines did not focus on language queries. Recently [sic], Google has started incorporating some NLP (Natural Language Processing) in their results. You can try it out by typing the same question in the search box yourself ( or clicking here ).

During my M.Phil. course, one of the tasks was to build a basic QA system and extend it however we liked. We used the TREC 8 dataset for evaluations. While building the system, I evaluated how current search engines (read Google) performed on this task. For this, I just queried the exact question and used the summaries of the top five results as answers. Evaluating at that time (2008), I got a Mean Reciprocal Rank (MRR) score of 0.212 over 198 questions. 156 questions had no answers found in top 5 responses.

This term, I am demonstrating for the same task. Demonstrators are usually PhD students who provide help and guidance to junior students. For pure geek fun and lack of better things to do while taking a break, I decided to quickly jolt down a JavaScript (read  JQuery ) based QA system. This time,  the resulting MRR score over 198 questions was 0.384 while only 79 questions had no answers found in top 5 responses.

The results show clearly that during the last two years, Google has significantly improved on answering NLP queries. In fact (IIRC), my baseline system back in 2008 (based on RMRS based matching of sentences from the top 100 documents returned by an IR system) could only achieve an MRR score of approximately 0.290, showing that the current results are much better than that baseline. I hope this decade sees some more developments/improvements in QA systems and I can ask a system What do you get if you multiply six by nine?

I’ve always said there was something fundamentally wrong with the universe. ~Arthur Dent