Chaoticity » Web

Google as a Question Answering System

awais — Sat, 06 Feb 2010 04:34:58 +0000

A Question Answering (QA) system is an Information Retrieval system which gives the answer to a question posed in natural language. For example, if you ask it Who wrote Hamlet?, it should answer Shakespeare. A few years ago (don’t ask me how many), search engines did not focus on language queries. Recently [sic], Google has started incorporating some NLP (Natural Language Processing) in their results. You can try it out by typing the same question in the search box yourself ( or clicking here ).

During my M.Phil. course, one of the tasks was to build a basic QA system and extend it however we liked. We used the TREC 8 dataset for evaluations. While building the system, I evaluated how current search engines (read Google) performed on this task. For this, I just queried the exact question and used the summaries of the top five results as answers. Evaluating at that time (2008), I got a Mean Reciprocal Rank (MRR) score of 0.212 over 198 questions. 156 questions had no answers found in top 5 responses.

This term, I am demonstrating for the same task. Demonstrators are usually PhD students who provide help and guidance to junior students. For pure geek fun and lack of better things to do while taking a break, I decided to quickly jolt down a JavaScript (read JQuery ) based QA system. This time, the resulting MRR score over 198 questions was 0.384 while only 79 questions had no answers found in top 5 responses.

The results show clearly that during the last two years, Google has significantly improved on answering NLP queries. In fact (IIRC), my baseline system back in 2008 (based on RMRS based matching of sentences from the top 100 documents returned by an IR system) could only achieve an MRR score of approximately 0.290, showing that the current results are much better than that baseline. I hope this decade sees some more developments/improvements in QA systems and I can ask a system What do you get if you multiply six by nine?

I’ve always said there was something fundamentally wrong with the universe. ~Arthur Dent

outwit

awais — Fri, 26 Jun 2009 20:26:02 +0000

I am playing around with a customized twitter client, temporarily named ‘outwit’. I’ll try to add up the features as I need them but its strictly an experiment for the time being. Let’s see if things go smoothly from here.

Creating Canonical / Normalized Links and URLs

awais — Mon, 16 Feb 2009 21:52:21 +0000

Here is a short intro on how to make sure that major search engines (Google, Yahoo, Microsoft) can be directed to see different URLs with the same content as a single ‘conanical’ URL. For example, the following links point to the same page but have different URLs

The solution is to select a single point as your representative URL and include this line in the HTML code.

Although a standard 301 redirect should work too, but this would be a bit easier for the non-techie designers and SEO enthusiasts to implement.

Am I asking the right question?

awais — Tue, 10 Feb 2009 18:35:22 +0000

Twitter asks ‘What are you doing?’
Google Latitude asks ‘Where are you?’
del.icio.us asks ‘What are you browsing?’

As in science, one of the most important things in web application development is thus asking the right question (at the right time). A properly phrased question can identify your niche along with giving you a guideline for keeping your focus on a single problem.
As some people might notice, I tend NOT to write much anyway. So until the writer in me is back (if there _is_ one at all), I’ll try to write something even if its a one liner.