I Wolfram Alpha

I'm going to have to be quicker in the future to stand a chance of beating my colleague, Alan Patrick, to publishing an article on a new service with as much hype as Wolfram Alpha, see Alan's blog post here.

So their search-cum-calculation engine passed the first step of my simple Turing test opener,

...well sort of:

"Are you a machine"

To which I got the reply:

Input interpretation: "Are you a computer?"

Response: "Yes I am a collection of computers."

But hold on, I'm being unfair here. Even by mentioning the Turing test, I'm implying it's an appropriate one to use. WA has never been billed as an AI solution and isn't that the problem? No matter what super search and response engine WA engineer, the standard we will judge them against is that of a human expert. As we use the service for the first time we judge them against the ideal of a custom written research paper targeting our specific enquiry with a precise and incisive executive summary all prepared by the world's greatest authority on whatever question it is we asked. Clearly WA, Google and anyone else is going to fall short for some time yet.

The more I use the service, however (which is not much as yet), the more it appears as though the input "question" or "search terms" I provide need to conform to invisible syntactic rules internal to the service if I am to get good results.

Searching on "London New York" will get you flight times between the two cities but "Flight times London to New York" won't.

I'm left wondering what the use is for a service where the most useful form of the results you get are determined by fuzzy syntax and notation. Maybe there is value in being able to get the form of results you want 50% of the time without referring to the help files. But then how do you know if there is a better format of results more suited to your needs than what you have been given - unless you have referred to the help files? But if you have checked the help files for the best notation and rules, why wouldn't you use strict notation so you know you will get precisely what you want every time?

Actually I do think there is value in a hit and miss notation guesser. We all have time constraints and as long as WA works for a sufficient percentage of cases, it will - on balance - provide value. Plus of course, WA provide guidance for those who can be bothered to look it up. But I suspect the real value of WA is in the back-end indexing and automation of quality assurance. It will take a little longer to explore the consistency and quality of the quoted resources than the initial review I have made so far but if the WA algorithms automate qualification of data sources and the quality standards are up to strict academic reference standards, I'm sure they are on to a winner.