broadstuff

Blimey - a piece in the Grauniad about how its the small guys who suffer from ISP traffic metering, not Google, and now this - Google PR has been busy . From the BBC:

Forcing Google to delete user data after six months could dent its ability to predict pandemics such as swine flu, said the search giant's co-founder.

Larry Page said he thought more debate was needed around the issue of storing user data.

The European Commission wants data ditched after six months but Mr Page said there were benefits to users.

"More dialogue is needed [with regulators]," he told UK journalists at a Google event in Hertfordshire.

I'll bet he did. Google has been fiercely resisting attempts to get it to curtail it's user data storage to 6 months, this is yet another sally in its PR campaign. But trying to use Scary!Death!Danger! tactics of a very rare thing to justify the massive minute by minute intrusion on millions of people's data is, at the very least, diverting. Also, its not clear how having 18 months user data on tap would help predict swine flu over the 4 weeks it happened. Google could just as easily just log the historical metadata (eg "flu" searches by say location and time, and delete the user data)

I was a bit disappointed by the BBC piece as it didn't really look into the opposing arguments, just repeated Googlespeak. Lets trust the EU doesn't also get bamboozled by this sort of feint.

Cory Doctorow, who I have a lot of respect for in so many ways, has written an article in the Grauniad arguing that ISPs, not heavy bandwidth users, need to be curtailed. I had to comment on it, as I think it encapsulates a lot of the muddy thinking around Net Neutrality. While there is a lot of good stuff around Net Neutrality (all bits are free to travel), the argument around paying for it is muddied by vested interests (all bits must travel free), and I will try and unpick it here. Firstly, Cory notes:

....politicians around the world seem willing to sacrifice their national interest to keep a few powerful phone and telcoms companies happy.

Take the Telcoms Package now before the EU: among other things, the package paves the way for ISPs and Quangos to block or slow access to websites and services on an arbitrary basis. At the same time, ISPs are instituting and enforcing strict bandwidth limits on their customers, citing shocking statistics about the bandwidth hogs who consume vastly more resources than the average punter.

He argues that if they are left to be able to throttle and filter bandwidth, it will limit the ability of the "Next Googles" to grow:

But the real action in network fiddling isn't the battle between giants such as Yahoo and Google. Both well-established, have armies of otherwise unoccupied "business development" people lying around, and are handily capable of fanning out across the globe and buying lunch for their opposite numbers at every telcoms operator on the planet. The real victims of network discrimination are the nimble little startups, the firms that are in the same position today that Google was in 10 years ago when it consisted of a few marginally funded hackers and some taped-together hardware under a desk.

Except that an ISP has little need to filter or block a small startup, as they are unlikely to be the ones who are the "bandwidth hogs" using up all the ISP's capacity. No, the "bandwidth hogs" are the big companies throwing vast amounts of Video down the pipes - like Google. If anything, ISPs filtering YouTube will give new players more, not less, chance of success as that way they will be attacking the "free ride" the large players are currently enjoying at the expense of the small ones (and its these big guys who are most opposed to network management - and spending PR money like water - as they are currently enjoying the bandwidth commons the most).

I'm afraid this sort of error occurs because a lot of the Net Neutrality crowd are great writers and lawyers, but less capable mathematicians - when Cory talks about the maths of bandwidth hogs for example:

The reality is that network usage follows a standard statistical distribution, the "Pareto Distribution," a power-law curve in which the most active users are exponentially more active than the next-most-active group, who are exponentially more active than the next group, and so on. This means that even if you kick off the 2% at the far right-hand side of the curve, the new top 2% will continue to be exponentially more active than the remainder. Think of it this way: there will always be a group of users in the "top 2%" of bandwidth consumption. If you kick those users off, the next-most-active group will then be at the top. You can't have a population that doesn't have a ninety-eighth percentile.

While true about always having a 98 percentile in any distribution, this is not the point. The point is that by culling the top 2% of users, the ISPs have reduced the stress on the network as, in a Pareto system, they drive a massively disproportionate amount of traffic. Another term for Pareto's Law is the 80/20 law, ie 20% of the users suck up 80% of the bandwidth. Cull the heaviset users of that 20%, and you cut a huge amount of traffic while nearly all the rest of the users see massively better service. In other words, culling the network hogs will help, not hinder, the new startups.

This is because, being a power law, of the 20% of bandwidth hogs, only 20% of those will use 80% of the 80% etc - you get the picture. (For what its worth, in a pure 80/20 power law, the top 2% of users will use over 50% of the bandwidth. Thats right, 50%. So culling them reduces the stress on the network by half. Yes, there is now a "new" top 2%, but the usage of the network has halved. Real network traffic laws are not as steep as this. but it illustrates the issue well)

And the reason ISPs are doing this is because as their networks start to hit capacilty limits, the cost of upgrade is huge and its not clear who will pay them to do it.

The real thorny issue is how best to extract revenue from those who are the heaviest users. Cory is right to point to the problems here:

Metering usage discourages experimentation. If you don't know whether your next click will cost you 10p or £2, you will become very conservative about your clicks. Just look at the old AOL, which charged by the minute for access, and saw that very few punters were willing to poke around the many offerings its partners had assembled on its platform. Rather, these people logged in for as short a period as possible and logged off when they were done, always hearing the clock ticking away in the background as they worked.

But untrammelled, un-metered usage leads to bandwidth hogging.....so an ISP is damned if you meter, damned if you don't. But if you meter you at least get money from the really heavy users, which also nudges them towards managing themselves.

And if you look at any other utility in history - gas, electricity, water, telephony - they all wind up metered to some extent (in fact fixed price + meter seems the endgame for nearly all types of utility) in their evolution because of this need to extract revenue for the infrastructure maintenance from its heaviest users. In fact, what the upstream bandwidth hogs - the big guys like the BBC, Google et al - really fear is that the ISPs will get better at executing 2-sided business models and charge those loading stuff onto the networks more, not those taking stuff off. The most rational outcome is heavy uploaders pay metered plus discounts, small downstream consumers pay flat rate to a limit then add-on charges.

This all has nothing to do with Net Neutrality (the Free Net) but is more a scrap for economic advantage along the value chain by those currently getting a Free Ride

But as Cory points out, and this is where Net Neutrality does have a strong argument, you cannot discriminate on one bit vs another:

In the classic "traffic shaping" scenario, a company like Virgin Media strikes a deal with Yahoo to serve its videos on a preferential basis, and then slows its customers' connections to Google, Hulu, and other videohosting sites to ensure that Virgin's videos are the quickest to load.

So long as they have paid the entry price, they must get equal - neutral - access to the net.

This is the core of the valid Net Neutrality argument and we support it unreservedly.

(There is also a discussion in the piece about public subsidy of Telcos that should be returned to people - its a good idea to stop subsidy, but it still doesn't mean ISPs shouldn't charge the heaviest users more. In fact, if one stops subsidies there will be even more pressure on the heaviest users to pay their share. Beware of unintended consequences.....)

As an aside - I do get fairly tired of having to constantly be the one saying "No, thats not right" to the New Kids on the block, and potentially be seen as defending powers that be - but some of the muddy thinking around the new world will hurt in the medium term if it isn't challenged now.

It would appear that some our Silicon Valley Tech darlings are still struggling with this New Meedja thing, and a few oaths are being sworn by people who should know better....

I've always been amused by Chris Anderson, Wired's editor, writing tomes on Free economics while continuing to flog his magazine, books and speaking time for cold hard cash. Do what I say, not what I do, is the mantra and Guy Kawasaki hoist him on this petard nicely at SXSW this year (see above YouTube video).

But now it would seem Wired magazine is being swept away by the theses he espouses.

...Mr. Anderson has yet to solve the equation for Wired. Under his editorship, the magazine is an editorial success, winning three National Magazine Awards last month, which tied it for the most honored magazine. And Mr. Anderson’s own profile is higher than ever, thanks to his books, which roll messy business trends into neat canapés that executives pass around. He gives 50 speeches a year for an estimated $35,000 to $50,000 apiece.

But that has not equaled success for Wired in the downturn. The magazine has lost 50 percent of its ad pages so far this year, ranking among the worst off of the more than 150 monthly magazines measured by Media Industry Newsletter. Only Portfolio, which Condé Nast shut down last month, and Power and Motoryacht fared worse.

That leaves Mr. Anderson, who makes his living promoting big ideas, trying to come up with one big enough to reinvigorate Wired’s business.

As the NYT article quoted above notes, Wired is being hoisted by its editor's petards.

In 2006, Mr. Anderson published “The Long Tail: Why the Future of Business Is Selling Less of More,” arguing that the Internet allows for the sale of an array of niche products rather than relying on blockbusters. In “Free: The Future of a Radical Price,” which comes out in July, Mr. Anderson proposes that businesses can profit from giving some products away rather than charging for them.

From a business perspective, Wired is being hurt by both those phenomena.

Similarly, Peter Thiel (founder of Paypal, funder of Facebook) has rounded on Valleywag as being the "Al Qaeda" of the Valley. Not that they have flown any planes into Sand Hill Road you understand, more that they have flung outrageous stones and arrows at the investments of the VC's

Thiel is hardly alone in trying to turn tech blogs into a rah-rah chorus. There is now an entire media ecosystem dedicated to disseminating CEO and investor spin. Thanks to Thiel, anyone who questions the publicist-approved message can now be labeled a terrorist. Whatever: Valleywag will continue to be a place that prints the truths that others are too polite to say out loud.

We at Broadstuff Towers have been told more than once that if we just turned ourself into a PR Repro-machine we could make more (OK, some) money out of the blog. But we do this for Love, of course, so you the reader can enjoy our intelligent scepticism and sharp wit

Postscript - the point of this post was not really to take a few shots at individuals, but more to expose two trends I see as quite strong in the Silicon Valley Ecosystem:

- The fervent espousal of things that others should do, while not doing them oneself. At its most cynical its the process of getting others to do work as part of a collective spirit while the espouser runs away with the loot. Open Source, UGC and various tribes and belief systems all have some of this going on.

- The PR machine ecosystem, as Gawker outlines - the force of spin from the Valley is incredible, you see it if you are in Tech but not of the Valley.

Forewarned, with examples, is hopefully forearmed.

In a parallel with the MP's expenses debate, it could seem that some mainstream journalists from high status papers also forget why they exist - as Salon notes of the NYT's Maureen Dowd:

Maureen Dowd's wholesale, uncredited copying of a paragraph written by Josh Marshall (an act Dowd has now admitted) -- for what I yesterday called her "uncharacteristically cogent and substantive column"-- highlights a point I've been meaning to make for awhile. One of the favorite accusations that many journalists spout, especially now that they're searching for reasons why newspapers and print magazines are dying, is that bloggers and other online writers are "parasites" on their work -- that their organizations bear the cost of producing content and others (bloggers and companies such as Google) then unfairly exploit it for free.

Cough!

Here's the truth as far as this blog is concerned.

We get people copying stuff off here all the time, its not hard to spot if you are web savvy. The difference between bloggers and the mainstream media is the bloggers nearly always link back, or at least credit us. The mainstream media so far, (with one exception - thanks El Reg) do not.

These people then claim they are proper journalists. But "Journalism" here seems to be defined here as "copying other people's work without attribution", and these people are paid professionals. In other worlds this is called theft or plagiarism, and thus this practice is professional thievery. (See Tom Lehrer song on YouTube above for discussion on how it is done)

As with our "honourable" MP's, the stones flung at blogging from these "professionals" would be far more credible if their own house was in order.

(There is an emerging trend for very big blogs to no longer link, so clearly its not an "Olde media" thing but a "Big Media power corrupt" thing - in fact, its the Pro Journos who have crossed to blogging who seem to have the dodgiest morals)

Every few years a new fad sweeps EnterpriseWorld as the New Panacea. We've had Re-Engineering, Total Quality, CRM, Balanced Scorecards etc etc in recent memory. The formula is usually the same - CXO hears about New New Snake Oil at conference after having all defences bashed down by superb booze, sycophantic babes and seductive promises from Smooth Salesguys, and comes back to BigCo Towers determined to get a program moving and "move the dial".

I'm afraid that Social Media looks like its tipping up as the next one, and what can be more Socmed today than Twitter. Of course Twitter can save your business, and here is the proof as AdAge (a neutral party if ever there was one) labels Twitter as "Proves Its Worth as a Killer App for Local Businesses". Proves? Killer App?

There are Five Tips for local businesses looking to use Twitter

TRACK EVERY SALE. Sure, Twitter's relatively cheap, but you still want to know you're getting something for the human effort. Naked Pizza's point-of-sale system codes the origin of every order -- was it from a specific coupon or a box topper? -- which allowed it to calculate that a recent "Tweetie Pie" Twitter promotion, accounted for 15% of his shop's revenue on the day it ran.

TWITTER IS NOT FACEBOOK. Twitter is more immediate -- if a person doesn't check their Twitter feed for an hour she could easily miss the message. Berry Chill CEO Michael Farah uses Twitter for real-time communication ("Spotted: Justin Timberlake at Berry Chill!") and Facebook for longer-lead calls to action, like enlisting focus-group volunteers.

CREATE A CONVERSATION. Don't blast promotions incessantly. Intersperse them with other nuggets of wisdom or news related to your products and industry or neighborhood. Or, if you have a broader social mission, use Twitter to communicate that. Naked Pizza co-founder Jeff Leach suggests that if these kinds of social technologies become game changers, there may be a day when companies' initial business plans take into consideration whether they have anything worth microblogging.

SELL LAST-MINUTE INVENTORY. Twitter's immediacy is its biggest strength -- so use it to pump up business during lulls or discount last-minute unsold goods, said Zack Steven, co-founder of LocalTweeps, a local Twitter directory, who caught same-day discounted tickets at the Guthrie Theater in Minneapolis via Twitter.

ALERT FOLLOWERS WHEN YOU'RE ON THE GO. Venture capitalist Fred Wilson likes to point to KogiBBQ, a Korean taco truck that drives around Los Angeles, alerting its almost 20,000 followers to its current and future locations via a Twitter feed.

I can't wait for every Laundry and Dry Cleaner, Pizza Parlour and Hairdresser to take this to heart, I am sure the planet is full of people like just waiting to subscribe to them all. Or having "@bozo - you just mentioned Pizza - Did You know that Joe Shmoe Pizzeria is round the corner" in your twtstream the minute you open your digital gob.

This.is.called.spam. People by and large don't like it but advertisiers desperately want companies to do it and pay for the priveledge.

The truth, sadly, is far more prosaic - a few early adopters will get some local advantage going while the airwaves are relatively unspammed, and its seens as cutesy - but when everyone starts blasting out the @messages it will just p*ss people off. Which means bribing 'em with giveaways to listen. Which means a giveaway arms race. Which means higher cost of sale. Which means doom, not delight.

So, what is Twitter good for? Its an emerging aide to that most boring thing, running a business soundly with attention to detail:

LISTENING - its a Pub/Sub system, so sub to people you want to listen to

ANALYSIS - of what peopel are saying about you, your ecosystem etc

CUSTOMER SERVICE - helping people with problems, informaton etc

REWARDING LOYAL CUSTOMERS - these are the people you want to subscribe to you, your "Tribe". You want them to sing your praises elsewhere though, not bombard them with cr*p and turn them off

Its not for nothing that one of the fastest growing roles right now is that of "Community Manager".

Twitter - and Social Media overall - are just tools, means to an end. They are no more panaceas than ERP or CRM or TQM or WTF else pops up next. The way you run a successful small business has always been to sell more than you buy, to keep a tight handle on cashflow, try and be on a rising S curve. be friends with the local powers-that-be, and to be in the right place, preferably at the right time.

And don't drink Snake Oil, even if it is flavoured as Kool-Aid.

So, the team at Broadsight Towers has been playing with Wolfram Alpha this morning following its public release ( We were not considered worthy of reviewing it in Beta, after all what do guys who built real time semantic search engines know ). It did return the correct response to "the meaning of life" (42) as well as:

- London New York”
- Position of Hubble
- Weight of a proton

Update - Dave notes that more accurately, it showed where Google's limits are:

- The top Google search for “position of Hubble” gives a NASA readout of the current lat/long, no diff there
- For “London New York”, Google gives lots of airline links and ads. In fact, that’s my main gripe with Google – results are often swamped with advertising links.
- For “weight of a proton”, Google gives a direct link to the answer, but doesn’t do the multiplication.

We also tried to search for the distance from the Canaries to Tobago (Andy being a keen sailor) ... it tried to give lots of info about the birds then about a place in Africa and finally (probably when we got the Spanish spelling correct – Gran Canaria) it gave in and told us the answer. (Incidentally, Google’s 4th answer was a correct interpretation of the question – but the wrong distance as it gave the distance to Costa Rica.! It was a flying info site, so we guess there is no direct flight on the route wanted !)

Encouraged, but thinking that before it overcame Google, it first needs to overcome the Googly, we threw it at Cricket, the ultimate test of semantic corkscrewing:

- Explain LBW - "Wolfram|Alpha isn't sure what to do with your input."
- Laws of cricket - "Wolfram|Alpha isn't sure what to do with your input."
- Who won the cricket world cup? - "Wolfram|Alpha isn't sure what to do with your input."

Not so good.

Google, of course, got the correct answers (Or rather pointed to them) - here are "The Laws of Cricket" for example.

- You have two sides, one out in the field and one in.
- Each player that’s in the side that’s in, goes out, and when he’s out, he comes in and the next player goes in until he’s out.
- When they are all out the side that’s out comes in and the side that’s been in goes out and tries to get those coming in out.
- Sometimes you get players still in and not out.
- When both sides have been in and out including the not-outs, that’s the end of the game.

Simple, see

We think it’s trying to infer semantic connections from content in web pages. However, a true “semantic web” would explicitly mark up the semantic meanings during the authoring process, and this short-cut is a slip and will put it on a sticky wicket in future.

In fact, we suspect that Google will be bowled over by something coming from left field, but no doubt the whole new search thing will run and run.....

Update - our semantic search fundi Paul Lancefield has been giving it the Turing treatment over here

Update - the US has woken up, responses not too positive so far.... but then its hard to know who is a neutral there.

A few days ago I noted an interesting thing - Twitter used as a realtime chatstream for a broadcast TV show (See here about Question Time) in the UK. Essentially it worked very much like a conference Twitterstream (more like watching a conference on Web TV in fact) where everyone comments and labels the show with one particular name - eg #bbcqt in the above case (#xxxx is a "hashtag" in Twitterspeak).

In the "Broadcast TV" case, the show is broadcast over TV, the Twitter audience tunes in to watch TV, there is an early "battle of the hashtags" until one is agreed as the "right" one, and the comments start to fly.

For our non-European readers, the Eurovision song contest is a hardy annual and is the epitome of cheesy - naff people, naff songs, naff everything - I hardly ever watch it.

Tonight however was different, the Twitterstream (see #Eurovision here) started to come to life with very funny comments, and so i watched the show on TV for the first time in aaaages. The commenting and conversation made it worthwhile. As one person remarked, its:

...Like having ur mates round without having to provide refreshments.

Interesting trend....one would need to simplify the whole hashtagging process to make it simpler, but the user interaction is very good. In fact, one could:

...have Internet overlay on tv. I want relevant tweets from friends scrolling across the bottom of my progs!!

The songs were even half (I said half) bearable with all the comments flying around, and the winner (Norway, watch video above) was cheesy and naff, as tradition demands.

I've started a poll on Twitter to take the basic data on how we use it, twt me (@freecloud) with the answers to:

During #Eurovision Were you

(i) using laptop/mobile,
(ii)watching TV/WebTV,
(iii) in living room/bed/other ?

I'll collate and post them up here and on Twitter - answers could be quite interesting.

Nice post from Meg Pickard with the above diagram showing the demise of Web 2.0 darlings. Meg:

...having recently been confronted with this image in a presentation (used as being indicative of current reality), I thought it was time that it was updated.

I present these updates without reference to or predicting the demise of web 2.0 or social technologies or anything like that. Just to be a bit more accurate.

Now, at the risk of giving away my grizzled age, I pulled out my "!00 best dot.com stocks to own" from 2000. Among the 100, without any checking I reckon about 50 (at least) are dead. Here are some bright stars you may recall:

CMGI
Excite
Exodus
PSINet
VerticalNet
eToys

Lots of other lesser known casualties, plus some dot.com Zombies that are still staggering around today but have destroyed huge value, like AOL etc.

Meg has used X for Deadpool, I wonder what would happen if she also put in "Z" s for the Zombies

Update - Oo, fight club - Meg Twt'ed:

Being misunderstood by a rather defensive Mashable. I wasn't passing comment on web 2.0. I was updating an outdated image.

She's not wrong - see here

In a blog post about the image, entitled “Game Web 2.Over?”, Pickard’s stance appears at times skeptical of the Web 2.0 “koolaid” and at others somewhat neutral. She does, however, avoid taking the stand that this image may inspire: that Web 2.0 was a big overhyped bubble and you were smart if you just spent the last few years sitting on the sidelines, pouring skepticism upon these ventures and not striking out on your own.

If you were one of the idle blog commenters who declared “fail!” on every company covered by Mashable during 2007 to 2008 (and there were many of you)…congratulations, you were mostly right! And by being critical rather than trying something for yourself, you successfully avoided failure. Well done you!

However, note that those green circles represent companies that were acquired…many for millions, tens of millions or (in some cases) hundreds of millions of dollars. In fact, I propose we take this image and turn it into a motivational poster for every web company taking a chance in the social media era

Well, the sum of the market caps of my "100 best DotCom Stocks to Own" book was measured in the billions in 2000. If I took out Amazon, eBay and a few others, the total today would be less than the price of the book. There are always a few big winners, and most are big losers, its a Darwinian Power Law world.

You may also be interested to know that Google was not covered.....

From Search Engine land

“We’re doing a better job of choosing the blog posts to include in clusters,” Hylton says. “We’re also working on changes to expand the number of posts we consider for clustering.”

One of the algorithmic changes is aimed at making sure the home page clusters reward the freshest and most authoritative blog content. Within any cluster, Google wants to find the posts that people are talking about the most.

“We have a lot of ranking signals. We’ve been tuning the ranking and clustering algorithms to make better use of those signals. One important changes is that we’re processing new links much faster, so the post that breaks a story and gets a lot of links is more likely to become the lead story.”

Just as well, its pretty dire - the much maligned Technorati still beats it hands down. To me its been evidence that they haven't to date "got" real time search - and the big risk they face is that with all the new real time search engines emerging, they really do have to up their game here.