The Techmeme Leaderboard only shows the top 50 sources for a rolling 30-day period. Looking at it today, TechCrunch, The New York Times, and Engadget are the top three sources most frequently occurring on Techmeme.

Fortunately, Singh found the Leaderboard an interesting development, one that he could take farther with a little data mining. He compiled four lists to find some basic statistics:

I wrote up some code to crawl and analyze Techmeme articles over the whole year (Leaderboard shows the Top 50 sources for this month). I took a snapshot of Techmeme at 1:00PM every day between beginning January – end of September of 2007.

I computed basic statistics, like number of stories by author and source, as well as more involved measurements like the top word mentions of the year – in total and by category (used simple NLP to clean up the text and remove stopwords).

His list of most stories by author, unsurprisingly, featured the prolific Arrington as the first named person on the list (unattributed stories dominated what Singh found), followed by Om Malik, Greg Sterling, Andy Beal, and Donna Bogatin.

Stories by source showed TechCrunch just ahead of Engadget, followed by Digg, News.com, and SearchEngineLand. Google, Microsoft, and the iPhone were the most mentioned words; again, no surprise considering their dominant positions in search, operating systems, and gadgets, respectively.

“When a group of seemingly independent actors are making decisions based on the same limited pool of information, they become more highly correlated, and thus “stupider.”

As a couple of the commenters on Tim’s post note, this is similar to what happens in both economics and biology. When the supply of economic actors becomes limited, innovation ceases; when everyone is chasing the same few stocks or the same investing strategy, the utility of that strategy quickly declines towards zero. As Bob Warfield puts it at Smoothspan, the Web adds to this problem because it “eliminates friction and encourages herdlike behaviour.”

That will come as no surprise to anyone who has noticed the proliferation of identical Web 2.0 apps or Facebook widgets, or the explosion of identical me-too posts on Techmeme — and part of Tim’s point about Techmeme is that the “leaderboard” is only going to encourage that kind of behaviour. Ironically, of course, Tim’s post has appeared on the site, and here I am writing about it (wow, this is so “meta,” isn’t it?).

Am I writing about it because I want to move up the leaderboard? Not consciously — but that might be one of the outcomes regardless of my motivation (and I know, as I mentioned in a comment on Tim’s post, that he is secretly enraged at the fact that I am higher on the leaderboard than him, and his post is really a clever strategic move designed to improve his position).

But does that make Techmeme bad? Gabe argues in Tim’s comments that it doesn’t, and I would tend to agree. As he puts it:

“I think there is a self-reinforcing effect with Techmeme, but I believe it’s (1.) overstated and (2.) as often good as bad.”

Do pile-ons occur? Obviously, they do. And do many of those posts essentially consist of a re-posted excerpt and a “what he said” kind of comment? Definitely. Every system has a certain amount of noise. But I think on balance the posts that do add something to the conversation — and there are many of them on the average day — bring enough value to make it worthwhile.

Take Tim’s post itself (more meta): the sub-links included Bob Warfield’s post, which I thought had lots of value, as well as an excellent one from Alexander van Elsas, who I hadn’t come across before. And in many cases — as Alexander points out in his post — I find even further interesting blogs and points of view in the comments section of the blogs that appear as sub-links.

Those kinds of value are very difficult to quantify, but they do exist. It’s a chaotic system, in some ways, like biology or the stock market. But on balance, systems like Techmeme help to bring value to the surface, if you are prepared to look for it.

Update:

Tim has posted a follow-up to his original post, in which he argues that regardless of Techmeme’s utility, the larger point he was trying to make is that if everyone is looking in the same place then they will ultimately find the same kinds of things — and that he prefers to look where others aren’t, in order to find out what might be coming next.

That’s a fair point — and worth thinking about. And what does Tim think is coming next? Using trends in financial markets as a guide, he says that he thinks there might be a backlash of sorts to the openness of Web 2.0, and a corresponding focus on keeping some data private or proprietary. An interesting thought.

Comments

Tag:

Well, that may depend on your motivation for publishing in the first place.

The discussion begins at the Guardian, set in motion by Bobbie Johnson’s musing on the release of last week’s TechMeme Leaderboard, which shows the top 100 online publications appearing on the site. Johnson noted that TechMeme didn’t generate near the traffic of the aforementioned buzz-networks, among others.

He writes:

But for the people who jostle for position on the site’s top 100, what’s the use of being part of an aggregator that aggregates but doesn’t send readers your way?

And so begins the discussion as to whether TechMeme is worth its hype among the A-listers. Robert Scoble calls it "an echo chamber" that sends a fraction of the traffic that Digg sends, but that’s not necessarily a bad thing, if you lean Aristotelian in your world view:

I don’t want a big audience. I want a smart audience. So far I’ve gotten exactly that from TechMeme.

Christopher Coulter, who, according to a quick Google search, may be a frequent antagonist to Scoble’s ideas, had no trouble articulating how both sides of the traffic game can be spun:

When traffic low, claim "superior" audiences or "community", when traffic high, claim the trend has finally taken hold, you of course, being the first to know. It’s a great shell con-game — you never lose.

Well, that would be the cynic’s viewpoint. Whether or not bigger is better (and that’s a debate that can go on – crassly – all night), said echo-chamber extended beyond TechMeme as reports of similar trickle-down-traffic experiences came to light.

Nick Carr says:

Techmeme seems a lot bigger than it really is, at least to some of us.

And at TechWag:

We have been on techmeme twice now, and it has driven less than 1% of all traffic to the web site. We may be user generated content web 2.0 heaven, but we are still increasingly reliant on web 1.0 search engines, and the very few top social networking sites. The top referrers to the web site, Google, yahoo, msn, live, and direct, long before we get to Digg, Slashdot, or other sites that tend to send spiky traffic.

Louis Gray qualifies the traffic question at least a little by mentioning how the primary readership following TechMeme are RSS subscribers, thus naturally reducing the traffic.

So, at least among this small niche crew, there seems to be a general agreement that traffic is not the motivator for headlining at TechMeme, reducing it down the aforementioned intimate-gathering versus drunken-massive-orgy debate.

Then what is the value of TechMeme? Well, it could be a case of credibility by association. Dan Blank, Director of Content Strategy & Development for Reed Business Information, comments that TechMeme has a PageRank of 7, therefore passing along some nice authority with its links.

Dave Winer, always the contrarian, adds a little game theory to the mix, by noting TechCrunch’s overwhelming presence on the site:

Since Arrington’s pieces tend to rise to the top of the page, pieces that link to them become more visible (they show up in the Discussion links), and the chances that another blogger is going to point to them go up. All it takes is one or two of those pointers to promote your piece to the top level, and that really boosts your visibility, and now that the Leaderboard is there, it could make that status semi-permanent, creating an even greater incentive to point.

As to whether a bigger audience is better than a smaller audience, that all depends on if you’re publishing for money or for conversation – if it’s the conversation you want, it doesn’t really matter.

To find out where the gurus rank, Rand developed an SEO Quiz, complete with a leaderboard to satisfy those with a competitive nature.

According to Danny Sullivan, the quiz isn’t too difficult (of course it isn’t for you Danny , and the prospective quiz takers can check their answers while the test is in progress. The following example will give you an idea of the quiz’s difficulty level:

What is typically regarded as the “optimal” keyword density?

The quiz answers are in the preferred multiple-choice format. Good luck.

Chris Richardson is a search engine writer and editor for WebProNews. Visit WebProNews for the latest search news.