The Twitter Profile Directory

“It has been frustratingly difficult finding users on Twitter unless you know their precise handle — so any improvement will be welcomed!” Schaefer tells WebProNews. “I’ve tried this new directory and still find it buggy but presumably it will be a great help to discovering Twitter users.”

“I think there is also a built-in benefit here to search engines,” he adds. “Twitter has made a number of search-friendly moves in the last few months and this is an example of this strategy. Of course, many Twitter profile pages have been indexed by the search engines for some time now, so it’s too early to say if the new directory is helping them find any unindexed profiles but my hunch is that is going to provide much more visibility to Twitter profiles on search.”

In the book, he had some good things to say about Twellow, which is certainly a more user-friendly way to discover Twitter users based on category and/or location (disclosure: Twellow is owned by WPN parent company iEntry).

The Embeddable Timelines Widget

“I have a very strong opinion about that,” says Schaefer. “I think in most cases, it’s really ineffective to embed a Twitter stream on a website for two reasons. First, if you’re using Twitter correctly, you’re having a conversation right? Why would you post one side of a conversation on your website? What good is that? At that point it seems like a gimmick just to show you’re cool or something.”

“The second reason is the potential risk,” he adds. “I was on a friend’s site and he had his Twitter widget front and center on his company’s landing page. Well, his latest tweet was a joking comment about the foremost users of Twitter that simply said ‘PORNOGRAPHY!’ Is that really what he wanted on the front page of his website? The tweets are completely out of context.”

“There could be some interesting applications to allow people to see tweets from a conference or something but again, a conference of any size is going to attract spammers to a popular hashtag,” he notes. “You run the risk of posting tweets that are inappropriate or even damaging.”

For more of our conversation with Schaefer, see:

Why Hasn’t Twitter Launched A TweetDeck-like Interface?

Without Realtime Search, Google Risks Pushing News Seekers Away To Twitter

Since then, SEOmoz has been doing somde extensive data gathering, investigating the situation further. Kurtis Bohrnstedt, the company’s “Captain of Special Projects” gathered a total of 2,678 directories, and only found 200 of them to be banned, but an additional 340 to be penalized (as in not de-indexed, but not ranking for obvious terms where they would be the only result that makes sense).

Still, that’s only 540 directories out of 2,678. It would seem that there are a lot more directories in the clear, but Bohrnstedt thinks this is only Google sending a warning, and that there is likely more to come.

“That is not to say the ones left unharmed are safe from a future algorithmic update,” he writes. “In fact, I suspect this update was intended to serve as a warning; Google will be cracking down on directories. Why? In my own humble opinion, most of the classic, ‘built-for-SEO-and-links’ directories do not provide any benefit to users, falling under the category of non-content spam.”

I wonder if that includes directories that are apparently built for SEO and links and charge webmasters for the chance to get links, but offer some form of editorial oversight.

Interestingly, when this topic was being discussed a couple weeks ago, one webmaster said he had a paid directory he hadn’t touched in years, which was unexpectedly seeing an increase in PageRank.

Last month, Danny sullivan and Pamela Parker put out an interesting report about Google’s apparent paid inclusion program, which comes in the form of sponsored comparison ad results. Google had referred to these as “a third kind of thing” between organic results and ads. They appear for certain types of searches like hotels, flights and financial products.

A Google spokesperson told WebProNews, “We’re changing the design layout of our hotel, flight, credit card and bank account results, which help users complete actions such as booking flights quickly and easily. We’ve always disclosed that Google may be paid when a user completes such an action; we want to be clear and consistent in how we do that.”

Danny Sullivan at Marketing Land posted a long report today about Google’s move from considering paid inclusion evil to embracing it on its own properties. As you may know, Google discussed paid inclusion in its IPO Founders’ Letter in 2004, where it said, under the “Don’t Be Evil” section:

Google users trust our systems to help them with important decisions: medical, financial and many others. Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating. We also display advertising, which we work hard to make relevant, and we label it clearly. This is similar to a well-run newspaper, where the advertisements are clear and the articles are not influenced by the advertisers’ payments. We believe it is important for everyone to have access to the best information and research, not only to the information people pay for you to see.

Sullivan quotes Google’s Amit Singhal from a conversation he recently had with him: “Fundamentally, time and time again, we started noticing that a class of queries could not be answered based upon just crawled data…We realized that we will have to either license data or go out and establish relationships with data providers…To be super safe, where we have a deal between Google and another party, we didn’t want to call those fully organic results, because they are based on a deal…After much debate, we said “OK, let’s be extra cautious. Let’s call it ‘sponsored’ so that we tell our users that there’s a special relationship that Google has established with someone.”

Google’s apparent view of the paid inclusion concept is pretty interesting considering its own quality guidelines around paid links. It’s interesting that Google doesn’t seem to have much issue with directories, for example, who offer paid inclusion without using nofollow on links, yet this very practice essentially equates to businesses paying another site to get links, most likely with the intention of getting better search engine rankings.

Sullivan brings up some interesting points about the FTC’s disclosure policies. Apparently, while Google’s paid inclusion results are disclosed as sponsored in actual Google search results, no such disclosure is made on the vertical search products these results are drawn from (Google Hotel Finder, Flight Search and Advisor).

I have to wonder how many of these paid deals Google has, how many include businesses paying Google for listings, and how many include Google paying businesses for data access.

The thread begins with a post from user Sunnyujjawal, who says:

While checking some sites links I found 50% free submission directories are out of G now. Will Google count such links in negative SEO or unnatural linking?

Schwartz concurs that about 50% of the ones he searched for did not have listings.

He points to one example: global-web-directory.org. Indeed, I’m getting no results for that site:

I’m not sure about the 50% thing though. I’ve looked at a number of others, and haven’t come across many that were not showing listings (though I have no doubt that there are more out there). Either way, there are still a lot of these sites that are still in Google’s index. We do know, however, that quite a few of them recently received PageRank reductions with the recent update.

This discussion happens to come at a time when we’ve been analyzing Google’s quality guidelines, and its treatment of a certain directory, Best Of The Web, which sells reviews for potential listings, which appear with links that pass PageRank.

Other directories that follow a similar model, may be experiencing similar treatment from Google. In that same WebmasterWorld thread, user Rasputin writes:

I have a paid directory that I haven’t touched for about 3 years, only gets about 25 submissions ($10) a year – strange thing is, I just looked and not only is it well indexed but all the internal pages are now showing page rank – for a very long time they were all ‘greyed out’ after the google clamp-down on directories a couple of years ago.

No idea when it came back, certainly nothing I’ve changed and pretty unlikely it’s attracted natural links.

That’s pretty interesting.

User Netmeg adds:

I don’t think free or paid makes anywhere near as much of a difference as to whether or not the directory is actually curated for quality. Because if it isn’t, what other reason is there for it to exist other than to create links?

That’s a very relevant point, and that seems to be Google’s reasoning, based on this video from Matt Cutts from several years ago:

“Standard directory listings remain in our editors complete editorial control, and as such do not need the nofollow tag,” Best Of The Web President Greg Hartnett told WebProNews. “An editor looked at those listings (pay for review or not) and decided that they meet editorial guidelines and as such merit a listing. We vouch for that listing, so why would we nofollow it?”

If you go to global-web-directory.org,’s submission page, it would appear that they violate Google’s quality guidelines. There is a pricing structure as follows:

Express Reviews – $2
Regular Reviews – Free
Regular Reviews with reciprocal – Free

While they advertise a paid review process, it’s clearly much different than how Best Of The Web operates. The only payment is for speeding up the review process, from the looks of it. Otherwise it’s free, and they’ll even throw in a reciprocal link for free. That could be the part that Google has a problem with. If sites are really being “reviewed” for quality, perhaps that is one thing, but if your’e saying flat out that you’ll give a link back, that might fall under Google’s “link schemes” criteria, discussed in the quality guidelines.

It does list “links intended to manipulate PageRank” as the first example, and it does look like the site attempts to show the listings’ PageRank right with the listings.:

If you really look around the site, however, you’ll find many category pages without listings, just displaying ads. It’s not hard to see why Google wouldn’t want this site in its index.

Update: There’s an interesting post about this issue at Search News Central, from Terry Van Horne. Terry writes:

Directories that would be candidates for this kind of “draconian” action were as good as de-indexed ages ago. We sent out our super staffer Mike, with our vetted list of directories to see what he could find. From that (top end list) we found 65 no change, 2 domains parked and 1 de-indexed site; roughly 1.3% were de-indexed.

Next we went to our friends at Steam Driven Media for the last 100 (based on TBPR) from a list of 1500. From this group we found 1 with low indexation and 9 deindexed/gone – roughly 10% affected. Keep in mind, we have no idea how long these sites were out of the Google index.

Van Horne questions whether directories are really “getting nuked or not”.

So far, we’ve not really seen anything indicating it’s as big a change as made out to be by the original poster in the WebmasterWorld thread.

Have you seen paid directories rising in Google? Free ones disappearing? Let us know what you’re seeing.

The new website, Yellowpagesoptout.com, allows people to select which phone directories they receive, or stop deliveries entirely.

"Our industry is taking a giant leap forward today by launching a clearinghouse site for consumers to control the delivery of directories," said Neg Norton, president, YPA.

"The site, supported by directory publishers across the country, illustrates our ongoing commitment to not delivering a directory to someone who doesn’t want one."

Features of Yellowpagesoptout include:

*Images of directory front covers and information about deliveries, so people can decide which directories to include in an opt-out request.

*An email confirmation about the users’ selections.
 

A few samples of comments we received about this:

"I have actually personally heard from someone who has bribed the editors multiple times to get listed with great-quick results."

1) It does not accurately include even a representative subset of the wealth of quality information on the Web
2) Allegations of corruption
3) Lethargy and languorous posting policies and procedures then DMOZ would disappear overnight."

Former Editor’s Take on the Corruption

M.J. Taylor, who is a moderator in our WebProWorld forum says she used to be a DMOZ editor. She addressed such corruption in a thread, and being how this is such a popular topic of discussion for our readers, I thought her two cents would be worth sharing here as well. She writes:

I can tell you why some websites got nixed in my categories. Sites that were very low quality, perhaps still under construction, for example, didn’t get listed. Sites that were second sites for the same business didn’t get listed. Affiliate sites were deleted.

Editors were very closely watched. I really find all the tales of corruption to be far fetched, as there was a great deal of supervision by very suspicious superior editors. I’m not saying there wasn’t corruption; I’m saying it got caught quickly.

Taylor says she was an editor for a few years and understands a little of the inside workings. For this reason, she says, maybe she finds it "easier to relax" because she knows "it isn’t personal."

What do you think? Comment here, or join the conversation at WebProWorld.

Dmoz has been brought back into the discussion as Google’s Matt Cutts appeared in a new Google Webmaster Help Video answering the following user question:

Why is Google still taking notice of DMOZ? Many have alleged that the editors are corrupt. It’s impossible to get them to list a site even if it is very relevant to a specific area.

"I know that people do have complaints about Dmoz, and we don’t show it in our one-Google-sort of tabs at the top of the page like we used to in previous years, but in some countries, it can be very hard to type in queries. It can take a lot of time," says Cutts. "For example in something like Chinese or Japanese or Korean, sometimes it might be easier to browse by clicking, rather than typing in the query, and so especially in those sorts of countries, it can be very helpful to show Dmoz."

"But we don’t use Dmoz in a lot of the ways that we used to. We don’t show the Dmoz categories or the Open Directory categories beneath the snippet, and we used to do that," he adds. "We don’t show it on the main page like we used to anymore. So if you’re frustrated, you can always try a different category that you also think is relevant. You can always go to editors up the chain. But in general, if you can’t get into Dmoz, I wouldn’t necessarily worry about it. There are a lot of other great places to get links across the web."

Dmoz continues down the slope it’s been on for quite some time in terms of unique visitors. Google not giving it as much play certainly must play at least some role in this. It does get over 18% of its referrals from Google:

Dmoz on its Own Future

Dmoz swears it still has plenty of life left in it, so if you believe the editorial department, there may be new opportunities from Dmoz down the road. In a post earlier this year, reflecting upon the last decade, Bob Keating, Dmoz editor-in-chief said, "Over the ’00 decade, DMOZ has grown to be one of the most successful collaborative projects on the web. It has outlasted its commercial counterparts, and continues to be relevant in the search industry. The keys to its longevity and usefulness are its dedicated community, its open, collaborative editorial model, its non-commercial nature, and open data distribution channel."

"While DMOZ receives hundreds of editor applications, and lists thousands of websites each week, it needs a new Plan – a new blueprint for the future of how the web is organized, and how human organized data is consumed," he says. "Using traditional web directories as a means for information discovery is a thing of the past. However, the need for organized web-based content continues to grow exponentially. The future of DMOZ does not lie merely in improving its toolset, making it more SEO friendly, or convincing others of its collective brilliance. Its future lies in turning the entire thing on its head."

Keating went on to list some goals for this decade, including the development of an API for Dmoz data to allow editors and developers to write new apps using it. He also wants to transform Dmoz from a fixed-path directory to "the largest faceted system for organizing information on the web," have it become a "major influencer" for bringing the semantic web out of the lab/enterprise and into the entire web, and transforming Dmoz into a "suite of products with multiple levels of participation and engagement."

Things have been pretty quiet on the Dmoz front since then. The only updates on the Dmoz blog have been from editors talking about their experiences editing specific categories. Perhaps that is because some of the aforementioned goals are in the process of being realized behind the scenes.

Note: With a great deal of talk in the comments about corrruption, you may be interested in hearing from a former editor on the topic. Read here.

Do you think Dmoz has a place in the future of the web? In the future of search? What kinds of apps would you like to see built upon a Dmoz API? Share your thoughts in the comments.

The reason Mozilla says it decided to start this directory is simply because there isn’t one like it. "There’s no central index of these tools," the company says in a blog post.

"As we’ve explored different tools we could create here as part of the Developer Tools Lab, we’ve come to the opinion that in addition to creating new tools, one of the best things we could do is help developers understand the broad universe of tools that already exist and expose some of the fantastic and amazing work that’s being done," says Mozilla’s Ben Galbraith of the Developer Tools Team.

With the Open Web Tools Directory, you can search through the following categories:

- Design
- Code
- Debug
- Test
- Deploy
- Docs

To view the directory, a modern browser with canvas is required. Galbraith suggests Firefox 3.5, Safari 4, Chrome 2, or Opera 9.

Of course to truly build a comprehensive directory of open web tools, Mozilla needs help from developers, and is calling for submissions. Submissions will be reviewed on a regular basis. They will not go live immediately.

The directory is clearly a work in progress, not just in terms of its contents, but also interface. Mozilla says it is working to add social elements that will allow users to comment and vote on the tools they like best. They are also working on a more accessible version that does not require a browser with canvas support.

The first question is:

Will Google consider Yahoo! Directory and BOTW (Best of the Web) as sources of paid links? If no, why is this different from another site that sells links?

When Google looks at whether or not a directory is useful to users, Google looks at: 

- What is the value-add?

- Do they go out and find entries on their own or do they only wait for people to come to them?

- How much do they charge?

- What is the editorial service that’s being charged?

"If a directory takes $50 and every single person who ever applies in the directory automatically gets in for that $50, there’s not as much editorial oversight as something like the Yahoo! Directory, where people do get rejected,"  says Cutts. "So if there is no editorial value-add there, then that is much closer to paid links."

The second question is:

We sell a software product, and there are 100s of software download directories on the web of varying quality. Could submitting our product to all of them hurt our rankings or domain trust/authority?

Answering this question, Cutts makes it clear that they are only talking about a software product ( .exe file), and not a website. "If it’s only a software product, then I wouldn’t really worry about it," he says. It wouldn’t hurt your website to have a link from those directories, he says.

If the directories are low quality, Google tries not to score them highly, but it doesn’t hurt to have your software listed in them.

There is a new feature on search results pages that shows any categories that might match your search terms. This is designed to make it easier to find profiles you are interested in, as well as let you see if there is a category that matches your interests without needing to scan the entire list of categories on the "All Categories" page. This will make Twellow easier to use and understand for a lot of people.

"This is something internally we’ve been wanting for a while, and Twellow users have also been asking for it. It really helps people see the bigger picture of Twellow when searching by pointing out other possible avenues for finding people," says Twellow Lead Developer Matthew Daines.

Twellow now has over a thousand different categories available for browsing and searching. What this means for Twitterers is that they have a great resource for finding Twitterers related to topics they are interested in. This makes Twitter more useful in my opinion, when you look at it from the "microblogging" perspective – meaning you’re reading tweets for information as you would blogs.

The Twellow team is always adding new categories. A good way to keep up with what has been added is to follow Twellow on Twitter, where the new categories are regularly mentioned. If there is a category you would like to see in Twellow, but it not there, you can simply contact the team, and there is a good chance they will add it. Twellow makes Twitter more useful, and I promise, I’m not just saying that because it’s a WebProNews service.