It adds to the noticeable undercurrent of fear that, fueled largely by protests over sweeping Internet legislation, has been permeating. Perhaps fear is the wrong word. Maybe unease is better, but it’s there and it’s undeniable. Internet privacy is just one of the trending topics adding to it. With that in mind, the efforts of DuckDuckGo are admirable.

The search engine with the funny name — hey, you have to stand out — offers a non-tracking service, one that doesn’t send the keyword you searches as the referral term to the site being visited from DDG’s search results. If privacy in relation to search is important to you, DuckDuckGo is certainly not shy about offering you an alternative. Over at the nifty TLD’d donttrack.us site, they make their case with a basic-but-informative explanation of what Google does when you search and what they offer, complete with screenshots of the process in action.

The page is presented infographic-style, and it’s quite effective. An example:

And:

It’s a little bit of a scroll monster, but because the information is important, not to mention, presented in an effective, humorous manner, the scrolling is easy to overlook. It should be noted that privacy in relation to search is not the only service DuckDuckGo offers. Their goodies page is filled with a number of useful resources, like facts, conversions, calculations and dates. Bonus, you can use them all right there, on-page. Further navigation is not necessary.

There’s also some geek credibility going on, something their zero-click info sources lists of Wikia outlets suggests:

• Wookieepedia – Darth Plagueis
• Halopedia – covenant loyalists
• RuneScape Wiki – zamorak robes
• Memory Alpha – Balok’s cube
• eWrestling Wiki – tromboner man
• Memory Beta – RIS Brunia
• The Psychology Wiki – tachistoscopes
• Star Wars Fanon Wiki – gulper serpent
• Muppet Wiki – pink monster
• Annex – sembia
• Digimon Wiki – Alphamon
• Dragon Ball Wiki – Pui Pui nice shot
• Dofus Wiki – Bherb’s Gully
• Fallout Wiki – Diesoon
• Final Fantasy Wiki – gippal
• Indiana Jones wiki – Maggie Stanforth
• The Pokemon Encyclopedia – treecko

As the world of the Internet is undergoing some strain as tech companies, governments and copyright protectors battle for “control,” one of the issues embroiled in the unease is privacy. If Google’s search behavior is just too imposing for your comfort, DuckDuckGo offers a viable, useful alternative.

It should be noted, DDG’s “We Don’t Track” service, as is the alteration to the duck in the logo, is in support of Data Privacy Day, which starts today, but continues on “in the surrounding weeks.” The goal of this movement is simple:

On January 28 and in the surrounding weeks, the United States, Canada and many other countries celebrate Data Privacy Day. Privacy is our shared responsibility. Participate in Data Privacy Day by educating students or parents, training employees, hosting an event or sponsoring DPD.

The official sponsors of the event are Intel and eBay, which is quite telling.

In a document obtained by TorrentFreak, it has come to light that copyright holders held a meeting with search engines like Google, Bing and Yahoo to make demands in regards to copyright. The meeting was held by the UK Department for Media, Culture and Sport.

The document begins with a lengthy diatribe on why search engines are hurting legitimate business in the UK by linking to illegal sites first instead of their Web sites:

Consumers rely on search engines to find and access entertainment content and they play a vital role in the UK digital economy. At present, consumers searching for digital copies of copyright entertainment content are directed overwhelmingly to illegal sites and services. This causes consumer confusion and significantly impedes the development of licensed digital entertainment markets in the UK. Search engines, as trusted intermediaries, should assist consumers in finding legal services and should not contribute to copyright infringement.

This paper proposes the introduction of a voluntary Code of Practice for search engines, overseen by Government, which would help to ensure that consumers are directed to safe and legal sources for entertainment content online and grow the UK digital economy.

The paper then goes on to list its propositions:

- assign lower rankings to sites that repeatedly make available unlicensed content in breach of copyright

-prioritise Web sites that obtain certification as a licensed site under recognized scheme

-stop indexing Web sites that are subject to court orders while establishing suitable procedures to de-index substantially infringing sites

- continue to improve the operation of the “notice and takedown” system and ensure that
search engines do not encourage consumers toward illegal sites via suggested
searches

-ensure that they do not support illegal sites by advertising them or placing advertising
on them, or profit from infringement by selling key words associated with piracy or
selling mobile applications which facilitate infringement.

The paper goes on to detail the voluntary “Code of Practice” for search engines that would actively direct consumers to “legal entertainment content,” encourage Web sites towards improved online behavior, ensure that consumers reduce their exposure to malware or scams, ensure the existing system of removing illegal content from search results works to optimum effect, and help ensure that search engines unwittingly profit from illegal content.

The paper quotes a survey that says consumers are overwhelmingly in support of search engines directing them to legal sources of entertainment. The paper says that consumers favor legal sites over illegal sites because they don’t want to break the law unwittingly by downloading from illegal sources.

The paper then details individual actions for each of the propositions listed above. There’s a lot there and it’s a good read. The paper is listed as “private and confidential” but it’s too important not to see. There’s some good stuff here on how businesses interact with search engines. I suggest you take some time out of your day to read it.

Proposals to Search Engines

I’m going to look at results for the same query across each one just as a comparison. I’ll use the classic “level 4 brain cancer” query that we’ve looked at on Google various times throughout the content farm/Panda update discussion this year.

We first used this example to show where non-authoritative information was being surfaced for important health-related queries in Google, prior to the launch of the Panda update.

Ask Jeeves

Ask Jeeves was founded in 1996, and eventually became Ask.com, although it’s still Ask Jeeves in the UK. While there was always an emphasis on Q&A, this is very much the case these days. Here’s what it looks like:

The results:

Alta Vista

AltaVista was founded in 1995. Eventually it was purchased by Overture, which was taken over by Yahoo. Last year, Yahoo indicated that it would be shutting AltaVista down, but so far it is still live, delivering Yahoo results with an AltaVista logo. Here’s what it looks like:

The results:

AlltheWeb

AlltheWeb came out in 1999. It was eventually bought up by Overture, which was taken over by Yahoo. It was just earlier this year that Yahoo simply started directing it to search.yahoo.com.

Excite

Excite was founded in 1994. Ten years later it was acquired by Ask Jeeves. Now it’s owned by IAC, which also owns Ask. Here’s what it looks like today:

The results:

Lycos

Lycos was founded in 1994. In 2000, it merged with Terra Networks to become Terra Lycos. In 2004, Lycos was sold to Daum Communications. In 2010, it was sold to Ybrant Digital. Here’s what it looks like these days:

The results:

HotBot

HotBot was launched in 1996 by Wired Magazine, and is now owned by Lycos. Here’s what it looks like these days:

The results:

Infoseek

Infoseek was founded in 1994, and was eventually bought by The Walt Disney Company. It then was rolled into Go.com, but has been replaced by Yahoo’s search, which is interesting, considering that Yahoo’s search has been replaced by Bing. Here’s what it looks like:

The results (they simply redirect to Yahoo search results):

WebCrawler

WebCrawler is a metasearch engine. It launched in 1994, was bought by AOL in 1995, and sold to Excite in 1997. Infospace acquired it in 2001. Currently its results pull from Google, Yahoo and Bing. This was actually my go to search engine before discovering Google. That seems so long ago. Here’s what it looks like today:

The results:

DogPile

DogPile is similar to WebCrawler, and is also now owned by Infospace. It was launched in 1996. Here’s what it looks like today:

The results:

Mamma.com

Mamma.com, a metasearch engine, was launched in 1996. The company eventually purchased Copernic, and changed its name to Copernic Inc. In 2009, Copernic sold Mamma.com to Empresario.

There’s currently a message on the Mamma.com saying that a new version is on the way. Here’s what it currently looks like:

The Results:

The moral of the story: a lot of search engines think eHow has the best result for “level 4 brain cancer”. That includes Bing and Yahoo. Google is going with Harvard’s MGH Brain Tumor Center for its top result for the query. Blekko is going with Cancer.org. DuckDuckGo is going with medical-answers.org.

When was the last time you used any of these old school search engines? Let us know in the comments.

And at this year’s Web 2.0 summit, Hammer announced the imminent launch of his own search engine – WireDoo.

Set to officially launch sometime in December, WireDoo is currently in pre-beta. WireDoo.com simply prompts visitors to sign up for email alerts when the service becomes ready to use. But Hammer gave Web 2.0 a taste of what to expect.

Hammer explained that WireDoo is a “relationship-driven” deep search that he describes as “searches that go beyond the generalities, that give you more relative information in a consistent way, in one environment.”

For example, say you searched for a home. Along with the traditional results – the listings with real estate companies, the location – you will also see related content like information on surrounding schools, neighborhood activities, nearby restaurants, etc. Click on that related information and you will go even deeper. Say you clicked on nearby schools, you might see related information about GPA averages, teachers at the school, transportation and more.

He noted that “it’s about relationships, beyond the keywords.”

WireDoo is going to have to show up to the party with something incredibly special to take on the likes of Google, Bing and Yahoo – but who knows? Maybe Hammer is hiding something awesome in those parachute pants.

Here’s Hammer discussing his new search engine:

And here’s this…because I absolutely tried and failed to resist it:

Key findings are as follows:

• Google accounted for 65.09% of all U.S. searches conducted in the four weeks ending August 27, 2011.
• Bing-powered search comprised 28.99% of searches for the month
• Yahoo! Search and Bing receiving 15.89 percent and 13.10 percent, respectively.
• The remaining 64 search engines in the Hitwise Search Engine Analysis report accounted for 5.92 percent of U.S. searches.

That last point really drives home just how hard it is to compete in the search market in the U.S. 64 engines together make up less than 6% and the top three are really the top 2 (I assume if you’re reading this, you’re familiar with the Yahoo/Bing deal).

Google is down by a percent, Bing-powered search is up by 3%. Yahoo specifically is up 5% and Bing specifically is up 1%.

Another interesting finding is that longer search queries, classified as those averaging five to eight words or more, increased by 3% month-to-month.

I wonder if the increase in longer search queries is a testament to people getting better at searching or the search engines not delivering as well on the shorter ones.

It’s hard for me to even remember a time when I didn’t have to ability to instantly pull up any bit of trivia online. Sure, a couple of years ago it wasn’t that instant – I mean, if I was out I would have to wait until I got home to Google it. But now with the abilities of mobile devices to quickly and efficiently browse the web, any answer to almost any question is only moments away.

Information gathering has become so effortless that I often times deny myself the internet so that I can remember something on my own. Do any of you guys do this? There’s an actor or actress from some obscure film right on the tip of my tongue. I could find it in 15 seconds with my IMDB app, but I don’t. It’s a little game that I play with myself, one that comes with a grand sense of accomplishment when I succeed. It’s the technological version of saying to your friend, “wait, wait, don’t tell me.”

But if I can’t think of what’s on the tip of my tongue in a reasonable amount of time, then of course I head to the internet.

And that tendency is seriously altering the way our brains store information, says Betsy Sparrow. She conducted a series of experiments, mostly on college students, and came to the conclusion that we are using Google as a back-up memory bank.

Basically, we are using the internet in place of another person in a theory proposed decades ago called “transactive memory.” Hashed out, by Science‘s John Bohannon in the analysis of Sparrow’s experiment –

According to the theory, people divide the labor of remembering certain types of shared information. For example, a husband might rely on his wife to remember significant dates, while she relies on him to remember the names of distant friends and family—and this frees both from duplicating the memories in their own brains. Sparrow wondered if the Internet is filling this role for everyone, representing an enormous collective act of transactive memory.

So what she did to test this theory is play a little trick on students. She gave them 40 different trivia statements – banal things like the fact that an ostrich’s eye is bigger than its brain. She then had them type the trivia into a computer. She told half the students that the information they typed would be deleted and half that it would save it.

And you guessed it, the half that thought the data was going to be saved did much worse than the half that thought it was going to be erased. The latter group knew they didn’t have a backup, so their brains worked harder to store the memories.

Google is now that backup.

Another part of Sparrow’s experiment is even more interesting than that.

She again gave her subjects random facts to enter into the computers. This time, she told some that each trivia item would be stored in one of six specific folders labeled things like “Facts” or “Items.”

She found that the subjects could remember which folder each fact had been placed at a more impressive rate than the actual fact itself. Even when the students forgot the actual fact, when asked a generic question like “which folder is the ostrich fact in?” the students could point to the right one most of the time.

In short, we are now becoming better at knowing were we can find information than actually retaining that information. We might be teaching ourselves to simply be “good at Googling stuff.”

“I have no idea, but I know how to find out!”

This is some pretty interesting stuff. I wouldn’t say that this proves that Google is making us all dumb, but it might be making our brains lazy.

"Bing travel crunches over a billion airfares on a daily basis to bring you Price Predictors, and we use that data to uncover cheap airline tickets every day," explains Bing’s David Lindheimer on the Bing Search blog. "With Autosuggest Flight Prices natural language capabilities, you can simply type ‘Fly to Chicago’ or ‘Chicago Flights’ and Bing will immediately recognize where you are, in this case Seattle, and instantaneously display the Price Predictor based on your location."

"Don’t worry about entering the formal city name or airport name," adds Lindheimer. "Bing Travel recognizes colloquial names like Chi Town."

The feauture must still be rolling out, because it’s not working for me yet. 

Meanwhile, Google is reportedly negotiating with the Department of Justice to prevent having to go to court over its acquisition of travel software service ITA Software (hat tip: Greg Sterling). Bing is a member of the FairSearch Coalition, whose sole purpose is to see that this acquisition is blocked. 

Google has been criticized for favoring its own content over competitors. It’s interesting that Bing is going so far as to suggest its own content before users even get to the search results themselves. 

"This update is designed to reduce rankings for low-quality sites—sites which are low-value add for users, copy content from other websites or sites that are just not very useful. At the same time, it will provide better rankings for high-quality sites—sites with original content and information such as research, in-depth reports, thoughtful analysis and so on."

Google’s announcement is a followup to what Matt Cutts stated in his blog just over a month ago:

"And we’re evaluating multiple changes that should help drive spam levels even lower, including one change that primarily affects sites that copy others’ content and sites with low levels of original content. We’ll continue to explore ways to reduce spam, including new ways for users to give more explicit feedback about spammy and low-quality sites. 

As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content. In 2010, we launched two major algorithmic changes focused on low-quality sites. Nonetheless, we hear the feedback from the web loud and clear: people are asking for even stronger action on content farms and sites that consist primarily of spammy or low-quality content."

Google notes that this update does not utilize the new data they are receiving from the "Personal Data Chrome extension" that they launched just last week. However, they are "very pleased" to report that 84% of the top several dozen domains reported to be blocked by the extension are also impacted by the algorithm change. This could be very bad news for the high profile eHow.com because it is routinely referred to in the media as a site with lots of low quality content. If people are voting via Chrome it’s likely they are one of the top domains Google is referring to.

However, our own small batch of test searches at WebProNews does not show any negative changes for eHow. For instance, eHow still ranks number one in Google results for "level four brain cancer", the same as we earlier reported. WebProNews will be further reviewing the impact of this change on eHow and other mass content producers over the next few days. Additionally, Google states in their posting that, "We’re working on many more updates that we believe will substantially improve the quality of the pages in our results". 

The message to content producers is clear, publish shallow content that is obviously SEO’d and disliked by users and eventually you won’t be rewarded by Google’s search engine. 

Now, they’re making more adjustments to Facebook’s internal search feature, which make it more useful. They’ve started separating the results that appear automatically into different sections. This includes things like: People, Pages, Apps, Groups, Shared Links, etc. 

Google is in the process of rolling out a significant update to its social search feature, throwing social results into the mix (as opposed to separate at the bottom of the SERP). A glaring hole in Google’s offering, however, is a lack of Facebook data – arguably the most important social data there is, given that Facebook has so many more users than any other social network. It would be nice to see articles "liked" and shared by Facebook friends in Google’s results (luckily Wajam adds some Facebook data to your Google results if you install a browser extension). 

Last fall, Facebook started adding "liked" articles to the search results. "Consistent with how we treat other Open Graph object types, we’ve introduced the ability to see articles shared by your friends in the search typeahead," said Facebook’s Namita Gupta.  "For instance, if your friend clicks ‘Like’ on an article at a news site, the article will appear in your News Feed and can now also surface in the search typeahead."

Now, with "shared Links" they’re organized better, and often appear above other results.

Facebook has the strongest collection of person-to-person interactions on the entire web. You’d be hard pressed to find many authoritative sites that don’t have some kind of Facebook integration, even if it’s just "like" buttons. As long as Google doesn’t provides this information in search results, it’s never going to be as good as it could be. 

This is one reason why Facebook is a potentially dangerous opponent of Google’s. To reiterate points made in my previous article, it’s not that far-fetched to see Facebook put more emphasis on search the way Twitter did, and Facebook already uses Bing for web results, and could easily fill in the gaps with those results. Bing has no qualms about supplying search results to other major search engines through the back-end (see Yahoo deal). What if Facebook just became a real search engine competitor – instantly personalized, with some sections powered by Bing. They’ve already pushed for users to make Facebook their home page. 

Let’s also not forget that Facebook recently acquired search company Chai Labs (founded by a former Google exec), and has stolen many Google employees away.  Also remember that Facebook has also been encouraging the addition of different types of information to be added to its internal network with things like Facebook Questions and Wikipedia-like community pages. In time, Facebook Places (combined with Bing Mpas) could evolve to power some powerful local results too.

And you never know what future acquisitions Facebook might make, which could play their own roles. 

Should Google be nervous? Tell us what you think. 

We thought it would be fun to look at what Google’s results could be with a little work. Google does have good quality results for most queries I’ve tested. The problem is that often times they’re letting the less quality results get mixed in, or even in some case outrank the higher quality and more authoritative results. This becomes even more of an issue when you get into the kinds of queries where the ramifications are potentially more severe – things like health, legal, and financial – where bad advice can be costly. Not everything’s as simple as "how to tie a tie". 

Take a query like "how to prevent kidney stones". Look at how different Google’s results would be if it eliminated (or at least dropped the ranking of) content that has no discernible, authoritative credibility. I had to go all the way to the third page of results just for the top ten. 

Here is what Google’s results looked like for the query: 

Here is what they would look like eliminating the questionable results:

Sources for the top ten on Google as returned:

1. JustHealthTips
2. eHow
3. WebMD
4. UCSD Medical Center
5. National Kidney and Urologic Diseases Information Clearinghouse
6. Jackson Siegelbaum Gastroenterology
7. HealthAndAge.com (by The Swiss Association for Nutrition [SAN])
8. Wikipedia Entry on Kidney Stones with Section on Prevention, well sourced, citing various medical journals 9. University of Maryland Medical Center
10.FoodandLife (no apparent credibility reference or links to credible sources)

Sources for the top ten when you remove the questionable results:

1. WebMD
2. UCSD Medical Center
3. National Kidney and Urologic Diseases Information Clearinghouse
4. Jackson Siegelbaum Gastroenterology
5. HealthAndAge.com (by The Swiss Association for Nutrition (SAN))
6. Wikipedia Entry on Kidney Stones with Section on Prevention, well sourced, citing various medical journals 7. University of Maryland Medical Center
8. American Kidney Fund
9. National Kidney and Urologic Diseases Information Clearinghouse
10. CBS Moneywatch Report, Citing International Journal of Urology, Global Healing Center, University of Maryland Medical Center, A German study, Journal of Urology, an herbalist, and an herbal patient.

That last one is debatable, but the results appear a great deal more authoritative and accurate. And I’m guessing you can dig deeper and deeper and find other results that could potentially rival these ten as candidates for the best results for the query. 

It might even be useful if Google included news result blocks (universal search-style) in the first page of results for more queries, even when there isn’t breaking news. Using the same example for "how to prevent kidney stones", search Google News, and you’ll find results from last month – news stories about things like "a closer look at the treatment of kidney stones," "Pre-PNL Nitrofurantoin Benefits Some Kidney Stone Patients," and "What’s the difference between antacids?" (which references kidney stones in it).

Even if these articles aren’t breaking news anymore, they are still some of the latest articles to discuss the subject, and may prove useful to somebody searching for how to prevent kidney stones. They’re certainly fresher than many of the organic results returned. Giving at least a small sample of the latest news related to a query can’t be a bad thing. At the very least, the user has a chance to be more informed about the subject at hand. Yes, if they want news, they can search Google News directly, but in many cases (such as this one), it may not even occur to the user that there has been any news on the subject. 

Like DuckDuckGo founder Gabriel Weinberg told us in an interview, many articles like the one you’re reading provide examples, and they’re simply anecdotal. This is just one example, and there’s no question that Google does better on some queries than others, I could dig up more examples (and maybe I will in the future…I already have in the past), but they’d still just be isolated examples.

The big problem is that there are so many examples to be looked at. It’s no secret that search quality is an ongoing battle, and Google has been pretty open about the fact that it is an issue, even if they claim their results are the best they’ve ever been. They’re still talking about improving the algorithm with regard to content farms, and releasing tools designed to contribute to that.