Applying Social Science to the Web

    September 20, 2007

Imagine waking up one morning, and not seeing your family before you leave for work. You get to the office and all of your colleagues are invisible.

You know they are there, but you cannot see or talk to them. At lunch, all of the restaurants are empty as you eat alone. The only time you might see anyone after a lonely day is when you visit your local social network club to interact with complete strangers.

Sound like a dreary science fiction movie? Not exactly: hundreds of millions of people mimic this lifestyle on the web today, browsing and searching without seeing another soul. We acquire information and often make purchase decisions in a vacuum, without the right context or peer knowledge. Our successes and failures on the web sites we visit are gone the moment we leave. Except for some isolated social network hubs where a marginal few will chat and share actual (or fabricated) experiences, there is little interaction and no collective wisdom being shared, preserved and passed on from generation to generation.

This lonely digital planet occasionally gets a makeover by omnipotent, god-like figures otherwise known as web masters and online merchandisers, who reshuffle the planet/site without consulting the invisible residents/web users. The more intelligent deities attempt to understand users by asking them to “rate this” or fill out surveys and reviews, or they will try to intuit personalities and preferences based on recent activity. You bought a Barbie? You must love figurines and unicorns! There are major pitfalls, known by social scientists for years but unknown by marketing professionals, with these types of explicit feedback, with susceptibility to serious survey bias, misrepresentation and voting fraud being the biggies. Another elephant in the room is that even the lonely web user has a myriad profiles- son, brother, juggler, and father of an 8 year old daughter (hence the Barbie).  Try as they will, these alien deities can’t understand the wisdom of the silent and lonely web users by observing explicit behaviors alone, as what you did yesterday often has little reflection on what you will do today.

Livening up this lonely planet is easier than you might think. Steven Johnson’s brilliant book Emergence described an amazing scientific phenomenon – a colony of ants can together form a highly intelligent society, as ants are visible to each other via chemical trails. The academic term for this miracle power is called emergent behavior, and it is the social science foundation for the wisdom of human society.  Applying this to our lonely web planet, web users can become “visible” to each other if websites can observe behaviors and connect them with like-minded users by their invisible “chemical” trails.  Product or content recommendations driven by like-minded peers are the web equivalent to passing down collective wisdom. If you are a high-end biker, you would appreciate insight and recommendations from fellow high-end bikers. This guidance not only helps us make better decisions, whether it be items to purchase, articles to read or in the case of web self-service, finding solutions to our problems, it also connects us to web users with similar interests. 

It is important to note that when applying emergent behavior to the web, the chemical trails must accurately reflect user interest, context and value. Just tracking clicks or page views is misleading, as these tend to be a function of site design rather than an indicator of whether the user actually likes the content. Webmasters must consider the full spectrum of user behaviors and cross compare with those of like-minded peers to arrive at true wisdom.

In short, recommendation engines can be the chemical trails that connect us with our peers online.  They can guide web users to products, content, search, email and advertising through observing other people’s past success and failure in the same context. The collective wisdom is dynamic and adapts to seasonality, news, fashion, trends and context, yet it is not susceptible to fads and self-fulfilling prophecy.  All wisdom can be carried on to the next group of users. A “real world” generation takes 25 years; online, a new generation can trail the pioneers by a week, a day or even an hour. A website with highly connected and self-governed users can build and evolve its intelligence almost instantaneously. In this way, the digital planet is no longer lonely.