1. Opt-out for anonymous user behavior: It will force Web sites to give consumers obvious ways to opt out of advertising based on their browsing history and Web actions.
2. Opt-in for using PII data : Users would also have to give explicit permission before these companies could link the anonymous searching and surfing data from around the Web to information like their name, address or phone number.

My prediction about Behavioral targeting and privacy is coming true. Earlier this year, in my yearly predictions I said that this is year we will see a greater push for consumer’s privacy.
Here is what I said
"Behavioral Targeting will continue to grow this year; however, there will be greater push for protecting consumer privacy. The privacy concerns will result in:

1. Clear instructions (or links) on Behaviorally Targeted Ads that will allow behaviorally targeted visitors to opt-out of Behaviorally Targeted advertising
2. Opt-in system – Some networks (maybe new ones) will move towards opt-in rather than opt-out (I favor opt-in over opt-out as I wrote in past. So I am making this prediction that this year networks will pay attention to it). A new types of networks or services might come up which will allow users to be an active participant in BT and control who can use their online behavioral data and how they can use it."

Use of customer’s data without their consent created an uproar in UK last week. In response to the mess created by Phorm and British telecom, Sir Tim Berners-Lee said that his data and web history belonged to him.
He said "It’s mine – you can’t have it. If you want to use it for something, then you have to negotiate with me. I have to agree, I have to understand what I’m getting in return."

This is just the beginning; I think, we will see greater push for consumer privacy as consumers become educated about how their data is being used to target them. I think it is time for publishers and ad networks to be proactive about educating customers on how their data is being used and give them clear options to opt-out (or better opt-in) of any targeting.

I am big proponent of Behavioral Targeting and Personalization but it has to be with user’s consent.

Comment

The longest, most detailed and, frankly, most baffling post was by Anil Batra. In my original take on this issue, I disagreed sharply with Anil about several aspects of the comScore findings. First, I argued that knowing the "real" number of visitors to your site isn’t useless and that not all analytics is simply an exercise in trend following and comparison. In the real-world, the actual numbers often matter and matter greatly. But that being said, the main thrust of my argument was that Anil was misunderstanding the potential impact of the comScore findings by treating them as something relevant only to reporting on total site traffic. Admittedly, this is the arena comScore is concerned with. But for a web analyst, data that suggests severe problems with cookie persistence have an impact far beyond mere traffic. Lack of cookie persistence will deeply color almost any analysis that persists across session: campaign attribution, "new" visitor analysis, "repeat" visitor analysis, "customer" analysis, attrition, sales cycle analysis, etc., etc. etc.

Let’s start with Anil’s arguments concerning traffic:

"The point I was trying to make is that you have to take everything in context. Going to Gary’s example of a conference, let’s say conference A tell you they attract 5,000 visitors and the other conference B says they get 4,000 visitors. Next day a third party comes out and says that all the conferences numbers reported by any conference are inflated and actual number is 75% of what they state then what’s the net result? Well Conference A is still better than conference B. Only thing is that they each now have 3750 and 3000 visitors respectively. Every conference in the world will have the same issue, their rank is still the same. I don’t think based on this information conferences will start charging less for the booth. However the rate per visitor has gone up for you but you can’t do much, that’s the market rate. Same argument goes for sites that sell advertising based on how many users they reach."

Unfortunately, however, advertisers don’t work in a world where there only option is the web. If I tell an advertiser that my reach is 1 million visitors, that’s going to be compared to other sites with a reach of 1 million visitors but also to radio, print, TV and more. So unless Reach just flat out doesn’t matter to advertisers, I fail to see how massive and consistent mis-reporting of web site traffic isn’t an issue.

Nor is it at all the case that we should expect cookie persistence issues to be the same for every type of site. I already pointed out two significant reasons why traffic estimates would be different for different types and different sites (% of Firefox users and Percent of Heavy Repeat Visitors). So it simply isn’t the case that every sites’ numbers will be equally inflated. Which pretty much seems to demolish Anil’s case, since every site won’t be effected equally.

So let’s go on to point number two – the impact of cookie deletion on many critical web analyses. Here’s Anil’s take:

"I understand Gary’s issue about repeat users and new users. But again, if you use two different systems they will report different numbers so which one is correct?
As Jacques Warren pointed out as a response to Gary’s post, the right solution (at this time) is to provide a reason for users to not delete their cookie (or give a reason to login). If Gary care’s about repeat users then I am sure he has strategies to get them engaged and give them a reason to login (or not delete cookies). Give users a reason to be loyal and they will be. Then you won’t have to worry about cookie deletion and hence your numbers will be accurate. Till you get to that level any number is a close estimate weather it is panel based or cookie based; and is not worth loosing sleep over."

This is the part I really don’t get. This isn’t an issue about comScore vs web analytics. When you use two different systems then one of them is more correct or both are correct or both are wrong. There is a real world we are trying to measure. And the problem is that when you do an analysis of "New" visitors and a significant percentage of your "New" visitors aren’t new, then your analysis sucks. It’s that simple. You aren’t looking at the right data and you have no reason to draw any conclusions from the data. And if you do forge conclusions from the data, they are probably wrong. For an analyst to suggest that very large non-random errors in the data don’t matter is, to say the least, perplexing. It’s as if someone told me that though I meant to poll Democrats but got half Republicans it won’t impact my survey findings on the Democratic Primary!

Jacques Warren may well be right – the only solution we may be left with is asking the measurement Tail to wag the web site dog. But how can anyone think this is something we shouldn’t be worried about (Jacques certainly seems to be!). First, we have to know whether this is true and then we have to let clients (or bosses) know that if they want measurement they have to fundamentally change their measurement approach to one based on opt-in principles. If you don’t think this is a big deal,  go talk to industries that live in that world.

Nor is it reasonable to say that a site should be able to engage visitors enough to get them not to delete their cookies. If cookie deletion is a function of mass deletion or browser exit settings, that simply isn’t an option. And we all know what great success the measurement community had convincing everyone that 3rd Party cookies aren’t a privacy issue. I look forward to telling my clients that they have to become evangelists for cookie persistence. I’m sure they will love that!

And not all sites are Amazon.Com – many sites aren’t looking to marry visitors – they just want to date them occasionally. It’s unreasonable to think that many sites can get users to opt-in for measurement purposes when all they are doing is reading an article.

I’m not taking the comScore study for granted. As I pointed out in my original post, there are several good reasons to doubt the results and it may be that we don’t have to fundamentally change our ways of doing business. But as far as I’m concerned, if you’re a web analyst and you weren’t worried about the comScore results then you aren’t getting it.

Which brings me Clint’s concerns about the study being so limited. I share his concern – and it’s probably the main reason I think we shouldn’t jump to conclusions about the quality of our data. There are just too many studies based on single points of reference that turn out to be seriously flawed when applied to much larger industry. And, as I pointed out in my original post, no site is likely to be more vulnerable to serial deleters than a major portal.

I also agree with much of what Jacques says – assuming that we really do find our numbers are this flawed. It will certainly mean that sites need to focus measurement much more on opt-in users – and find many new ways to drive that level of commitment. I won’t pretend I think this is easy or that it won’t have a big impact on our business – and I’d much prefer a truly workable technical solution. And there are, as I discussed as well, technical methods for screening off the effects of cookie deletion from many kinds of analysis. If the community as a whole ends up buying into the comScore numbers I think that all of these directions will emerge as very important.

Comments

]]> http://www.webpronews.com/more-on-the-great-comscore-debate-2007-04/feed 0 http://www.webpronews.com/an-analytic-road-map-2007-03 http://www.webpronews.com/an-analytic-road-map-2007-03#comments Mon, 26 Mar 2007 16:05:22 +0000 Gary Angel http://www.webpronews.com/?p=36440 Before I delve into the my real topic for today, I wanted to briefly remark on the comment James Gough left about my 10 Reasons we all have Ulcers post:

]]> Before I delve into the my real topic for today, I wanted to briefly remark on the comment James Gough left about my 10 Reasons we all have Ulcers post:

If we were aware that it would take at least 6 months and $70k to get even a basic version of Omniture installed we would have never even begun in the first place. The problem with the likes of Omnniture and websidestory is they take months of technical time to implement and then becomes a constant battle to keep the tags up to date as you deploy new microsites and pages. We have moved to a solution that requires no complicated tagging and have finally allowed marketeers to control and believe in the metrics. implementation and the time to manage ongoing is the biggest cost to a business of installing these products and vital companies look for products that have a light foot print but rich reporting.

I can’t argue with James’ assessment of the costs – it can be done cheaper and quicker than that – but it can also be a lot more expensive. And what he says about keeping the tags up to date and implementation and management time are pretty much dead-on. These are particularly severe problems for big-time publishers. Frankly, I’d be very interested in hearing what solutions James settled on and what they are doing with it. I’m a big believer in a light tag myself – but I haven’t found an implementation out there that doesn’t sacrifice some things about Omniture and WebSideStory that I really like. I’ve asked James to elaborate and, assuming he does and is okay with it, I’ll post it up here.

I could probably take up any of the 10 "Ulcer items" and spend at least a few posts on each – but I’m going to focus on #9. Not only because I think it may be the most important but also because I want to use it as a lead-in to series that will tackle real "how-to" issues for some of the most common types of site analysis. Number Nine was:

9. Not Having a Road-Map

Probably even more important than a good method to getting where you want to go with web analytics is having a clear analytic road-map. I think the biggest challenge for most organizations is after the honeymoon (post-implementation) – when everyone has gotten over the joy of just "having data" and actually wants to do something with it…

When I say that this is the biggest challenge for most companies, I mean it. Nowhere, in my opinion, is failure more likely or more common. And it is at this stage that companies are most likely to engage a company like Semphonic and, I hope, get the most value from us.

But stating the problem is not exactly putting forth a solution. It’s all very well to say an organization should have an analytic road map when the real problem is that no one knows what might come first much less what might follow in the next 10 months. It’s like asking a novice chess player to describe an elaborate attacking combination when they are still trying to learn the basic moves.

So what is an analytic road-map, what goes into it and how can an organization go about creating one?

What is an Analytic Road-map?

When I talk about an Analytic Road-map, I have something very particular in mind. In essence, it’s a plan that lays out a series of analytic projects. Each project is a discrete analysis. And the plan specifies which projects are going to be done and in what order. That’s it. A bare-bones, simple road-map might be sketched like this:

Functional Site Analysis -> Internal Search Analysis -> Tool Loyalty Analysis -> Home Page Real-Estate Analysis -> Landing Page Analysis

Simple.

What goes into the Road-map?

This gets a lot more complex. Any sort of analysis might go into the roadmap. And the actual make-up of a road-map is inevitably a complex compromise. Three factors – none trivial – need to be sorted out when building a road-map. First and most important is what types of analysis are most likely to have a large impact on site success. What makes this particularly challenging, of course, is that you have to predict this before doing the analysis – and we all know that many analytic projects simply don’t end up providing interesting information. That can’t be helped, but there are some good guidelines for tackling the value question that I’ll try to explicate.

The second consideration for the road-map is how hard an analysis is likely to be and what types and amounts of data are essential. When you put together a plan, you ignore these issues at your peril. Again, you don’t always know what data you’re going to need. But it’s not always a mystery either. Some kinds of analysis will clearly take lots of long-term data, others are obviously going to require external data integration. You’ll pretty never want to start with either – but you shouldn’t ignore them either. Many of your most valuable analytic projects are going to fall into one or both of these camps.

Finally, when you build a road-map you need to consider the sophistication of the analysis relative to the organization and the extent to which one analysis may depend upon or deepen another. You might think about this as similar to putting together a curriculum for a student. Introductory classes provide a foundation and language which can be gradually deepened. In an academic curriculum, it’s usually assumed that the deepest classes are the most valuable. You’d skip the introductory classes if you could – it just isn’t possible for most us. In analytics, on the other hand, there is no such correlation. The simpler analytic projects you might start with may drive as much or more value as very complex ones. It’s important, in analytics, never to equate complexity with value. In my experience, the relationship is more likely to be inverse than direct!

How do you create a Road-map?

When we help clients put together a road-map for analytics, we start with what type of site is involved. The type of site (eCommerce, Lead Generation, Ad-Based, Customer Support, Operational, Branding, etc.) dominates every other consideration when setting the analytic table. For eCommerce sites, we’re going to be choosing from a grab-bag of analytic projects that include: Functional Analysis, Cart Drop-off, Cross-Sell Opportunities, Personalization Strategy Analysis, Internal Search Optimization, Completer Optimization, SEM Checkup, Longitudinal Analysis, Up-sell Analysis, Market Basket Analysis and Re-Assurance strategy. Typically, we’ll begin with a Functional Analysis. I like to put this first because it sets the table so well for analytics in general – providing a great way to baseline performance and get a common working vocabulary. A Cart analysis is somewhat de rigueur – but it isn’t an analysis I look forward to. It often bears no fruit these days, and if I think the cart has already been well optimized I look push this one back. For many eCommerce Sites, internal search is vital – and it’s rarely been fully optimized. So that’s an area I’d be strongly inclined to bring to the front. Likewise, if personalization hasn’t been well addressed the Personalization Strategy will almost certainly bubble to the top. In each case, it’s the likely impact to most sites that makes me push these projects forward. You also need to have nose for the money and to follow your intuitions about how well it’s being spent. If a site is investing heavily in Search Marketing, then the hard dollars flowing out will almost always make this a top candidate for the roadmap. That’s especially true if you lack confidence that the effort is being well conducted.

For a Lead-Generation site, some of the same types of analysis also come to the fore: Functional Analysis, and Personalization Strategy are almost always near the top of list. However, Completer Optimization is an analysis that is quite simple and often has considerable upside for Lead-Generation sites. So I like to put this one near the top of the list. In addition, Longitudinal Analysis is usually a critical component of Lead-Generation (focusing on which channels ultimately drive lead-quality) – but I try never to schedule right up front because it typically involves significant integration. You can’t ignore lead-quality issues, however. So it’s imperative that once you’ve put together a couple of easier wins that you actually tackle this issue.

Ad-Based sites have quite a different focus. As with other sites types, my inclination is still to lead with a Functional Analysis. And Internal Search analysis (often more important to these sites than any other tool or page) will nearly always come next on the list. But after that, the essential analysis is one focused on building a model for the impact of each site component on consumption. This analytic model will form the basis of nearly every other piece of work you do, so it pretty much has to come early in the process even though it is always tricky. If the site is sophisticated, I might be inclined to tackle Personalization Strategies next. Media sites have been less inclined to adopt personalization strategies than have high-product mix eCommerce sites – but I think the upside potential for media sites is even greater.

The sense I hope to give from these remarks is how to begin thinking about what to tackle first and where to go from there. You start with what seems obviously important in terms of usage and value on your site. Consider what’s involved in a possible analysis. Then start to fit the pieces together in a plan. Keep in mind that while some plans may be much better than others, there’s no such thing as the "right" plan. If you can’t decide which of two projects to do first, don’t agonize over it. Pick one then slot the other in behind it. No big deal.

On the other hand, if you can’t decide which of ten projects to do first, you may need help!

What makes a Road-map valuable?

Like any other significant organizational effort, you can’t expect to get much from analytics unless you have a plan and can manage to your success. Analysts, like everyone else, need guidance about what to focus on, how long to spend on it and what to think about for the long-term. The Road-map is a way to provide all that – it’s also a great tool for building consensus within an organization about what analysis is for and what problems most need addressing. It’s also a way to set expectations for what analytics is going to produce.

Like most other plans, an Analytic Road-Map isn’t meant to be an iron-clad blue-print. Changing business circumstances, the results of each analysis and even learnings about tool capabilities can and will change the priorities that shaped the plan originally. But if you have the plan, chances are you’ll be much clearer in thinking about when change is necessary and why you are doing it. You’ll also know, when you go chasing off after a new problem what you are giving up or postponing. That, in itself, can be a valuable defense for measurement organizations who find themselves being passed like a public handkerchief from problem to problem by Senior Managers with fleeting (and pressing) data whims. Sometimes, those short term needs are in fact more important than even the best-laid plans. But sometimes changing directives come when you haven’t done a good enough job communicating what you actually think you SHOULD be working on.

Comments

]]> http://www.webpronews.com/an-analytic-road-map-2007-03/feed 0 http://www.webpronews.com/key-issues-in-web-anaytics-implementation-and-rollout-2007-03 http://www.webpronews.com/key-issues-in-web-anaytics-implementation-and-rollout-2007-03#comments Thu, 22 Mar 2007 02:08:53 +0000 Gary Angel http://www.webpronews.com/?p=36349 I spent most of the last two weeks on the road – teaching down in San Diego at the WebSideStory DMU and manning a booth at the Omniture Summit. Going out and talking to so many people is always interesting (if a bit daunting for an essentially shy guy) – and if you take out the travel parts it was all pretty enjoyable.

Disciplined Search Engine Optimization
Establishing a Measurement and Process Discipline for SEO

This coming Wednesday Paul Bruemmer and I are going to be doing a webinar sponsored by WebSideStory on SEO – talking about how to put a discipline of measurement and process around a serious search optimization effort. It’s a pretty interesting topic – because I think SEO, particularly from a measurement standpoint, is very poorly understood. We’ve been working with Paul and RedDoor on various things for a while now, and I think you’ll find his methodical approach enlightening and refreshing in a discipline that often seems mysterious and chaotic. You can register to join us at http://www.websidestory.com/promotions/active-marketing-webinar-1115.html)

Visitor Segmentation: Segment Methodology

The discussion of segment creation capabilities in the last post reflects how vast the differences are between various systems and how very different their visitor segmentation capabilities are. Today’s topic – how segments get created – is simpler and less diverse.

Essentially, there are two main divisions in segmentation methodology. The first main divide is between sampling-based segmentation systems and those that use all of the data (comprehensive). The second issue is whether segments are created in real-time or delayed.

These aren’t necessarily either/or decisions. Some tool sets have various components and approaches that span pretty much every combination of these alternatives – and there are some pretty good reasons why each has a place.

Let’s start with sampled data vs. comprehensive data. To me, the issues here are pretty simple. Vendors provide sampled data for one simple reason – performance. It’s much easier to deliver fast answers against sampled data than it is against comprehensive data – especially if you’re talking about a large web site with tens or hundreds of millions of requests monthly.

In a perfect world, I suppose you’d like to have near instantaneous analysis against comprehensive data. But this isn’t, of course, a perfect world. And the real question is how much you lose when you employ a sampling methodology.

On the whole, I think sampling solutions are very viable. Sampling, done correctly, can almost always provide answers that are near-enough – especially given the built-in slop factor inherent in web analysis. New users of web analytic solutions are frequently (and rightly) put off by the fact that “nothing ever ties!” Rightly or wrongly, though, you get used to some level of imperfection. Indeed, I think one of the virtues of sampling is that it puts your expectations about the data in a firmly reasonable place.

Compared to some other methods of data trimming (like dropping infrequent paths), sampling is very much to be preferred. Sampling rarely distorts the data into unrecognizable forms – whereas data trimming will frequently do just that in situations where the data has a very long tail.

In addition, a great deal of customer segmentation is for purely analytic purposes – not to support management reporting. And for analytic purposes, the difference between sampled data and comprehensive data is quite often not important. This is also one of those times when it’s nice to be able to check samples against comprehensive data – to either validate conclusions or spot-check for cases where your sampled data is returning suspect answers.

Which brings us to the second main divide in segmentation methodologies – real-time vs. delayed segment creation. And it’s probably obvious that there is a deep relationship between these two issues. Real-time segmentation may be impossible without sampling – so one of the biggest potential benefits to sampling is enabling the analyst to make and report on segments without having to wait hours or days.

How big a deal is this? It’s actually pretty important. If you are using segments to support management reporting, you probably won’t care much about this. After all, if you’re going to be using a segment for the next couple of years, it doesn’t much matter if it takes a day or two to create. But most segmentation is for analytic purposes – and needs to be responsive to changing needs. What’s more, an analyst often doesn’t know if a segment is going to be useful. So if you have to wait a long time to view the results of a segment, it can make the cycle times on analysis frustratingly long. This is especially problematic if your system places caps on how many segments you can create (this is pretty common when segments are being built on a vendor’s data warehouse). We’ve more than once used up our quota of segments because of segment definition errors, mistakes in judgement and just plain wrong guesses about what might prove interesting. And believe me, it isn’t fun to tell a client you can’t finish an analysis because you can’t create the segments you NOW know you really need!

So here is the recap on segment methodology: ideally, you’d like to be able to build segments in real-time against comprehensive data. But, for analytic purposes, it’s much better to have real-time segmentation with sampled data than heavily delayed segments with comprehensive data. And, if you are doing serious analysis, it’s important to have either unlimited segmentation or a very large number of available segmentations. On the other hand, if you’re focused on segmentation for management reporting, then comprehensive data is much more important than real-time capability – and you probably won’t need as many available segmentations.

What’s more, while the perfect solution would be a single tool providing unlimited, comprehensive real-time segmentation, it isn’t much worse to have a suite of tools that offer a choice of real-time segmentation against sampled data and delayed segmentation against comprehensive data. This type of solution will still meet the needs of almost every situation quite admirably. And here’s the good news – more and more tools are supporting a rich set of visitor segmentation methods – enough to insure that you can do what you need quite of bit of the time. That’s a big change from a few years back and is one of the real bright spots in the web analytic toolspace.

Comments

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

If you’re a regular reader of this blog, you’ll probably have noticed one pretty glaring area of omission. I’ve almost never talked about tools. There are reasons for this – not the least of which is that with five or six different solutions all commanding a fair amount of marketplace respect it’s hard to talk about tools and not lose a good chunk of your readers. In addition, I think there is a lot to be said for keeping as much of your thinking about web analytics as tool independent as possible.

One of our clients paid us a compliment recently that really stuck in my mind. She said, “What I like about you guys is that you don’t think about what the tool can do – you think about what you want to measure and then you find a way for the tool to do it!” I think the more often that’s true, the better – and one of the main features of the Functional methodology I’ve been banging away at is that it’s pretty much tool independent.

As my SEMphonic co-founder Joel Hadary loves to say, “A fool and a tool is still a fool.” Tools do matter though. And no matter how clever you are with a tool, each and every one has basic limitations that can’t be transcended and strengths that can be exploited.

So I’m going to try some blogs about tools over the next couple of months – not a formal series like Functionalism but enough, I hope, to make a noticeable dent in the topic. I’m not going to cover every tool and I’m not going to do any classic product comparisons. Instead, what I’m going to cover is a mixture of how tool capabilities and real-world needs seem to me to come together or fall apart in specific cases. I’ll probably talk quite a bit about SiteCatalyst and HBX since those are the tools we use most frequently. But I’m also going to talk about some products we haven’t used much but that seem to me to add real value to the web analytics tool kit in one respect or another.

I’d also like to point you to a couple of “White Papers” we’ve produced recently that are very tool focused. The two are somewhat different in tone – largely unintentionally – but I think nicely illustrate both sides of what you need to know about tools. The first is called the Art of Omniture. It’s not on our web site yet, but if you post a request here I’ll send you one. This paper is a fairly broad overview of some of the most important Omniture capabilities. It includes a good discussion of the various customer segmentation options in Omniture (Warehouse, ASI, Discover) and some pointers on building and using visitor filters. It also has a very detailed set of tips for solving or avoiding some common problems with the Excel tool. So pretty much all of the paper is focused on using the tool when you already have a specific task in mind.

The second paper was written for HBX and is also a Tips & Tricks article. However, it provides much less detail about specific tool issues and more about how to think about using the tool when you have certain business/analysis problems. There are some techniques in here (like using Campaigns to track Longitudinal behavior) that are quite clever in their own right – but they also illustrate some common business problems that you may not even have considered if your field-of-view has become tool-restricted. I think tool-blindness (focusing on what your tool provides instead of what you need) is a real problem in analytics – especially since most analysts and even vendors only really work with a single tool. You can get the HBX article on our web site at http://www.semphonic.com/resources/whitepapers.asp - it really isn’t a White Paper at all but it seemed like the best place to put it. I’m working on a considerably expanded version of this, so I’d love to hear feedback or suggestions for it!

I’m going to try and write both ways about tools (how to use them, how to think about using them) in some upcoming blogs. And I’m also going to talk about what I’d really like to see web analytics tools do that they currently don’t. Why? Well, in addition to giving vendors some good ideas, these suggestions serve to clarify what kinds of problems analytics should be solving and what we need to know to really solve them.

Comments

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.