Web Analytics Interview

    August 28, 2007

I’m actually in Hawaii right now, so I’m just not up to producing my regular Sunday Blog. It would take a truly dedicated blogger to produce anything under these far too idylic circumstances.
Maybe Marshall Sponder could do it, but I’m just going to post an interview I did with Jethro Wallenburg a few weeks back for his blog. I liked the way it came out and it sets the table – directionally – for my next blog. We have some major new additions to the Semphonic team and I’m going to be talking about who and why in my next post!


Can you start by talking a little bit about your background?

[Gary] My degree is in Philosophy – and people certainly will see hints of that in my writing. But there aren’t many jobs in philosophy! So when I got out of college I started working in computers and became one of many self-taught computer programmers. I ended up building several commercial software systems that focused on financial analysis and I got very interested in data analysis in general. To this day, I find few things as satisfying as writing computer programs. I think it of as the great CRAFT job of the modern era.

[Jethro] How did you get into the web analytics industry?

[Gary] In the 90’s I started out working on large-scale database analysis systems for Visa and American Express. When the internet started to explode, a lot of people I knew spun out of those places and were asking me if I could help them understand what was happening on their web sites. It seemed like a natural transition – and one with loads of growth potential. So I co-founded Semphonic in ‘97 to focus on web analytics. One of the few good "market" calls I’ve ever made in my life!

[Jethro] Your company uses the term ‘Functionalism’. Can you explain in easy-to-understand words what that mean?

[Gary] When we first started doing web analytics, we focused on pathing. Then on correlation to conversion. But eventually we recognized that the type of analysis we were doing didn’t help much for many non eCommerce sites and also dealt only with a small subset of pages with any effectiveness. Most of our clients had big web sites with lots of goals that didn’t necessarily involve an online sale and lots of pages that weren’t about selling a specific product. Functionalism evolved out of that recognition. Essentially, Functionalism is a simple idea: pages on your web site are designed to accomplish different things (navigation, convincing, converting, informing, thanking, etc.). So to measure their effectiveness, you have to select KPIs that are appropriate to their function. Functionalism just is a set of page types and the best KPIs we’ve found for those types. It’s absurdly simple, put this way, but it actually helps prevent many extremely common mistakes in web measurement.

[Jethro] Is ‘Functionalism’ new wine or old wine in new bags?

[Gary] On the one hand, I’d like to say that what matters is that it works. Sometimes, the bag is pretty important to the product. I’ve found that one of the biggest challenges in web measurement is helping all the other stakeholders in a website understand what it’s for and how it works. Functionalism is beautiful for that – and that’s a lot about the bag. On the other hand, I think what is truly new about Functionalism is the care with which KPIs are matched to function. Lot’s of people have recognized that there are different types of pages on a web site, but they always ended up with subjective rules about what made those pages good or bad. Functionalism isn’t about that at all – it’s about how, exactly, to measure those pages. I don’t think that work had ever been done systematically.

What I see in the real-world is that most analysts trying to do web analytics don’t have a recipe at all – they use something I call KPI stew. KPI stew is when you throw every measurement you have against a wall and see if any stick. Believe me, having a recipe makes a tastier dish!

I’ll mention also that Functionalism is public domain – you can go get the White Paper from our web site (you don’t even have to register) and it will tell you exactly how to do it. It isn’t some weird secret incantation we chant only while hidden deep in our offices.

[Jethro] What differentiates the SEMphonic web analytics consulting engagement from those with other companies offering web analytics consulting services?

[Gary] I’m going to forgoe the usual stuff about how much we care, years of experience, yada, yada, yada… right? When you get down to it, there are a couple of things that really matter when you hire a consulting firm. I think the first – and probably the most important – is the quality of thought you’re getting. Pretty much every consulting situation is unique; so you need to feel comfortable that the people you’re hiring are actually going to add value to YOU. Obviously, it would be fruitless and silly for me to try and compare our "quality of thought" to other consulting firms. But that’s why things like the Functionalism White Papers and Blogs are important. They give people a chance to really assess what we know and how we express what we know – both vital parts of being a consultant. I also think our BS quotient is unusually low!

After quality of thought, some differentiating factors that matter are things like practice specialty, structure of the firm, and how much leverage a consulting firm actually brings to the table.

We do have some practice specialties. For example, we do lot’s of work on optimizing sites that don’t have classic conversions and focus less on traditional retail. So I think we’re better at Lead Gen, Ad-Based and Branding sites than many people. We’ve also built up a significant practice area around Search Marketing and Internal Search. We’re not a PPC or SEO firm – we do what we call SEM Analytics – which is really just web analytics about SEM. It’s a really interesting field and we’ve been pioneers in that space.

In terms of structure, we’re exclusively a web analytics consultancy (and a hands-on one at that). We don’t do design. We don’t do Bid Management. Just analytics. That makes it more likely that you’ll be engaged with someone with real experience and expertise – not easy to find these days. We’re also not about a single product. When you work with professional services teams from a vendor, you’re always working with consultants who are necessarily product focused. We (and I think most independent consulting companies) are business focused. It makes a huge difference in the type of work you’ll get.

Leverage (re-using your intellectual capital) is sometimes hard to find in web analytics. Tools like Functionalism do make it genuinely easier for us to deliver consistent value to clients. But like anyone else in consulting, I’m always wishing there was more leverage than there actually is!

[Jethro] Where are we going with Web Analytics (something you only know)? For example, what will 2010 bring us…?

[Gary] If only I know it, we probably won’t be there by 2010. Real-world practice always seems to move so much slower than we expect it to! I think it’s clear that there’s an increasing merger between traditional BI and Web Analytics. This is true both in terms of data integration and tools. That’s certainly going to accelerate and I see no reason why, in three years, the two disciplines will be separate in any meaningful sense. In addition, I think we’ll start to see much more "data-driven" analysis within web analytics. Tools like Touch Clarity do this kind of analysis in the background for machine optimization. But the analytic tools don’t provide any similar data-driven segmentation tools for analysts to use. That’s a shame, and I think it will be an area where one or more tools have a chance to really distance themselves in terms of functionality.

[Jethro] How does SEMphonic fit into that picture?

[Gary] When I first cut my teeth in large-scale database analysis and database marketing, the hot tools were cluster and neural network analysis. The most sophisticated companies were just starting to use these tools to do predictive modelling on campaigns. You know what the hot tools are twenty years later? Cluster and neural network analysis for predictive modelling. Analytic Measurement is one of those things that only companies who really commit to it will do well. And the techniques, while complicated, are unlikely to change dramatically. What’s more at stake is culture and process.

Online marketing IS driving more companies into a real "culture of measurement." How does Semphonic fit into that? We’re a very hand’s on firm. I don’t see us fulfilling a role like Eric Peterson – helping companies build culture. I think we are more likely to be the agent of choice for companies that want to build the best analytics into that culture. There just aren’t that many people or companies that are or will ever be genuinely good at sophisticated analytics. That’s the market we play in. It fits us as people and I think there will always be a significant place for great hands-on practitioners.

[Jethro] Can you give us 3 useful tips for a successful Web Analytics project?

[Gary] The nature of this answer depends a lot on the project. Here are some fairly general tips that apply to most types of web analysis. First, don’t "explore" data. Not only are web tools lousy at data exploration, data exploration in general is a waste of time (I’d say almost the same about data visualization, by the way. It’s profoundly misunderstood). It’s vastly more effective to start with a known problem and even a hypothesis about that problem. That’s just one of many reasons why you can’t do useful web analytics without understanding the underlying business. Second, never start with what your tool can do. Start by thinking about what analysis you need to answer your question. Then think about how the tool can be made to get the data. Way too many analysts start with what their tool can accomplish. This becomes a box that traps the analyst. Finally, never finish an analysis without coming up with a list of possible changes. If you’re just reporting numbers you’re wasting everyone’s time. The vast majority of decision-makers don’t want data – they want opinions informed by data.

[Jethro] What are the most made mistakes in WA projects and how can you avoid them?

[Gary] By far the most common mistake in web analytics is to misread correlation for cause. Web sites are navigational interest funnels. What do I mean by that? Think about Exit Pages. People used to talk about Top 10 Exit pages as a meaningful report. It isn’t. It’s garbage. And the reason is that a web site is an interest-funnel. The longer a visitor stays on it, the less likely a visitor is to leave. There is almost no web site we’ve every measured that doesn’t fit this paradigm. So what’s the highest exit page on most sites? The highest Entry page. These types of navigational effects are EVERYWHERE in web analytics – and dramatically influence just about every standard report, from Exit Rates to Correlation with Conversion. That’s one of the reasons a method like Functionalism is valuable. It’s carefully thought out to avoid most errors based on self-selection and navigation.

[Jethro] What is more important for a successful WA Project: the tool or the consultant?

[Gary] I’ve taken issue before with the 90/10 rule – which I guess makes me a traitor to my class. But the question as phrased isn’t really meaningful. In one sense, the answer is obvious. People are far more important than tools. I’ve written before about Robert McNamara – a world-changing Systems Analyst – and the most powerful computer he ever used had punch cards. But that’s kind of misleading. Take genius out of the equation. For most of us, we can solve a huge math problem much faster with a calculator than without. If you don’t give your analysts decent tools, you’re wasting their time and your money. So yes, a tool without an analyst is useless. An analyst without a tool may occasionally be valuable. But 99% of the time you need both. And the quality of tool makes a very significant difference in the value of the analyst.

[Jethro] How do you think a web analyst should spend their day?

[Gary] If it were up to me, eating and watching movies.

[Jethro] ……………

[Gary] Seriously, you’re never going to spend all your time looking at data. Two things I said earlier really apply here. One, an analyst needs to have a plan. When you’re doing analysis, you should be doing it with a specific problem and set of questions in mind. I NEVER start an analysis without a pretty good idea of what I’m going to look at. And if I don’t already have that, it’s the first thing I think about. When I’m working with our new analysts, I typically hand them a pretty comprehensive outline of what to look at when they begin a project. It makes them far more productive. Second, I said earlier that an analyst has to understand the business. Too often, web analysts (even when their not consultants) don’t really engage with the business. That makes for bad analysis. Analysts need to talk with and be part of the decisions their influencing. If they aren’t, then they’ll never really know what matters in the data.

[Jethro] Do you want to share your thoughts on the following subject with some words?

Google Analytics? I have mixed feelings. This is a great tool for free. But it lacks many essential ingredients for good web analytics. So if you are serious about web analytics, it’s the wrong way to go even as a start. Why? Because most of the real cost of a web analytics package is in the learning, culture and human effort that surrounds it. So if you are fairly confident you’ll need enterprise analytics, you’re better off investing in enterprise web analytics. That being said, this is a great tool for small and mid-size web sites especially if focused on SEM. In addition, you can use it in conjunction with other tools to take advantage of things like the Google Optimizer.

Web Analytics in Europe? I’ll quote Phil Kemelor, our VP of Strategic Consulting, "I think European vendors stress their local presence and support capability in stronger terms than North American firms. North American firms focus a great deal on functionality, features and scalability." This shows itself in all sorts of ways – including the absymal level of customer support that our North American clients routinely endure!

Visual Sciences? The integration of the VS Tool and HBX has great potential – maybe to be the best combination out there. But as a product and company they haven’t fully executed on that vision. With all of the acquisition rumours, there is a lot of uncertainty around the company. I’d really like to see that get resolved because this is very popular choice with out customers and the combined solutions have much to recommend them.

Omniture? This is clearly the leader in web analytics right now. Unless you have either cost issues or strong reasons not to go with a SaS solution, you’re probably going to be looking at Omniture. Discover 2.0 is impressive, though I wish it was priced like Discover 1.5! Lot’s of our clients haven’t made that jump for price reasons – which is frustrating for us. Also, with Omniture, you’re looking at a really rich analytic solution but you’ll definitely have to work harder on the implementation. This is a company with a clear vision, a very strong product offering and lots of market momentum. What’s not to like?

Webtrends? I think they’ve been making considerable strides toward once again being considered a real force in the industry. I like the warehouse direction – I think it fits the emerging focus on integration that I talked about earlier. They also seem to be moving toward more data-driven analysis – which is important. In addition, some companies are still leery of SaS for web analytics. Either for security reasons (which I think is foolish, but you can’t argue security) or because of heavy internal data integration needs (which isn’t foolish at all). Hybrid solutions like WebTrends and Unica definitely have an advantage in these markets. And maybe that niche will turn out to be a pretty good one.

[Jethro]Thank you very much Gary.

And don’t forget to register for X Change and join our fantastic line-up of experts for in-depth conversations on web analytics and measurement!