With today’s Functional Page Class, Completers, I think there’s another profound lesson to be learned. When I go into a bank and deposit money, my teller (and even my bank machine) routinely finish by saying something like “You’re all set, here’s a record of your transaction. Is there anything else we can help you with today?”

Now on most web sites, I can be pretty confident that when I purchase something I’ll get part of this with the Thank You page. I’ll get the record of my transaction (usually conveniently emailed to me). I’ll get the “You’re all set.” Usually in the form of the Thank You. But how often do I get the “Is there anything else we can help you with today?” Not bloody often.

I’ve mentioned before that one of the guiding principles of Functionalism is that it should be able to analyze every single page on a web site. And Completers are the type of page that traditionally has been completely ignored. Completers, you see, are your “Thank You” pages – whether for a sign-up, a registration, or an order.

In the “conversion” paradigm, you can’t do much with Completers. They are either 100% associated with orders or pretty close to zero percent associated (depending on how you want to look at it). Either way, you aren’t going to find out much about your Completer page by looking at conversion percentages.

So what’s a Completor Page for? Well, it has the three functions we’ve already talked about: confirmation that the visitor was successful, access to a record of a transaction and opening up a new dialog.

There are potentially measures for all three of these things – but the first two are almost always handled reasonably well. If you are looking for signs that your “handshake” with the customer isn’t being understood, here are a few: a high number of refreshes on the “Thank You.” Next steps to help, back into the process or Customer Support or order/account tracking pages. Calls from this page (use a unique 800 number please!). These are all danger signs – but they are rare.

So let’s talk about “opening up a new dialog.” Everybody knows that your existing customers are your most valuable asset. You’ve just created one. And you’re just letting them go – with a page that says “Order Confirmed” and not much else. Why? Isn’t there something you want to tell them?

Listen, this doesn’t have to be – maybe even shouldn’t be – a sales thing. You already had an opportunity to upsell in the cart and checkout. But wouldn’t it be nice to point your new customer to how to get help, to product resources, to some related sites, to some helpful information, to something – anything – other than goodbye? And bad Completer pages are by no means limited to taking orders. Some of the worst I’ve ever seen are on pages taking leads or when I submit a request for a document. My lord, if I just asked you for your White Paper on dieting with chocolate cake isn’t there something else you want to offer me (like a big spoon).

I definitely believe in the Long Goodbye. This isn’t telemarketing. Your customers can go anytime they want. They aren’t going to be upset by your offering them options on the “Thank You” page!

So the most important Functional measure for Completer pages is re-engagement with the site. We measure this in two ways – tracking against hard exits and tracking against offered routes. Why two ways? Many customers who are offered “Thank You” pages will break back to the Home Page or use other top navigation options. That isn’t bad, but it means you haven’t engaged them with your offers. So, like a router, you should measure re-engagement with the site via intended offers.

Part of the discipline that measurement provides is a formal process – we built this page, what do we want it to do and how are we going to measure that it’s doing it? And by answering these last two questions, you can often come to a better understanding of how the page should work – even without doing the measurement! Unlike many Functional techniques, this one is almost a complete slam dunk – and it can result in a substantial payoff on many, many sites.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

(Quick Note to Readers: you’ve probably noticed that we don’t show comments on posts – we made a decision when we began that the Blog is not intended as a discussion area but more as a corporate viewpoint. I don’t know how wise that is, but we do read and reply to every comment, and they often trigger new posts. So I do welcome comments even if don’t log them – and feel free to comment on that as well.)

The question was this: what is good score for a Router page – 50%, 70%, 10% – and is there a standard benchmark an analyst can use? The question applied to Router pages, but it could just as easily be asked for any other page type in the Functionalist library.

I’ll start by saying that this is a question we get frequently from out clients. And not just about Functional concepts. Clients want to know how their conversion rate compares to the competition, how their PPC performance stacks up and so on and so on.

These are good questions. In fact, knowing how you compare to the competition (or to some gold standard) is one of the most valuable pieces of contextual information a marketer or analyst can have. It’s also one of the most difficult to obtain and use correctly – and it’s something we’ve usually had to punt on.

Let me explain why.

Let’s start with the obvious. You can’t compare your web site to just anyone’s. A portal is a fundamentally different beast than an e-commerce site, a customer support site, and operations site or a multi-purpose corporate site. The performance of pages for these different sites will invariably be different. There is no possibility of a meaningful comparison – even at the Functional level. That’s why benchmarking firms go out of their way to compare apples to apples. Nor is likely that a seller of a 100K service will have similar page performances to a seller of a $20 product. The sales cycle is just too different to make the way visitors use your pages comparable.

But how similar does an apple need to be? Here, I think there’s actually some good news. Without something like Functionalism, I think the apples are never going to be similar enough. Even your closest competitor will have such different sourcing, online campaigns, pass-by traffic and merchandising mix that basic conversion rates will be meaningless. Let’s say your closest competitor has a 4% conversion rate and you have a 3.5% conversion rate. Is your website underperforming? Possibly. But what if you also know that you have twice the organic volume of the other site on words that tend not to drive to conversion? And without that extra volume, your website conversion rate would be 4.1%. Are you better? Possibly. But there are sure to be a thousand other differences that make comparisons almost useless. Nor is conversion a “gold standard” – you certainly need to understand revenue per customer and probably even net income per customer.

Variations in website traffic quality are much larger than in same-store comparisons by region for traditional retailers. That means that many reasonable comparative measures in the bricks-and-mortar world don’t work well when applied to the internet.

Functionalism can help to bridge that gap. By comparing pages with specific functions, you’ve netted out a considerable quantity of the traffic quality variation and self-selection that can otherwise mar comparisons. But comparisons will still be vulnerable to many significant differences.

Indeed, this problem isn’t isolated to understanding competitive benchmarking – it makes it a continuing challenge to measure page performance over time. Because your own traffic mix is always changing, the visitors you drive to your web site are always varying in quality. If you add a PPC program, chances are that every KPI on every page is going to change – sometimes dramatically and sometimes subtly. This isn’t all a bad thing – it can help you understand how your PPC traffic differs from (and is similar to) other channels. But it also means that a simple comparison of before and after page performance won’t necessarily be meaningful.

The problem of measuring improvement when multiple variables are being changed will always be with us (and not even multivariate testing will solve them all). Much of the real work for an analyst is trying to insure that aside from the variable you’re testing, everything else is as constant as the real-world will allow.

All this being said, there are ways to think about the KPIs within Functionalism that can help you. With Routers, for instance, exits are much more sensitive to visitor quality than sideways routes. If you change sourcing and see a significant up-tick in exits, this might cue you a simple decline in visitor quality. And if there is a decline in visitor quality, you should see a matching decline on the landing page. If you don’t, then it may be that the page really doesn’t work as well for the new source. And since sideways routes are much less vulnerable to visitor quality, a Router paging losing a majority of its traffic to sideways routes is nearly always in need of re-design.

Even more important, you need to think about Functionalism as supporting a process – one of continuous measurable improvement. You use Functional KPIs to measure an existing state, suggest possible changes, and then try them. By A/B testing, you can screen off virtually every exogenous effect. But even with simple time-based rotation testing, you can be pretty sure – in the absence of dramatic changes surrounding your site or business – that you’re measuring a real effect.

Part of the reason you can be sure is that the KPIs allow you measure relatively subtle shifts in behavior over fairly short periods of time. So you can see if a Router is performing differently over a time frame that makes significant outside effects unlikely.

All of which leads me back to benchmarks. Of course, if 90% of the visitors on a Router page exit or go sideways it needs a re-design. But 50%? 30%? That’s harder – indeed – impossible in the abstract – to say. Perhaps using the Functionalist paradigm and gathering information from a wide variety of sites, we might begin to see enough true patterns emerge to make a good industry-vertical benchmark practical. We, unfortunately, are well short of the amount of data necessary to see if those benchmarks are even possible – much less publish them back to the world.

However, by isolating functions and appropriate measurements, you can get a much better sense of how pages in your web site compare. This can help you decide which ones to target testing changes for. And it can provide you with a nearly bullet-proof method to tell if you’ve gotten better.

Naturally, you’d still like to know how you really compare. And someday, perhaps, we’ll have apples to apples comparisons on the web that are as useful and available as those in traditional retail. But while Functionalism may bring that day a tad bit closer, it is a long way from making it a reality.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

You’d probably be surprised at how carefully traditional retail has studied the problems of navigation at almost every level. Store planners have made considerable study of product groupings and aisle placements. It isn’t by accident that the two things you want are often at opposite ends of the store. Or that the ultimate impulse buy – candy – is sitting right next to the cash register along with reading material for your bored eyes. Retailers try to understand the various shopping basket mixes that drive shoppers – and how to place goods both next to – and away from each other – to promote maximum cross-sell. Impulse buys need to be next to common necessities. Common mixed baskets need to be widely separated to force shoppers to traverse aisles. Aisle endpoints are loaded with impulse buys. It all makes perfect sense, and it doesn’t happen by accident.

One level up from this, you can think about how mall escalators are often arranged. They aren’t positioned for maximum convenience – but to insure significant store traversal. Indeed, even the arrangement of a mall is a matter of considerable study. Anchor stores (the core shopping stores that draw visitors) are placed at the ends. Boutiques line the connecting ways – drawing in pass-by traffic and supplementing overall traffic by adding to the experience.

Every website faces a similar set of challenges about how to move people from one area to another, where to place add-sells, when the shopper needs to rest and when the shopper needs to move, and how to get the shopper to the area(s) of the store they want. And the Functional page type that does most of the heavy lifting in this regard is the Router page.

Here’s the basic idea as stated in the whitepaper: Router Pages are those whose primary purpose is to move visitors into particular sections of the site. The presumption is that there is fairly substantive information about what the visitor might be interested in and these alternatives are presented as navigational elements in the body of the page.

Probably the most common Router pages exist as “mini-homes” underneath the top level navigation from the Home Page. Go to most web sites and pick an area like Products or Services or Support and you get a page with lots of drives to various sub-sections of the site.

One of the things that defines this page-type is that the visitor – by arriving – has indicated some interest in the topic. That’s why the home page is not a pure router – and why Search is a special case of router. So if a visitor arrives on your Main Products Page, you expect that visitor to drill down to content about Products. Another aspect of the Router page is that the page itself is generally light in content to “Convince” or “Close” the visitor to buy. It is trying to get the visitor to the right “Convincer” pages – not sell them itself.

So how do you measure Routers? By how well they move visitors to the Content they are supposed to. That means that for Routers (unlike most other pages) the type of action you’re most immediately interested in is what happens next. Routers shouldn’t be measured against conversion – because they aren’t responsible for selling. They shouldn’t be measured for return visitors – because they aren’t trying to get visitors to come back. In other words, you don’t measure the performance of your escalators by sales – you measure how often they deliver traffic to each door.

For Router Pages, that means that the basic KPI you’re focused on is the percentage of visitors who followed appropriate routes versus the percentage who didn’t. We call appropriate routes Body Routes, since they are typically the routes linked in the main body of the page. In addition, we like to measure various groupings of “bad” routes including “sideways,” “back-outs,” and “exits.”

Sideways routes are typically other top-level navigation routes. These aren’t horrible, in other conceptual schemes they wouldn’t raise an eyebrow. But if a router page shows lots of top-navigation next steps then it isn’t doing its job properly. Back-ups are another type of sub-optimal route – cases where a user goes from home-page to router and then back to home page. Again, this isn’t what you want of these pages. The last “bad” route is an exit – and, of course, an exit is almost always bad.

This grouping of routes is one of the cases where you have to do some extra work in the measurement. When you classify a page as a Router, you need to decide what the “intended” routes are. Grouping routes into the Intended, Sideways, Back, and Exit buckets makes for a great reporting template – one that meaningfully captures the comparative performance of each page as well as providing real information about how it may be sub-optimal.

The Functionalist KPIs also include a set of measurements for Re-Surface behaviors – cases where a visitor drills down on the Router Page as intended, then comes back up to the page.

We like to break-out these cases specially, because we’ve found that the performance for the page (especially for exits) is strongly effected by this behavior. Where sub-pages don’t do a lot of cross-linking, visitors may re-surface then exit. This can make a Router page look much worse than it is. In addition, re-surface behaviors may reveal useful information about where a visitor goes next and whether personalization of the page at this point might yield dividends. It’s quite possible that on re-surface views, a Router page should include a “Closer” element to try and drive the sale.

By far the trickiest aspect of router pages is measuring (and separating) re-surface from initial land behavior. This isn’t always an issue, and before bothering with a more complex analysis, the analyst should check and see the percentage of visits that contain multiple pages views of a Router.

One method for studying re-surface behaviors is to create segments based on visits with a single page view for the target page and those with multiple page views. By comparing the next steps for the single page view segment with the total, you can see the choices visitors made differently when re-surfacing.

Web sites, of course, aren’t like traditional retail in many respects. Visitors can jump will-nilly from here to there. They are more likely to be single product shoppers. Your competitors are always just a single step away. So you can’t expect to make the same choices as you might in a bricks-and-mortar world. But the lessons learned in traditional retail can help clarify your thinking about what you are trying to measure and how you think about the elements of your web site. You don’t measure your escalators by conversion – and you shouldn’t measure your Router Pages that way either!

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

The tone of the post is a bit odd – and I can best describe it as “I don’t like your idea and besides, I had it first!”

Here is a quick summary of Matt’s “historical “scheme:

“A scoring method was created to tabulate the performance of each page based on how well each page served – and was intended to serve – a variety of functions (e.g. explanatory, navigational, information collection and sales functions, among others).

A portion of the scoring method was based on heuristics to analyze the function categories noted above. We also extended the heuristics to score how well each page addressed the following three value/benefit categories: Functional, Economic and Psychological

Another portion of the scoring method was based on historical web analytics data.”

There are several things that seem problematic with this approach and readily distinguish it from Functionalism. Let’s start with the fact that Functionalism doesn’t have any heuristics. That seems like a pretty big difference to me. There are no rules of thumb in Functionalism for scoring pages. There are no subjective design qualifications. Matt is right to be skeptical of the degree to which Functionalism reduces the need for brilliant practitioners and is worried about the amount of effort it requires. The system he believes it mirrors would be rife with just those issues.

Functionalism is all about web measurement and it is rooted in actual web measurement practice. True, the analyst has to classify pages. But he doesn’t score them, he measures them using specific KPIs. It’s a huge difference. Functionalism actually has no method whatsoever for scoring pages. It has techniques for measuring them correctly.

And the page classification system doesn’t require the analyst to decide whether a page is attempting address “psychological” or “economic” concerns. In fact, the classification system is designed to be extraordinarily easy to use – I think any reasonable observer would expect virtually everyone involved with a web site to be able to easily and accurately classify the pages. And the substantive judgments about page performance are entirely contained in the web analytics KPIs – which are rich, detailed, specific to function, and carefully thought out and documented.

Matt does mention that a second portion of the scoring method is based on historical web analytics data. That gets pretty short shrift if you ask me. One sentence? And I’d be interested to see the KPIs that measure how well a page is performing its “psychological” function. But things are even worse than this, because the general groupings Matt suggests are far too broad to be useful. Categories like “sales” are inherently much too broad to be interesting. Part of the goal of any classification scheme has to be to implicitly segment pages so that their distance from the goals you’re measuring is relatively constant. Where this isn’t true, your measurement will give fantastically misleading results. All you’ll really be measuring is how close to conversion your pages actually lie in a visitor’s path. This is exactly the kind of measurement we’ve seen far too many times – a complicated scheme to produce bad results that everyone immediately ignores.

The bottom line is this – the system Matt is describing isn’t better than Functionalism, it isn’t Functionalism and it isn’t a precursor to Functionalism. It’s just a complicated system for scoring pages based on some unarticulated set of rules about design and navigation coupled with some basic web measurement statistics.

Now obviously, something like Functionalism doesn’t spring full-born into the world. There are plenty of intellectual predecessors. Nor do I believe that Functionalism is the be all and end all of web analytics. There are plenty of other good techniques – many of which we ourselves also use. I hope that readers of the Blog series on Functionalism will appreciated how much sweat equity and actual practice goes into how we do and talk about web analytics. I can un-blushingly claim that I think we are more honest, less salesy and more useful than the overwhelming majority of blogs and white papers out there. The proof of that, of course, is in the pudding – but I’m more than willing to let the whitepaper speak for itself on that account!

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

I was talking to a web measurement guy the other day when he casually mentioned that his business partners had given him 300 KPIs they wanted to track! This took my breath away. That’s a lot of KPIs. Now I don’t know if these were good, bad or indifferent or how sensible the thinking behind them actually was. But it does seem to me that businesses can easily be infected with a kind of measurement envy. I can image a conversation like this actually taking place – “You only track 300 KPIs – we track at least 400!”

This profligate KPI stew is the web analytic version of the multi-vitamin – including every conceivable vitamin and mineral to the nth degree and lacking nothing but the essential nutrients and energy to actually keep you alive!

What drives the creation of this sour concoction? The lack, of course, of any guiding principle about how and what is worth measuring. When you don’t know why your measuring things, almost any conceivable number might have a point. “Single Access Pages?” Throw em in. “Exits?” Gotta be important, right? “Average Page Time?” Doubtless a concern. “Top Referrers?” Somebody must need to know.

You can think of Functionalism as a recipe – and the ingredients are the KPIs you use at each step. If the recipe is a good one, then you should get a tasty result. If the recipe is just a mishmash of ill-considered ingredients, you’ll wind up with nothing but KPI stew.

So it’s time, at last, to move into the particulars of Functional analysis – the KPIs by page type. I’m going to tackle each page type in a separate entry, and I’m going to start with Engager pages.

Here’s the functional description for an Engager:

Functional Description: A page whose primary job is to grab the visitor’s interest and get them to do something (almost anything!) on the web site. In general, the pre-supposition to an Engager page is that there is little known about the intent of the visitors landing here. Where more is known about what a visitor desires, there is usually a specific set of directions that the page is expected to drive toward and the page is generally better (or also) classified as a Router.

I often look for elements of the traditional sales-cycle when I’m thinking about how to understand web pages. That’s why pages like “Closers” exist – they are a direct extrapolation of how in-person sales happen. Is there a non-virtual analog for the Engager? Well, in a way an Engager page is like a store-window. The store-window isn’t generally used to sell a specific product. It’s about bringing visitors into the store. So the focus in a store-window is on engagement – showing the most interesting products even if they aren’t particularly likely to sell. That’s why car dealers put flashy sports cars on a pedestal in front. It’s why store windows at Christmas have elaborate displays.

On the other hand, a web site doesn’t have a door that is distinct from its window. So a pure engager may have consequences on the web that it won’t in a store window. That’s why it almost always necessary to think about an Engager page as having Routing functionality as well. On the web, people enter your shop through your store windows – so you have to leave them a good path to tread!

So what are the KPIs for measuring an Engager page? They include the primary measures of basic success: % Engagement Links, % Return Visitors, Organic SE Entry %, Exit Rate and Exit Propensity plus some measures of route effectiveness: Subsequent Page Consumption and Subsequent Success.

The first measure simply tracks how many visitors click through into your content (become engaged). This is a simple but seriously important measure for Engagement.

Tracking returning visitors gives you a way to think about your psychological engagement. Naturally, this isn’t going to be a pure measure – since the other pages of your site are going to have a significant impact on return likelihood. One interesting technique is to measure the return propensity for lands on this page versus lands on others. This tends to capture both the page itself and it’s functional routes.

Organic SE Entry % is useful because Engagers these days have to grab search traffic as well as human. Few pages won’t include some measure of SEO effectiveness – and it’s essential to take this into account for high-level pages on your site.

Exits are, of course, the antithesis of engagement. So the basic Exit rate for an Engager is an important measure of its success. Exit Propensity (a measure of Exit Rate by Depth) helps you understand Engager performance when it isn’t a landing page. This does happen. In addition, by calculating the Exit Propensity for direct lands to an Engager, you can remove the confusion that often results from re-surface behaviors (cases where visitors drill-down to content then come back to the Engager page before exiting).

This set of KPIs comprise the basic measurement of Engagement Function. The next two KPIs are designed to capture high-level routing performance. Subsequent Page Consumption measures whether the paths visitors chose turned out to be deeply engaging. This is especially important for publishing sites.

Measures of Subsequent Success vary. They will always include conversion, of course. But they may also include reasonable conversion proxies – behaviors on the site that are useful for identifying visitor engagement and qualification. In many cases, the Functionalism Paradigm provides an excellent tool for understanding good conversion proxies. Instead of using total page consumption as an intermediate measure of success, you can focus on the page consumption of “Convincer” or “Closer” pages that actually indicate you moved a visitor into the real sales-cycle. That way, routes like “careers” and “press” and the subsequent page views they generate won’t be counted.

The fact that nearly every Engager page is necessarily a door as well as a window (exceptions might be a Flash that pops up off an HTML page and then returns the visitor to the page) means you need to look at more than one type of KPI. Engagement KPIs let you track how well your page is grabbing interest (the window). Routing KPIs let you track whether your entryway is properly directed (the door). Together, they provide an effective way to measure and compare Engager performance.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

You can read the first part here.

(You may want to visit http://www.semphonic.com/resources/whitepapers.asp and download the White Paper on Functionalism as a detailed technical companion piece to this series).

The idea behind Functionalism is simple and rooted in common everyday practice of most analysts – there are different types of web pages and they do somewhat different things. Since this is a truism, it is also something of a truism that pages designed to perform different functions will need to be measured differently. This, in a nutshell, is the guiding principle behind Functional analysis.

But it should be almost immediately apparent that to be a real method, Functionalism must be more than this simple principle. A principle – even a collection of principles – is not a method. There are certainly a very large – probably infinite – number of different conceptual schemes that could be evolved to describe what pages on a web site are designed to do. And for each of those schemes, there are certain to be a very large – probably infinite – number of possible measurements that could be evolved.

It is for the same reason that a simple idea like “web analytics must be driven by an understanding of customer goals” is not a methodology just another principle (and, probably, just as true and correct as the fundamental principle behind Functionalism). But there are an infinite number of ways to map customer segments ranging from simple to incredibly complex and from useful to useless.

So a method must provide direction about how, when and why specific measurements can be identified and should be used.

So the real heart of Functionalism is the conceptual scheme for categorizing pages (what they do) and the richness and practicality of the KPI’s attached to understanding the performance of each type of page.

A much more complete list of page types is given in the White Paper, but here are a few sample ones that provide some flavor to the conceptual approach:

Routers: Pages whose function is to move visitors into specific places on the site.

Explainers: Pages whose job is to help the visitor understand some aspect of a product or service but not necessarily drive to a sale.

Closers: Pages that are supposed to get visitors to enter a conversion process.

Re-Assurers: Pages built to re-assure the visitor about some potentially problematic issue or concern (privacy policies are a common example).

Completers: Thank-you pages – designed to signal the completion of a process and – in some cases – drive to additional engagement.

These pages are particularly applicable to a traditional e-commerce site – especially for a larger company. However, the Functionalist library of Page Types extends to many other types of pages and sites (like Billboard Pages on publishing sites). One of the strengths of the basic approach and the library as built is that it can address the needs of a significant number of sites and site cases – it isn’t limited to one type of site; even one type of site as widely used as basic e-commerce.

A second aspect that emerges from this list is that the methodology is designed to provide viable measurement systems for nearly every page on a site. Many analysts have grown accustomed to only measuring the success of pages that have a strong relationship to an easily established success metric (like conversion). This results in a kind of myopia where many site functions drop-off the optimization table – as if it isn’t possible to have better or worse Explainer pages just because they aren’t meant to sell a product. Even worse, it means that many pages that should be doing something more (like Completers) are left un-attended because they don’t obviously have a function in the sales process.

Part of our goal in specifying the Functionalist methodology was to build a system in which the analyst could reasonably measure the performance of almost any kind of page that marketers and designers believe is worth building.

In this respect, there is a fair amount of substantive page classification work that we’ve done but not included in the White Paper; though what is in the White Paper is more than adequate for many, many sites. This isn’t because we meant to be stingy – but many page types we’ve run across and thought about aren’t very common – and others we are still struggling to define a really good set of KPI’s for.

Which brings me to the second part of what I take to be the meat of Functionalism – the page type specific KPI’s. In thinking about the KPI’s for page types, we had several important rules that we wanted to follow. The first, of course, was the basic Functionalist Principle – we wanted KPI’s that measured in some respect the success of the page vs. specific functions. Second, we believed that the KPI had to be producible in at least one of the common measurement systems used by our clients (HBX, SiteCatalyst, Webtrends, Google Analytics) – and we gave preference to KPI’s that could be produced in all or several of these systems. Finally, we wanted KPI’s that could be understood in language relevant to marketers and web site designers.

In most cases, page types will have more than one KPI. That probably isn’t ideal, but it is inevitable. It’s very rare that a single measure can capture everything you need to know about a page’s performance. Let’s look at one set of KPI’s – the suggested measurements for Router Pages:

• % Body Routes
• % Routes by group (body, top, back)
• Exit Rate
• Exit Propensity
• % Re-surface (% of visitors who drill-down then come back up to the Router Page)
• %Re-surface Body Routes
• %Re-surface Routes by group
• %Re-surface Exits

In the White Paper, we typically show the most essential KPI first in bold – here, it’s the % of Body Routes. The White Paper provides a detailed description of what is meant by Body Routes – but for our purposes lets just consider Body Routes as the links within the main content page that the Router was designed to drive to.

So the essential Functional Measure of a Router Page is the % of links that it drives to its intended targets – hey, that’s why it’s called functionalism – the KPI’s are specifically built to measure the page’s real purpose! This is simple, easy to calculate and yet is a type of analysis that is frequently missed. The most novel aspect of the analysis is that it characterizes many links (such as other Top Nav or Home Page links) as “bad” (i.e. lying outside the function of the page).

The next two measures are just elaborations on the same theme. In addition to knowing how many visitors went in the intended direction, it’s useful to know how many went in three different un-intended directions (back, top – or lateral, and exit). Each of these outcomes is sub-optimal, but in most cases a top is better than a back and a back is better than an Exit. Again, these are all simple to calculate and form the basic measurement for this page type.

The Exit Propensity is just a technical elaboration on Exit Rate. It is slightly more elaborate to calculate (I’ll be providing KPI calculation details in coming blogs) but provides essentially the same information as Exits.

The next set of behaviors repeat the original set but focus on Re-Surface behaviors – cases where a visitor comes to a Router Page, drills down and then comes back up again. Why do we have these metrics? Quite simply because our real-world experience measuring this page type had been that some Router Type pages perform much better on Re-Surface than others and that there were personalization and ad serving strategies that could sometimes take advantage of re-surface behaviors. So the fact that these metrics are there is really a claim about the real-world of web site behavior – namely, that of all the kinds of things you might measure about a page like this, re-surface is among the more interesting for this type of page (we rarely look at re-surface rates for any other type of page).

As in some other cases, the secondary metrics are rather more complex to actually do than the primary metrics. However, re-surface metrics are obtainable in both SiteCatalyst and HBX (I’ve include an extract from my post on the WA Forum on calculating this metric) and meet, therefore, our basic requirements.

I think the Router Page KPI’s are a reasonable illustration of what emerges from Functionalism – a highly specified way to think about the success of almost every page type on your site. And while no methodology can or will ever replace sound thinking, common-sense and business knowledge, it’s our belief that even high-quality, experienced analysts can greatly leverage their work by applying many of the basic concepts in Functionalism.

Like most formal methods, Functionalism is simple in concept – almost surprisingly so. And, like most formal methods, it embodies a great deal of what many practitioners will consider normal everyday practice. This can easily lead to it being wrongly dismissed as “just a pretty gloss on what we already all know and do.” By supporting “what we all know and do” with a very formal, clear and well-articulated structure, it can – we believe – make “what we all do” quite a bit better!

Addendum

Extracted from the WA Forum – Calculating the Re-Surface Rates in HBX

You have to create a visit based active segment with the criteria being that the page in question was viewed once. Then you create an active segment (visit) with the criteria being that the page in question was viewed more than once. Now, you have a % re-surface by dividing the two visit counts from each segment. Even cooler, by subtracting the one-visit “next pages” counts from the all visits (no active segment)”next pages”, you get the difference between initial view next steps and subsequent view next steps. That’s a great way to find out if visitors use a page differently on re-surface.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

I’m going to start this series on Functional web analytics with a kind of intellectual history lesson. We’ve been doing web analytics consulting for about as long as anyone, and over the past (nearly ten now) years, our practice has embraced, built, used and abandoned a whole bunch of different tools, techniques and even comprehensive methods for doing web site analysis. Which is also to say that (at least until now) we’ve never found anything we could hang our hats on and say, with some confidence, this is “the” – or even “a” – right way to do web measurement.

By tracing through this slightly revisionist history, I hope to show why we think Functionalism addresses specific failures in the other techniques we’ve tried and how it emerged from our real-world attempts to do useful web measurement.

When we first started doing web measurement, we came from a world (data-based credit card marketing) where customer segmentation was king. And our first thought was to apply this same customer-based philosophy to the web. Unfortunately, our initial customers knew absolutely nothing about their visitors and had no way of linking visitors to customer or prospect information. Because of this, all of the bellwether demographics that are so useful in card marketing were just out the window. With nothing to work with, this approach came to a short and untimely end – but it was far from the last time we thought about or tried customer segmentation methods!

So we decided that good web analytics needed to be behavioral – and we started looking for tools that would illuminate web behaviors. Needless to say, the tools at our disposal were virtually worthless. So we began a two-year project to build our own toolkit. And the centerpiece of that effort was a detailed pathing tool. It seemed to us, as it has to so many others, that if you could just see and understand the paths visitors take on your site then you’d be able to powerfully tune your marketing.

We ended up with a pretty darn good pathing tool – one that was well ahead of its time and would still stack up reasonably if not favorably to the tools in HBX or SiteCatalyst today. But what we found was that path analysis was rarely useful. I remember discovering that one of our clients had something like 16 million unique paths in one month! And when we tried to make sense of paths, we found that top paths were generally un-interesting and obscure paths were too numerous to consolidate and understand in any reasonable manner. Web sites simply allow too much open-ended navigation to make visual path analysis useful.

However, one of things we learned doing path analysis was that a lot of behaviors were more interesting when you thought about them as occurring within a group of pages. Our path tool eventually encompassed pathing at content levels. But as we de-emphasized path analysis we began to think more and more about hierarchies – grouping related pages on a site together and then analyzing the users’ movement from group to group. Even better, we began to see that groups of pages often provided interesting statistics in their own right.

So our internal tools began to emphasize content hierarchies – and we spent a good chunk of time in our consulting engagements grouping pages into different logical structures and then looking at basic KPI’s like visitors, visits and page views at the content group level. At this point, we re-examined our previous thinking about visitor segments and came up with an idea that I’m still convinced was really good. We borrowed a bunch of our old neural network models for segmenting card customers and we modified them so that instead of taking demographic inputs they took behavioral cues – specifically, data about how often visitors viewed and visited site areas. From this, we build profiles of visitors that were far more interesting than our original attempts (enterprise software vendors take note – this is still a good idea!).

Alas, this approach foundered on three big rocks. First, though our statistics about content groups were interesting they were rarely particularly actionable. And while our behavior segments looked really interesting, similar objections applied. Cues from visitor behavior are often extraordinarily difficult to find in web actions – even with powerful neural network approaches. We tried to get sites to try dynamic content serving based on the segments, but this was asking a lot. In addition, the segments themselves didn’t often suggest any particular personalization strategy. We often found ourselves using the segmentations to justify personalization decisions we arrived at subjectively. Finally, and most importantly, our segments didn’t really incorporate any outcome data – so we began seriously investigating tracking to conversion.

It was about this time (perhaps three years ago) that we scrapped our internal tools (mostly) and began using tools from Enterprise vendors. As powerful as some of our tools were, they weren’t going to compete for ease-of-use, speed, ease-of-implementation, flexibility and features with the Enterprise packages that had begun to emerge.

Over the course of time, we looked at a number of web sites studying conversion data. And one of the things we began to realize was that a vast amount of conversion was ultimately multi-session. In one classic case, a client had asked us what visitors did right before buying (a very expensive) product. Turned out, by far the most common behavior was Land on Home Page and Click Buy. The reason? The average buyer had visited the site 11 times previously over nearly three months and consumed 150 pages on the site.

In short, everything was interesting except the behavior in the buying session! So we began to focus on a very sophisticated approach of tracking the correlation from groups of content to over-time conversion by visitor segment. This, in my opinion, is the general state-of-the-art for skilled practitioners in our profession. And a year ago, it was pretty much our standard method.

So what’s wrong with this method? There were a couple problems with it that became increasingly apparent the more we perfected our methods. First, there were many, many changes which, it turned out, couldn’t be measured with this method. Overtime conversion necessarily is impacted by many exogenous effects on a large web site. Nor is it realistic to suggest that a wording change on a page is going to have a statistically measurable impact on final site outcomes. But we were often supposed to be telling site designers whether such changes were good or bad. And, in cases like SEO optimizations, the consistent answer that the change wasn’t measurably negative began to seem like a foolish cop-out.

Second, we found that the method itself, while powerful at measuring good v. bad for a whole web site, provided precious little direction to designers and marketers about how to improve. The mere fact that thing x out-performs thing y doesn’t provide information about why or whether more thing x would also be good (or necessary). This lack of buy-in and direction for site designers and marketers increasingly began to seem like a fundamental problem with our approach. Too much web analytics was going to waste because the natural consumers didn’t know how to use or understand the results.

Third, the method was inappropriate in cases where pages weren’t directed toward conversion. For large web sites, this is a significant component of all pages – and a method that says nothing about their usefulness seems wrong. At first, our tendency was to believe that if pages didn’t drive to conversion they should just be eliminated. But it’s clear that this view is quite un-realistic for any large company.

For all of these reasons, we began to feel that this whole method, while reasonable and powerful in some respects, was also too lofty to be useful in many analytic situations. At the same time, we had begun to find that we could address each of these issues in specific ways by treating web pages (and content areas) as having a specific functional purpose. And we’d begun to build up a library of measurements (KPI’s) that were specific to those purposes.

When we explained these to channel marketers and site designers, they got it immediately. We could see the lights go on. “Ah – I get it. This page is supposed to move people here and it isn’t doing the job and I can even understand how you proved it isn’t doing the job.”

And the more we codified page types and KPI’s, the better the approach began to look to us – and the more different types of sites and pages we could fruitfully analyze – hence, the creation of the Functional methodology.

That’s about it. The whole sad, wandering history of how Functionalism was born, like many babies, in countless bad decisions, mindless passions and time ill-spent! I’ve left out many blind-alleys and I’ve streamlined history to try and make maximum sense of our experience – but I have the feeling that most web analysts have been down similar paths and made similar discoveries, even if they’ve only been at it for a year. With the tools available today, what took us years to build and discover can now be found out in weeks or months.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Gary Angel is the author of the “SEMAngel blog – Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.

President of SEMPhonic, Gary Angel gave me a sneak preview of this white paper and I was very fascinated by the new ideas presented before me.

KPIs at times, are way too high level and totally site oriented but by drilling down deeper by going to the page level will give Analysts that much more insight on how to make educated business decisions. What they present in the paper will really make good Analysts begin to use this approach in order to establish the importance of each page.

Tag:

Add to Del.icio.us | Digg | Yahoo! My Web | Furl

Bookmark WebProNews:

Manoj has been working in the search engine marketing industry since 2002. He started out as a software developer but now provides in-depth web site analysis using web analytics.

http://www.enquiro.com

Manoj is also the author of Web Analytics World. Web Analytics is an essential component in developing a successful
online campaign. Help convert visitors into customers by understanding
them.