Marketwave Web Analytics Software

A common question in web analytics, since most tools use a cookie to identify returning visitors. Some tools fall back to IP-address and browser agent, but that will not help either when multiple computers are used.

In some cases website visitors have to login which is a great alternative to increase the recognition of returning visitors. But what to do when you do not require users to login and you feel that they might use multiple computers more than average?

A real-world example from an online insurance website shows you one of the solutions. Based on the conversion per marketing channel the people responsible for online advertising were sure that a lot of visitors hit one of their banners or links before converting. During the order process people have to fill in a specific registration number for a car insurance (legal requirement, many people don?t know this number), which increases the chance that they abort the order process. By proving an email link the visitor can choose to receive an email with a link to resume the order process. Unfortunately it is quite likely that this will be done on a computer at home, while the order process started on a computer at work. No cookie, no similar IP and thus no way to match a conversion with a previous marketing campaign response. As a result the reported ROI on campaigns is too low, because many orders cannot be matched with a campaign.

The solution is to use an identifier which is available in both parts of the order process. For this car insurance it is the license plate number. Before visitors have to fill in the code (which they can only find on their car registration papers) they always fill in their license plate number which shows the premium for their car. The license plate number is matched with all marketing campaigns they have responsed to (Google Adwords, MSN banner, etc.). When they complete the order process the license plate will be stored in the web analytics tool on the order confirmation page. By combining this information with the list of license plate numbers and corresponding marketing campaigns almost 50% of the orders without any campaign information could be allocated to one or more campaigns.

Disclaimer:

Sometimes a little out-of-the-box thinking is required to get valuable insights out of your web analytics tool. Adversitement is a premier European partner of Omniture and is specialized in advanced web analytics projects.

Met vriendelijke groet,

Mirte Romanillos

Earlier this year, a previous post addressed using web analytics to optimize a paid search campaign.  This drew some attention, as web analytics (WA) are one of those things that everyone knows they need, but not always sure exactly what actions are called for in a given set of data. I suppose some of this can be blamed on the usability of the dashes as it?s pretty easy for them to get unwieldy?which can lead to a full blown case of "analysis paralysis." 

It?s time to share some direction on using WA to optimize natural search campaigns. The following reports can be found in almost every web analytics package and are fairly consistent, but depending on campaign goals, the insights and actions taken may vary:

1. Top Referring keywords. Look at the long tail here; 25% of the queries performed are ones Google has never seen. The referring keywords to the site will show themes and patterns that are otherwise difficult to discern.  More than once we've changed directions on a campaign because we're seeing volume around a type of query we've never seen.  Many times the impetus for a decision like that is born here.
2. Referring Domains. This metric is essentially telling you who your biggest traffic drivers are.  Search engines and sister sites tend to occupy the top of the list.  However, we see some interesting things happening in the middle to end of this referral list.  This is a great way to: A. Measure the effectiveness of a link building campaign (if we're seeing visitors from sites we built linking relationships with that we haven't before that's a good thing right?) and B. Find site themes and verticals that you may be able to generate buzz with (e.g. if we've got a particular blogger reviewing a product or service, this can inform the type of content on the page which could lead to greater link bait).
3. Click paths. This metric is more or less telling you how people navigate through the site.  There are a lot of things to be determined here but a big one is the effectiveness of your site layout.  If we're seeing a lot of people having to go through a few clicks and all ending up on the same page, it would indicate that there's an opportunity to engage people more effectively by improving that click path.
4. Paid vs. Natural. This is an excellent metric for identifying gaps.  With paid search, we can quickly target the high volume terms and use ROI/conversion rate data to inform our decisions on where we want to compete organically.  Likewise, the places where we're seeing a lot of activity in organic terms where we don't have paid coverage can help us expand a paid campaign.
5. Geographic referrals. The more targeted and niche the web becomes, the more important geography is (and no, the irony of this isn?t lost on me).  Nevertheless, we've had instances where a flurry of offline promotions leads to a surge in a particular geo-specific market.  Certainly the offline team will want to know that the radio blast in Philly led to an $X lift in revenue. We'll use geo data to develop new content, launch targeted landing pages, and in some cases, even modify service offerings to better target geo revenue sources.
6. Visiting trends. Almost every analytics package puts this metric on the forefront of their dashboard so you can see how many visitors you have this month verses last month and so on.  Resolution Media uses this to evaluate seasonality and optimize accordingly. Correlations between seasonality and referring keywords are also helpful in determine where linking opportunities could be. 
7. Top Landing Pages. Lots of useful action items come from here.  This metric essentially tells you where your buzz is.  Keep these pages fresh and make sure your users can access them easily.  Seeing where human visitors land is a good indicator for what spiders are crawling in on as well.  From there, this data can be used to optimize the structure and internal linking scheme of the site.   For example, if we note that Page A is a much more popular landing page than Page B, C or D, we should make sure B, C and D are linking to A with optimal anchor text (based on what themes and keywords are on page A).   
8. Conversion Rates. This is the million dollar metric right here!  When visitors come to the site, are they doing what we want them to ultimately do?  How often?  More than they were? Conversions are what answer these questions.  From there, we may have a number of action items we need to take based on what the data is telling us.   Don?t take brash actions if conversions suddenly drop (or spike).  However, KNOWING when those spikes or drops occur, and looking at what other things happened around it (see almost any other metric listed here) as soon as possible is absolutely essential to taking the right actions.  It could lead to campaign spending changes, landing page optimization, re-targeting keywords, building new link bait and a host of other scenarios.
9. Bounce Rate. This is one of those metrics that I think varies quite a bit from project to project.  One bounce rate may be great for one kind of site and a total failure for another.  It's essentially telling you how many people happened upon a single page on your site and didn't bother to go elsewhere.  We generally chalk that up to them not finding the information they were looking for.  If we've got pages targeted specifically around one or two keywords, we may be looking for a lower bounce rate than a page that casts a wider net.  Measuring bounce rate according to page type is essential to evaluating the effectiveness of our content and the messaging. 
10. Browser type. It's weird how this was a metric that fell out of favor for awhile and is starting to make a comeback.  I'm talking about mobile here people!  If we've got a project that is a multi-media extravaganza with elements that aren't visible to a spider or a mobile browser, and our browser type metrics are telling us a significant amount of traffic comes to us from this type of user, then this should absolutely impact how we present that information. 

There are a host of additional web analytics reports that lead to the optimization of a natural search campaign.  Reading back over the list, much emphasis was put on site usability, which supports that driving traffic to the website is half the battle, and having a website that drives the traffic to take a desired action can be just as important.

Many times, statistics from web analysis can be misleading. It is all too easy to end up doing the wrong thing based on analysis of website statistic. Here are some pitfalls for the following metrics:

1. Number of Leads by Keyword

   It is very important to measure conversions by keyword. This lets you know where to focus your marketing efforts. However, many B2B sites and especially those with high cost items have a relatively small number of conversions. In addition, the conversion may occur:

   • During the 2nd, or later visits when the original keyword used is now lost (due to cookie erasing, subsequent search using product name, etc)
   • By a coworker of the original searcher who lands directly on site since the URL is now known

   Since the numbers are small and may not be traceable to the original search, in many cases it is statistically invalid to decide on actionable items based on the number of leads by keyword. In those cases it is best to find proxies for conversions. Possible proxy candidates are time-on-site or engaging actions.

2. Percentage of Leads

   The percentage of leads from total visitors or any other segment is not always relevant. In many cases the profit from a B2B sale is big enough so that a good lead can justify the cost of a campaign even though the percentage of leads is small. The absolute number of leads is more important in this case. If you see that the percentage of leads from traffic is going down but the absolute number is going up--you can should go out and celebrate your success.

3. Absolute Number of Leads

   Measuring the absolute number of leads can mislead you. Yes, I know I just contradicted the previous point. Unfortunately you may sometimes notice the number of leads is decreasing. Before you panic, it is useful to measure the percentage of conversions. This is because there are many times when seasonal or other time-based factors impact total traffic.

   If we just measure absolute numbers and we see a 50% drop in leads we start to panic. But if you see that the percentage of leads is consistent over the last few months you know that the reduction in number of leads is due to a drop in traffic.

   You should then analyze to see if the reason is seasonal or other factors we have no control over. If it is seasonal, we can relax--although we should still try and improve the percentages. If it is a factor we do have control over, we can then start to panic and work to rectify the situation.

   It is useful to measure percentage of conversions to use as an early warning sign, however our main goal should be to increase absolute conversions until the expense of increasing them outweighs the profit.

4. ROI

   This important metric has great PR but it is undeserved. As long as you are profiting from a campaign, the return on investment should not be use to eliminate ad campaigns. You can use it:

   • If you need to reduce your advertising budget this metric then becomes necessary in order to guide you to which areas it is best to reduce the budget.
   • To measure your optimization efforts

   If you are selling on your web site, make sure to measure the revenue for each conversion and not just the number of leads. Not all conversions are created equal.

5. Number of Downloads

   Not all downloads are created equal. Whitepapers are typically downloaded earlier in the sales cycle. In addition, you may get many non-qualified people downloading the white paper who are interested in the subject.

   Data sheets, on the other hand are usually downloaded by people later in the sales process and who want to see detailed specifications of your product. By measuring downloads you are lumping these and other different segments together. Best to measure whitepaper downloads separately from data sheet downloads.

   An upsurge in white paper downloads in October may be because students are studying the subject described in your white paper. On the other hand, an upsurge in data sheet downloads is usually great news--unless you find out that it is your competitors doing all the downloading.

6. Using Numbers that are Statistically Valid

   In many cases metrics do not have enough information to be statistically valid. Unfortunately there is a tendency to want to come to conclusions fast. This could be because:

   • You want to prove something and are over eager to bring the testing (with the results you wanted) to a conclusion
   • There is pressure to present actionable items to others

   Avoid the pressure. I have seen many tests where the results flip flop once or twice before the numbers are valid.

7. Experience and Knowledge.

   Numbers are great and no one loves them more than me. However, they are just numbers and have many disadvantages:

   • There is still a lot of information they don't include. For example they don't explain why people do things
   • There may be mistakes in the data
   • The conclusions may not make sense and by being stubborn and digging deeper you can usually find the reason
   • In addition, they can be manipulated to prove preconceived ideas-sometimes uncounsiously. As my seventh grade math teacher said: Figures don't lie, but liars figure

These are some of the web analytics pitfalls and mistakes we have come across. I am sure there are many more. If you have any, I would love to hear from you.

Continuing our Things I Wish I Knew... series, Daniel Shields offers his Top 10:

1.  Not Everyone Knows What You are Saying

The ecommerce workplace is made up of very diverse areas of expertise. Where a few folks here and there might get what you are saying with little effort, most of the people whom you work with are not steeped in statistical understanding, much less its vernacular. For the good of the company, and the sanity of the analyst, it might be advisable to find an in-house reader who gets most of it, but can point out where language might be an issue.

2.  Customer Support Exists for a Reason

A major impulse to try to overcome is the same which keeps a person from asking for directions in Culpepper, Virginia. You might know you are lost, and you have a pretty good idea that people know how to help you, but for some reason, it means more if you turn the map around and fold it different ways a dozen times to see if it makes a difference. Taking this ego-shot and allowing it to remain an open window for progress is a key item to reaping greater rewards from vendor solutions and the myriad applications which complement them. Shave a couple hours off your time and simply ask for directions.

3.  Experts Have Expertise

Early about in my freshman year of analytics, I made assumptions about the vast cache of experts in the field of 'web analytics'. Mistakenly, some of those were that each was a virtual encyclopedia of best practices and skill in every facet of the world of online measurement. Luckily, with the guidance of some gracious consultants and my boss, I was able to understand the landscape in terms of where each expert rooted their strengths. Knowing that made each a much more valuable source of information in terms of topics which applied to specific parts of my obligatory reporting.

4.  Management Needs Analytics

In order to make critical decisions in an online retailer, it is an absolute necessity to understand the ways in which numbers impact functions of the business. Managers make decisions on a daily basis which require output from measurement. This is as true for marketing as IT, Sales, Advertising, or Development. Making reports available to managers to align business goals is a major step in driving a web analytics process into a company. Getting that point across to management and administration will facilitate resource allocation as well as recurring discussion on points of impact.

5.  Analytics Needs Management

Much like respect, analytics is a two way street. It is clearly an input/output function. There needs to be 'Follow-Up' and 'Feedback'. An analyst, or internal owner of data, needs to see the results in order to balance improvement across the goals and initiatives which are helping the business grow. Not getting the discussion to start around web analytics data internally is, in my estimation, probably the most detrimental to installation of useful analysis.

6.  Prepare to Stand Your Ground

The first lesson of quantitative analysis is that numbers are not emotional. Math exists outside of the human psyche. Therefore, it should not have representation or meaning attached to it outside of the definitions of the operations. This part of the memo is not extended to the rest of the people whom analytics affects. People, their work, the graphics which make up a site, and the world in general cannot extract emotion from their decisions. The lesson here is to know well in advance that if you live by the numbers, prepare to survive by the numbers.

7.  Who Knew Experimentation Could Be So Much Fun

I guess I get a little excited when it comes to building and executing complex experiments. For me, stepping into the world of multivariate testing and usability was almost a fulfillment of a vocation. When I was interviewed for my position, Mildred the GM at CableOrganizer.com, explained that the job consisted of tedious math and required certain stepping stones to maturity. Learning to perform on page A/B and multivariate experiments was a minimum. With some training provided by the WAA's own Robbin Steif, I found myself neck deep in possibilities. It was a mix between graphic design, marketing, and science which appealed to every part of my thirst for fulfillment.

8.  Analytics Needs Regular, Consistent, and Objective Validation

Take nothing for granted. At every turn, little pieces of HTML code and applications being worked on or edited by some party have potential impacts on the data which we hold so dear for decisions. These should be perceived as threats and guarded against accordingly. There were three occasions where our major analytics vendor code was completely eliminated from important pieces of our site template in 2007. Each resulted in loss of data. In other instances, data was artificially inflated in terms of traffic due to scripts and crawls which were being run for one purpose or another; again, data rendered problematic until the good stats were boiled out. While every failure was not attributed to analytics, its ultimately the confidence of the data which suffered. The lesson: keep up with your data and know when things look funny...then, explain away the happenings.

9.  Live and Die By Measurement

As an analyst, eventually there is an epiphany that a world of measurement is easy to manage as a vertex of both tasks and value over time. By taking on certain actions or initiatives based on analysis, you champion causes for improvement. When the figures from the work come back in a statistically relevant state, some ought to clearly be wearing your signature. Be they good or bad, they are results associated with changes based on your suggestions. So, over time, your suggestions gain merit based on your experience, and the returns, in earnest, should be appropriately attributed and the rewards shared. A good employer will recognize your contribution and construct a system whereby your achievements will be based on your skill, ambition, experience and initiative in helping the company achieve its goals. When those goals converge is the sweet spot for income, bonuses, and benefits.

10.  Follow Your Data: Analysis is Not the End of Analytics

Being able to produce reports and materials to place monitors on is wonderful, but making sure the data is being used for the right purposes is a job beyond the analysis. When work in analytics began for me, I had scheduled deadlines on which to deliver my take on the previous week's performance versus some comparable information. Over time, I started to realize things were slow to change based on the insights which I extracted and outlined in my papers. After a couple weeks I started basically shouting out about certain points trying to drive them home, until I began to feel like it was becoming useless to produce reports. By the end of the 3rd quarter, I was approached by Paul, the VP and COO from CableOrganizer, who noticed my cries, to move my actions from only reporting into managing major facets of resources to get the jobs done based on the analysis I was providing. With that, the growth started to produce value which was much more desired than mounting paper columns.

Daniel Shields is a talented young web analyst working for CableOrganizer.com, Inc. and focusing his efforts on conducting highest-quality research on web visitor behavior.

Daniel and the team at CableOrganizer.com are also developing Wicked Business Sciences, which has created patent-pending ecommerce technologies geared toward business improvement through measurement and personalization.

We?ve all heard the classic conundrum of businesses today ? ?Half of our Marketing efforts are working, we just don?t know which half?. Site Analytics strives to answer that statement by gathering data to allow business to align their message, media, offer and channel with their desired audience to optimize the results and return for the company.

However, in online business today, the conundrum has gotten much more complicated. Where does the analytics of the website end and analysis of the customer begin? While we construct our KPIs and dashboards to capture the successes of our campaigns, ad placement, engagement and conversions to measure the success of our company objectives, in the end isn?t the end goal really about the customer and the measurement their experiences and successes (or lack of them)?

Spending Growth on Customer Experience

In a recently published report ?Customer Experience Spending Intensifies in 2008, by Megan Burns?, Forrester Research identified areas where North American business leaders are looking to focus their attention for 2008. In comparison to the previous year Forrester found some significant spending and planning increases, notably:

• 84% total increase in efforts to improve online usability
• 78% total increase to improve cross-channel interactions
• 80% total increase to make online interactions more enjoyable

While the research points to the continued growth of the analytics industry and covers increasing size of the customer experience budgets, another trend identified by Forrester drives the end focus of all the measurement back to the customer and their interpretation of success.

?We expect these trends to lead to more customer-centric cultures and processes by enabling firms to be more disciplined in their approach to customer experience??

Why focus measurement on the Customer?s Experience?

The Internet channel is becoming a (if not the) primary channel for most businesses today and Ecommerce continues to grow with online growth rates significantly outpacing the offline growth rates in most industries. So why is Customer Experience a top spending focus for 2008 and why did almost all (91%) of executives surveyed in the Forrester research report say customer experience will be either very important or critical to their 2008 efforts?

Simple really? your customers are trying to complete their transactions, your site is letting them down, and they are switching to a competitor. And by the way, the results are getting worse instead of better.

Harris Interactive conducted the third annual survey of online consumer behavior, sponsored by Tealeaf®, and the findings were alarming:

• 9 out of 10 consumers experienced an issue that caused them to abandon a transaction
• 53% of users who experience website issues contact the company?s call/contact center to resolve but 49% of users who contact a company after experiencing a web-related issue were still unable to have that issue resolved.

The result of this poor customer experience translated into two immediate waves of online abandonment:

• 42% of users say they abandon or switch to a competitor when they experience even one online site issue
• 52% say they stop doing business with the company entirely, and 76% either stopped doing business entirely, decrease the amount of business they do, or lodged a complaint with the Better Business Bureau.

The third wave is even more risky. This is the threat to your brand loyalty and long-term customer value as a result of the poor customer experience and inability to complete an online transaction. With continued growth in blogging, social networks and viral content, your site failures are broadcast to an increasingly receptive audience actively seeking unmitigated third party reviews.

Indicators of a poor customer experiences are not limited the obvious but still prevalent today: site errors messages, performance issues and broken links ? but include functional and business process challenges, and issues centered on usability and site-design. Collectively, however, they all have one commonality? they all forced the consumer to abandon the transaction.

For the third year running, nearly 90% of users responded that they had experienced an issue that caused them to abandon a transaction. This rate of ?failure? is extremely high and is not improving?in fact, it?s actually getting worse. Considering there are significantly more users and transactions every year, with a consistent rate of failure, the number of individuals and transactions adversely affected by issues each year is actually increasing.

The first threat (the ?first wave? of abandonment) is very real, with 42% of users saying they abandon or switch when they experience even one issue. These users have little tolerance for failure today and that tolerance will only continue to decrease until leading ebusinesses focus their attention and budgets on improving the customer experience on the website and in the online support centers. Consumers expect the online channel to work as well as offline channels such as storefronts, branches, catalogs and agents with 82% saying they expected the online experience to be the same as the offline.

The second threat (the ?second wave? of abandonment) is significant and is newly identified by the 2007 Harris Survey. The heightened rate of churn for online customers ? with 52% saying they stop doing business with the company entirely and a full 76% who either stopped doing business entirely, decreased the amount of business they do with the company, or lodged a complaint with the Better Business Bureau is a serious threat to online businesses that demands that call/contact centers be equipped to handle the needs of online consumers, or risk losing them to competition permanently, since the tolerance for poor customer service after web-related issues is extremely low.

The third threat is just as challenging, and perhaps even more risky. This is the threat to your brand loyalty. One interesting example the survey identified is that the single most important factor to consumers in doing business online was website security. However, the survey also found site issues to seriously undermine consumer confidence, specifically relative to online security and privacy concerns.

While businesses today tighten belts and budgets for tough times ahead, Forrester expects metrics and executive attention to the customer?s experience to rise to top of mind, and budget. Forrester addressed these identified gaps in the true understanding of the customer.

?Most firms today struggle to measure the quality of their customer experience. To establish a framework for measuring customer experience quality, firms should identify key customers, the most important moments of truth in the customer experience continuum, the criteria customers use to evaluate those critical interactions, and metrics ? both subjective and objective ? that capture how well the organization met customer expectations in each area.?

Effective understanding of the customer does not stop with well designed and built site analytics dashboards and Key Performance Indicator reports; that is just the beginning of visibility into the complete online experience and your customer?s behavior and experiences.

It would be incredulous to believe an online business today could operate without detailed website analytics to gather the bits and bytes to measure the website?s effectiveness, In light of that:

• Why is there a gap in extenuating site analysis into the success of the customer?
• Why are customers continuing to experience issues completing online transactions?
• Why are online conversion rates stagnant across so many industries?

Web Analytics alone can?t provide all necessary measurement and optimization data to improve the customer?s experience.

Eric Peterson, from WebAnalytics Demystifed Inc. discussed these thoughts in a recent paper ?Customer Experience Management and Web Analytics, From KPIs to Customer Transactions? covering the foundational needs to combine multiple measurement disciplines for e-businesses today to understand true user behavior and allow companies to improve their customer? experience.

?In today?s e-business environment, both Web Analytics and Customer Experience Management systems together should be considered foundational to website measurement and optimization. These similar-yet-distinct systems each contribute to a site owner?s ability to recognize, react, and respond to the ongoing challenges they face. Used together, these two technologies are collectively able to resolve the ?What, Where, When, and ?Why? of visitor interactions on the Internet.

The most forward-thinking companies have already recognized the value of investing in solutions beyond Web Analytics in order to measure and optimize their web channel. By understanding the true strengths and weaknesses of Web Analytics products and how Customer Experience Management systems can best be leveraged, web site owners will be able to extend their web measurement and optimization processes to achieve far greater levels of success ? ultimately by improving the site, serving customers better, and increasing site revenue.?

Measure the Customer not the Website

The strongest companies today are taking action to align internal processes and measurement with cross-channel experiences like the website and call-center teams to improve measurement beyond the browser and making their customer experience their top-priority. Not surprisingly this translates to firms that put customer experience on the radar screen at the executive level are best positioned to improve the success of their customer?s experience across the entire enterprise.

While it?s debated today in site forums, user groups and blogs whether ?Web Analytics is Hard? or ?Web Analytics is Easy?, the recognized need to improving the customer?s experience is universally accepted by analysts, practitioners, experts, pundits and of course by the real end-focus of all these efforts, the customers themselves.

Learn more about how you can leverage Site Analytics using Customer Experience Management
Download all referenced reports from this article along with a Tealeaf customer case study

References

Forrester Research, an Independent Research firm in February 2008, ?Customer Experience Spending Intensifies In 2008? by Megan Burns with Harley Manning, Olga Melnikova, and Steven Geller.

The Two Waves of Online Abandonment: The 2007 Harris Interactive Survey of Online Customer Behavior, Sponsored by Tealeaf®

Forrester recently surveyed 287 customer experience decision-makers from large US firms about their 2008 plans. Almost all ? 91% ? said customer experience will be either very important or critical to their 2008 efforts. See the February 7, 2008, ?Obstacles To Customer Experience Success, 2008? report.

The Two Waves of Online Abandonment: The 2007 Harris Interactive Survey of Online Customer Behavior, Sponsored by Tealeaf®

Eric Peterson, WebAnalytics Demystifed Inc, 2007 ?CUSTOMER EXPERIENCE MANAGEMENT AND WEB ANALYTICS, From KPIs to Customer Transactions?

Continuing our Things I Wish I Knew... series, Alex Cohen offers his Top 10:

1. You are Not the First Web Analyst - You do not need to invent web analytics. Somebody has encountered the problem you have. Establish a great base of knowledge by buying books like Web Analytics: An Hour A Day, joining the Yahoo Web Analytics Forum and subscribing to every measurement blog you can find.
2. Go to Emetrics NOW - Your world view is likely to be very myopic: all about your tool, your website, your business. You need perspective. The eMetrics Marketing Optimization Summit will open your eyes, especially if you're just starting.
3. Your Tool Can Do More Than You Think - Most people assume that what you get out of the box is the limit of your tool. This is usually wrong 99% of the time. You must not be afraid to ask your vendor about what else it can do.
4. Start a Blog or Business - If you don't really, really own the numbers you're responsible for, you'll never really, really learn the data. Pick some side project, a blog or a business, and measure the hell out of it. Trust me, you will learn a ton.
5. Automate Your Life - I'm repeating June here, but you simply must automate as much as possible. You will be stuck in Excel hell unless you can use technology better.
6. Test! Survey! - Repeat after me: not everything you need to know is inside of your conventional web analytics tool. Say it again. Now, do it. There is NO excuse not to start gaining experience. If you listened to #4, then you don't need anyone's permission.
7. Learn Other Disciplines (like SEO and Paid Search) - You will be better at your job if you understand what you're measuring. Start dabbling in paid search, SEO, affiliates, email, WHATEVER. Just stop focusing on measuring and start focusing on doing the things you measure.
8. Communication is the #1 Skill You Need - Measurement without action is failure. If you cannot communicate your findings and persuade people to act, you will not be effective. Learn to present. Master the executive summary. Be one with PowerPoint.
9. Be Not Afraid of Technological Terms - I'm not a technically oriented person. But, the very nature of internet marketing requires that you at least grasp the basics. The nature of web measurement requires that you grasp a step above the basics. Like it or not, you need to tackle this sooner rather than later.
10. Teach Early and Often - It is very easy for people to start relying on you to measure. Unfortunately, this can quickly become limiting to your career growth. Measure for manager and he'll optimize for a day, teach him to measure and he'll optimize for life!

Alex Cohen writes Digital Alex, a marketing strategy blog. He handles Strategic Account Management, web analytics and multivariate testing at Commerce360, a search marketing software company based outside of Philadelphia.

When analyzing online advertising begin by looking at impressions. Delivered impressions tell you how many people were exposed to your ad. Impressions are passive. When an online ad is served up and someone views it, it is counted as an impression.

The purpose of someone seeing your ad is to entice them to visit your website. In order to visit your website, a customer interaction such as a click is needed. The click sends a customer to a company website. Once a customer arrives at the designated website - that is considered delivered traffic. Delivered traffic can be defined as user clicks and website visits after exposure to an ad.

How is your ad performing? Is the ad getting customers to visit your site? A quick metric to use is delivered traffic rate (DTR). DTR is total delivered traffic divided by impressions served. If your DTR is 2% or more you have efficiently accomplished your goal of getting people to your website. A two percent DTR indicates the robustness of your ad, and is a general rule of thumb. I don't think there are case studies on DTR yet.

Once customers reach your site what are they doing? Good metrics to measure that are actions and leads. An action is a way a customer can express interest in a product or service on your website. A lead is usually a customer whose interest in a product or service is expressed by electronically submitting personal contact information.

The unofficial relationship between actions and leads is 6:1. In other words every six actions should net you one lead.

Action rate and lead rate determine how well customers are interacting with a website. Action rate is defined as total actions divided by total delivered traffic rate (DTR). The action rate speaks to the robustness of a site. The higher your action rate, the more people interact with your site.

The lead rate can be defined as leads divided by delivered traffic rate (DTR). The lead rate allows you to see the relationship between leads and delivered traffic. It is a funnel effect. A customer can not submit a lead unless they are at a website. A customer arrives at a website as delivered traffic.

The final metrics to look at are cost per action, and cost per lead. Cost per action is total cost divided by total actions. Cost per lead is total cost divided by total leads. These metrics let you know how much each action and lead cost. CPA and CPL vary depending on industry.

By analyzing metrics such as impressions, delivered traffic rate, action rate, and lead rate you can determine how well your advertising had done.

Successful insights and optimizations from Web Analytics (WA) data hinges on analyzing timely, consistent, and accurate/clean data; all online advertising should be tagged with WA URL tracking parameters to provide a complete and centralized view of site activity and conversions. Now that disclaimer is out of the way, here are 10 quick and simple ways to use WA data to optimize Paid Search (PPC) Campaigns:

1. Traffic Volume & Conversion Rate Comparisons ? Comparing performance between Paid and Natural traffic, paid engine to paid engine, and Paid keyword to Natural keyword offers a great opportunity to set goals and find new and missed opportunities. If keyword ABC has an 11% conversion rate from Google Natural, but 2% on Google Adwords (paid), a PPC optimizer should be making comparisons between landing pages, copy, etc. to make drastic changes versus minimal ?tweaking?. Without the WA/Natural performance data to benchmark against, an optimizer would have been content (and quite happy) with doubling the conversion rate to 4%.
2. New Keyword Opportunities ? Looking at keyword level performance data from both the natural search engine reports and the internal search box queries is important to do monthly to find new paid search keyword opportunities. Look for new misspellings, high volume converters, or run a vlookup to identify missing paid search keywords. Add new keywords, increase appropriate paid ranks/bids where gaps exist, and focus budget on top converters to find instant gains.
3. New Traffic Source Opportunities ? Domain referral reports show where visitors are coming from the associated volume and conversion rates. Look deeper than the big search engines, what do the converting sites have in common? Is there a way to increase exposure to this audience with Display ads or Contextual Ads? Do you notice a lot of mobile traffic converting? Do you see an increased average order size from non-US visitors? Is it time to think about getting budget for PPC mobile ads or international search engines?
4. Budget Forecasting ? Flat month-to-month PPC budgets seem to be the standard, yet I have never seen a single business where conversion rates or volume doesn?t change month-to-month. Seasonality is evident in WA data and monthly ad budgets should reflect those ebbs and flows, down to the engine and campaign levels. Just because certain search engines get more press doesn?t mean they should get more budget, unless conversion rates dictate it.
5. Segmenting User Behavior ? Using click path tools, do you notice similarities in Ask.com paid traffic that is different from Yahoo? What about people who search on brand keywords? Visitors who have been to the site previously? You bet ? there is no average user, why give them a single website experience? Personalize user experience based on what you already know?or are seeing unfold that visit.
6. Identifying ?Influencer Keywords? ? A 2005 DoubleClick study showed consumers search anywhere from 2.5 to 6 times before making a purchase and refine those searches from generic keywords to brand or product names prior to conversion. When judging the success of your generic keywords like ?mortgage?, ?computer?, ?car rental?, etc., it?s important to use WA data to find out if they do indeed bring converting users, even if it?s for that initial introduction to the site.
7. Identifying Conversion Latency ? As previously mentioned, not everyone converts immediately. What is your same-session conversion rate? How many days pass before a visitor returns to convert? How many sessions does it take before conversion? Page views? Time on site? Don?t judge PPC success until the latency period has passed. Know your audience?s sales cycle and build that into campaign expectations and testing windows.
8. Finding Day-Part Opportunities ? Sundays and Mondays will vary in conversion volume and rates, as will 9am and 9pm or MSN to Google. Turn your PPC spend dials up and down to maximize ad exposure; keep in mind ?AdScores? penalize excessive bid/budget changes. We?ll save predictive modeling for a future post.
9. Hot Products, Top Converters ? With a limited budget and possibly thousands of Product URLs, which product pages get the attention (detailed keyword lists, customized copy, page-level optimizations, budget share)? WA tools typically have a product analysis/comparison report to view quantity of orders, product page visits, revenue/order size, profit margin, conversion rates and more, by product. For most e-commerce companies, the 80/20 rule holds true; see what moves the needle (or doesn?t) with WA data and focus your limited resources where appropriate.
10. Landing Page Analysis/Bounce Rates ? There are typically multiple landing page choices for your PPC destination URLs ? home pages, category, sub-category, product pages, geo, seasonal promo pages, and more. Bounce rates and entry to conversion stats are immediate indicators of what is or isn?t working. Use A/B or multivariate tests to improve both page ?stickyness? and conversion rates.

Having WA access is major advantage for Paid Search optimizers ? use it if you got it and keep in mind that these 10 tips are only the tip of the iceberg. The real challenge to using WA data to optimize our campaigns is gaining access. I?ll save that diatribe for another article, but a quick quote from Andy Beal sums it up well: ?Too many clients guard their web data as if it?s the secret recipe to KFC. For us search marketers, it?s like navigating a plane through a thunderstorm without instruments.?

"The Web Analytics Report 2008" is a long report. No, it is a "very long report," writes its principle analyst, Phil Kemelor, who is vice president of strategic consulting for Semphonic, in the report's opening words. "That is intentional." It is 343 pages of user guide, thorough backgrounders on web analytics applications, technology and acquisition -- and the meat of the report, in-depth reviews of 15 web analytics products and services. Long reports are bad when they are poorly organized. This is not a bad report. Nobody should have to read the whole thing, but that is only because nobody should have to read everything about every product.

There is no perfect technology, so good reviewers, particularly in emerging markets, must go beyond capabilities and reveal what's missing, what doesn't work and how to avoid known mistakes. It is thrilling to know what one could do, but it is profitable also to know what not to do. This report has no shortage of reasonable skepticism and warnings. The healthy doubt shows up early, in an introductory section about standards. Referring to the WAA's standards, the authors observe, "It will be interesting to see whether consensus can be built around these standards." Indeed.

Disappointingly however, the authors seem to have made no effort to discern vendors' current or planned compliance with the WAA standards [PDF], leaving that homework to the reader: "While we can expect some vendors to tout that they are compliant with WAA standards and perhaps even change some existing reporting terms, you may end up with more investigation work to validate these claims." This seems to say that standards are important, but not important enough to matter when choosing among vendors and products. Or perhaps it was just too difficult a task to include in this sort of report -- but it would have been good to know why the authors decided to leave the further investigation to us. It seems as though they didn't even ask the vendors the obvious question -- are you now or do you plan to follow the WAA standards? Given the industry's frustration with comparing analytics numbers between systems, greater attention to this issue would have raised the report's value considerably. Perhaps that will be the subject of the next report.

The report does a good job of identifying the industry's dirty little secrets that make some metrics -- unique visitors comes to mind immediately -- less than solid. A "data Accuracy" section describes the strengths and weaknesses of data capture technologies. Like every honest practitioner, the report acknowledges that many numbers are best used to identify trends, not absolute quantities. Caveat emptor. Or perhaps caveat counters, since no product or service is immune to the vagueness of certain Internet technologies.

Part of choosing a web analytics solution is to understand the business process it fits into. The report does a fine job of walking the reader from having no analytics to evaluating, choosing and implementing the solution. This is not merely a technical description, but a description of how analytics can meet management objectives, relate to or be part of organizational units (marketing? IT? finance?) and how it can interact with the people who build and maintain the web site.

In an emerging market, web analytics constantly faces new challenges as new web technologies are developed. The authors identify blogs, Flash, RSS, user-generated content and the use of qualitative analysis (surveys and such) as areas where vendors are racing to keep up with how data is delivered and gathered. At the same time, they propose that we are on the cusp of a fourth generation of web analytics (where do the years go?), in which "analytics tools begin to mature into the 'brains' behind website marketing and content generation." If true, this is not entirely good news except for those of us who secretly long to become the center of attention. The same data we generate for reporting can and will be used for personalization, but since when was it a good idea to use the same system for production and reporting? Am I really going to tie data from Omniture, etc., back into my production system? Probably not. The good news is that this sort of bull... hyperbole only shows up here. The other 342 pages are practical and down-to-earth.

The report's greatest strength is its scenario-based comparison of web analytics solutions. The rest could have been written by any reasonably bright person from information found on the web. The comparisons clearly came from in-depth discussions with actual users of each of the products and services. Each offering is rated many ways, based on types and purposes of the site being measured, who will manage and use the system and its reports, what kind of offerings -- software, services, methods -- the vendor offers. The authors and editors obviously put great effort into organizing the data into tables that make it easy to look up and compare the various offerings. If they hadn't, the report would probably be 700 pages and nearly useless.

The result is a report that overcomes the vendors' marketing noise to truly differentiate the products and services being reviewed. For each scenario, the offerings are rated as "Likely," "Possible" or "Unlikely" to meet the needs. Screenshots and descriptions back up the conclusions. For example, to choose a random offering, IndexTools is ranked as ?Likely? to suit multi-site analysis and interactive marketing analysis, but unlikely to be suitable for an application-driven web site. More important, the authors tell you why they came to those conclusions, which makes it easy for you to decide if you agree or not. Often, the greater value in these kinds of reviews is in the thinking behind the conclusion, not the conclusion itself, so this report earns high marks for explaining itself.

There isn't much missing. Robots and spiders are treated as a problem to be filtered or blocked, rather than being essential to anybody who wants to be found via search. Open-source analytics packages are described very briefly at the end, which isn't as serious an omission as the lack of any explanation as to why they are described so briefly. Are they that useless?

Bottom line? This report is well worth the price. While staying grounded, it paints a picture of what web analytics can be and how to get there. It makes you want to go out and measure something.

About Nick Arnett

Nick Arnett is Director of Business Intelligence Services at Liveworld Inc.

About CMS Watch's Web Analytics Review

View a sample chapter of the Web Analytics Review from CMS Watch (registration required). The list of vendors and the table of contents are available without registration.

I remember what it was like to walk through the door at my brand new job, my very first job as a web analyst, wondering what I?d gotten myself into. In retrospect, what did I wind up learning the hard way? What would been helpful to know up front? What should I have been prepared to expect? With that in mind, here are 10 things I wish I knew when I started in web analytics:

1. You will sit between the techies and the marketers. Figuratively, and maybe literally. Make friends on both sides of the fence.
2. You will learn all about your business. Not just the stats part. Not just the web part. The work you do in web analytics will only make sense once you?ve put it in the general context of your business.
3. Ahem, what is this thing you call a "Visit"? Know your standard web metric definitions by heart, and be able to recite them concisely for people who ask. They will ask.
4. Dirty, dirty, dirty. Numbers won't match, they won?t add up, they won?t make sense, sometimes they won?t even exist. Know how much dirt you?re willing to live with, then accept it and move on.
5. You will learn to love the query string. You will come to see it as a beautiful haiku. You will know it backwards and forwards. You will repeatedly explain its usage to people who need to append campaign codes to URLs.
6. CSV stands for "comma-separated value" ... it's a file format, every data analyst's friend, and - inexplicably - it doesn't even have to be comma-separated. Huh.
7. Operators are standing by. Know the support hotline number for your commercial web analytics vendor of choice, and don't be afraid to call. If you have one sticky note on your monitor it should be that number. Actually two sticky notes. The other one should say, ?Patience is a Virtue.?
8. Don?t fall into the ?report monkey? trap. Manually-repetitious activities are not a good use of your time, so automate wherever possible. Strive to spend your cycles doing thinking fellers work, and leave robot work to the robots.
9. You are not alone. Right now there are other web analysts sitting at their own desks, somewhere between the techies and the marketers, and they?re facing exactly the same issues that you are. You will meet them at Web Analytics Wednesday.
10. Think long-term. From the very beginning, think about where you want your career to go and make every effort to develop in that direction. Your entry-level position in web analytics can/should/will lead to other things, so know what you're targeting and go for it.

About June Dershewitz

June Dershewitz is currently Vice President of Analytics at Semphonic, a leading web analytics consultancy with headquarters in San Francisco and offices in Boston and Washington, DC. Read June?s blog at http://june.typepad.com/.

Theories are always theories of practice,
whereas practices are always practices of a theory.
Fernando Pessoa

As part of my M. Sc. Thesis on Online Behavior Analysis and my present job as the responsible for Applied Web Analytics at easynet search marketing, I have been reading a lot of academic and industry papers on the fields of Web Mining and Web Analytics. While researching Online Behavior across academic journals, books and blogs, the differences between the industry and the academic approaches seemed to be substantial.

Even though both fields have the same overall objective, i.e. measure/understand/improve website usage, they differ significantly on their approaches. Web Mining focuses on the analysis and prediction of visitors? usage in order to improve website performance and/or recommend products or links based on users? behavior. Web Analytics focuses on the direct improvement of conversion rates and the stickiness of the website: how to design and organize the website in order to improve sales and engage/retain customers. Of course, there are intersection points, like a Venn diagram, but I would rather see something like an Euler diagram; the problem would be to define who is who? Is Web Mining inside Web Analytics or the other way around? (Please leave a comment if you know the answer).

One interesting fact is that Web mining published research uses solely server-side data, while client-side data is extensively used throughout the industry. In some cases academics also use third party data samples, but I will not discuss it in this article, since this source of data is clearly less accurate when analyzing online behavior. Here is a short description of both methods:

Server-side

The most common approach in academic research. Every time a visitor to a website requests any information (for example, when a visitor clicks in a link to go to another page in the website) the server of the site registers this request in a log file. The log file can have several different formats, but Extend Log File Format saves the following information: the IP of the computer that requested information, date/time at which the transaction was completed, time taken for transaction completion, bytes transferred, records whether a cache hit occurred, and the referrer.

Client-side

Also called a ?page tagging? solution or the use of an application service provider (ASP), this technology is widely used throughout Web Analytics industry. It consists of inserting a small piece of Javascript code (which is not allowed to be cached) in every page of a website. This means that every single time that a visitor opens a page, this Javascript code is activated and the visitor information and actions are saved in a separate file.

Both data collection techniques have their qualities and limitations. However, Client-side data collection is more accurate than server-side when analyzing customers? behavior, since it counts every single visit and pageview to a website (unless the customers closes the page before the script is loaded); log files are affected by cached pages by the Proxy (the network connection provider) or the user?s browser, which can send a page to a visitor without registering a log file in the server. The cached information is lost whenever analyzing log files, reducing the accuracy of the customer?s information.

In addition, the Javascript code is not read by crawlers (mechanism used by search engines in order to rank websites), which generates high amounts of traffic and are not representative of customers? behavior. Many of the crawlers can be excluded from log file analysis; however, it is a very time consuming task, and many of them are not recognizable.

See below a short review of both fields and the analytical process used in both.

Web Analytics Review

The objective of Web Analytics is to understand and improve the experience of online customers, while increasing revenues for online businesses. This can be done by studying the ways customers navigate in a website. According to the Web Analytics Association, the official definition of Web Analytics is:

"Web Analytics is the measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage."

Web Analytics is not solely a technology to produce reports; it is a process that proposes a virtuous cycle for website optimization. This process enables the marketer to attract visitors and measure customer acquisition costs against profits. In addition, it helps identifying visitors? segments, showing how the most profitable visitors are behaving in a website and where they are coming from. Based on the best practices of the field, a framework for analyzing a website performance contains the following steps:

1. Define the website?s goals.
2. Define metrics in order to measure goal achievement; this can be done through the creation of Key Performance Indicators (KPI) which show whether the website is getting closer to its objectives or not.
3. Accurately collect data and save it on a local or external database for further analysis.
4. Analyze data in order to create alternatives, the actions the decision maker will take to improve the website to reach its goals.
5. Test the alternatives in the website in order to check which action should be chosen.
6. Deploy the improvements, analyze the results, and use the outcomes as feedback for updating the objectives and continue improving the website.

Web Mining review

Eirinaki and Vazirgiannis (2003) researched the field of Web Mining for web personalization, presenting an encompassing review of the field. The authors define web personalization as follows:

"(?) the process of customizing a Web site to the needs of specific users, taking advantage of the knowledge acquired from the analysis of the user?s navigational behavior (usage data) in correlation with other information collected in the Web context, namely, structure, content, and user profile data."

Web personalization is an output of web usage mining. An interesting approach to the web personalization process is proposed by the same authors as a five step module:

1. User profiling: identifying information specific to each visitor and create a database with preferences, characteristics, and activities of each user.
2. Log analysis and web usage mining: organize and process all the information gathered through data mining and statistical techniques in order to discover usage patterns and segment customers by their behavior.
3. Content management: the process of classifying and categorizing the information so that it is easily retrievable and accessible by visitors.
4. Web site publishing: technologically allowing users to receive the information contained in the website, be it by making it available over the web, by sending feeds of content daily by email or by a mobile phone.
5. Information acquisition and searching: retrieve and analyze information from sources other than the website itself.

Further reading

Berkhin P., Becher J.D., and Randall D.J. (2001) ?Interactive path analysis of web site traffic?, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p. 414-419.

Borges J., Levene M. (2000) ?Data Mining of User Navigation Patterns?, Revised Papers from the International Workshop on Web Usage Analysis and User Profiling, LNCS Vol. 1836, pp. 92-111.

Eirinaki M., Vazirgiannis M. (2003) ?Web Mining for Web Personalization?, ACM Transactions on Internet Technology, Volume 3, No. 1, February, p. 1-27.

Eirinaki M., Vazirgiannis M., and Kapogiannis D. (2005) ?Web Path Recommendations based on Page Ranking and Markov Models?, Proceedings of the 7th annual ACM international workshop on Web information and data management, SESSION: Web ranking and retrieval, pp. 2-9.

Garofalakis, J. and Kappos P. and Mourloukos D. (1999) ?Web Site Optimization Using Page Popularity?, IEEE Internet Computing, Vol. 3, Issue 4, pp. 22-29.

Garofalakis M.N., Rastogi R., Seshadri S., and Shim K. (1999) ?Data mining and the Web: past, present and future?, Proceedings of the 2nd international workshop on Web information and data management, p.43-47.

Kaushik, Avinash (2007) Web Analytics: An hour a day, Sybex, California.

Moe W.W. (2003) ?Buying, Searching, or Browsing: Differentiating Between Online Shoppers Using In-Store Navigational Clickstream?, Journal of Consumer Psychology, Volume 13 (1&2), p. 29-39.

Moe W.W., Fader P.S. (2004) ?Dynamic Conversion Behavior at E-commerce Sites?, Management Science, Vol. 50, No. 3, March, p. 326-335.

Montgomery A.L., Li S., Srinivasan K., Liechty J.C. (2004) ?Modeling Online Browsing and Path Analysis Using Clickstream Data?, Marketing Science, Vol. 23, No. 4, Fall, p. 579-595.

Peterson, Eric T. (2005) Web Site Measurement Hacks: Tips & Tools to Help Optimize Your Online Business, O?Reilly Media, California.

Phippen A., Sheppard L., and Furnell S. (2004) ?A practical evaluation of Web Analytics?, Internet Research, Vol.14, Number 4, pp. 284-293.

Sarukkai R.R. (2000) ?Link Prediction and Path Analysis Using Markov Chains? Proceedings of the 9th international World Wide Web conference on Computer networks: The International Journal of Computer and Telecommunications Networking, p.377-386, June 2000, Amsterdam, The Netherlands.

Senecal S., Kalczynsky P.J., Nantel J. (2005) ?Consumers? decision-making process and their online shopping behavior: a clickstream analysis?, Journal of Business Research, Vol. 58, p. 1599-1608.

Sterne, Jim (2002) Web Metrics: Proven Methods for Measuring Website Success, John Wiley & Sons, Canada.

WAA website, Web Analytics Definitions

Zhu J., Hong J., and Hughes J. G. (2002) ?Using Markov Chains for Link Predictive in Adaptive Web Sites? Proceedings of ACM HT?02, Maryland, June.

Interview Summary:

"Measuring Widgets"  What are widgets? What are the measurements for widgets companies track? What level of technical expertise is needed to measure a widget deployment? What types of reports are available for widgets? How can widget metrics be integrated with web analytics data? Interview with Jodi McDermott, Director, Product Management for Clearspring Technologies, Inc. (Jodi blogs on Widget Analytics at widgetanalytics.wordpress.com/).  Interview date?November 21, 2007 by Jennifer Day with the WAA Research Committee. Time--23:55.

MP3 File

Research Committee Project: Measuring New Media

In the 2005 Membership Survey, WAA members expressed a strong interest in better understanding analytics best practices. This Research Committee project, Measuring New Media, tackles the impact new media and technologies (Ajax, flash, RSS, blogs, streaming media, podcasts) are having on our sites and on our analytics. Measuring New Media interviews experienced analysts to uncover best practices and lessons learned with these new media technologies. Presented in podcast form, Measuring New Media's interviews address questions as:

• Can we determine the business impact of new media?
• Can we determine adoption/operational performance?
• Are new media metrics being integrated into existing site metrics, and how?
• What measurement challenges do analysts face with these emerging technologies?

By Robert Blakeley, Product Manager
WebMD

A web analytics program consists of many elements. One of the important elements in the process is the report. This step often does not get the attention it should, taking a back seat to the effort of getting and compiling the numbers (on the Analytics side) or day-to-day operations (on the Business side). It is sometimes abbreviated or omitted altogether in an attempt to save time. This is a mistake. This article focuses on that reporting activity and the structure the report itself.

Web analytics helps your team understand how well your Web investment/campaign is doing and how you might be able to do better. But that understanding is not useful to the Business if the insights are not shared in a way that the rest of the organization can focus on and get behind. Getting their attention usually means making Web Analytics part of your regular business process. Part of that regular process is the reporting.

Reporting is the analysis deliverable. You are sharing what you learned. It allows you to disseminate the knowledge throughout the organization. Good reporting can also drive the Business to act on that knowledge so the organization can do better.

What Makes a Great Report?

For reporting to be effective, it must be provided in a way meaningful to the recipient. A great report focuses on its audience as well as its subject. Most people will glaze over when confronted by rows and columns of numbers. Numbers-only reports tend to float to the bottom of the product owner?s to-do list, unread. That is not what people need, but all too often that?s all that is provided.

Great reporting provides insight and recommendations. It is these insights and recommendations that people can understand. These recommendations should focus on managing for value as well as performance. Further, written recommendations tend to become to-do lists that the Business implements.

Great reports share these style attributes:

1. Clarity ? The information is easy to understand. A clear sentence does not require the reader to decipher how the sentence is to be interpreted. Do not use ambiguous or complex language. Avoid using technical jargon. Avoid restating the obvious. Direct and succinct declarative sentences are almost always better than marketing prose.
2. Completeness ? Completeness provides context. The report should be at the appropriate level of detail, covering all the necessary variations or conditions, but making sure you are not sacrificing clarity for trivia that adds no real value.
3. Accuracy ? The information contains no factual errors and does not mislead the reader. Neither overstate nor understate what you need to convey.

Remember you are trying to provide understanding through your reporting. Keeping these three points in mind will help support that effort.

A Reporting Outline

In the past, I have successfully used a six-part reporting format. It can be used for either ongoing tracking or short term testing. (I selected short term tracking for the abbreviated examples below). Each section is intended to provide understanding about what is being reported and what it means:

1. Purpose
2. Test Conditions
3. Results
4. Analysis
5. Lessons Learned
6. Recommendations

Purpose

The intent of this section is to provide an understanding of the goals and objectives of the report to the reader. It outlines why you have created this report, what you are tracking, and why are you tracking it. Here you declare the purpose of the test and the hypothesis being tested. Communicate how the test or tracking supports the corporate goals including any quantified targets and relevant milestones.

For example:

?This report provides the results of an A/B test on copy for the first page of the Web account creation process. It was run June through July of 2006 by Michael Jones, Janice Abrams and Rob Blakeley.?

?The test was initiated as a result of a review of the Web site by consultant John Score. John recommended improving the benefit copy on the page. The purpose of the test was to improve the click through rate from the first page by improving the perceived benefit of getting a Web account.?

?The rate of Web account creation has been declining and the Business seeks to reverse that decline. Customers with Web accounts are a primary source of prospects for services and, in addition, 25% more likely to purchase goods and services than a non-account customer.?

Test Conditions

Outline how the test or tracking was conducted. Was it an A/B split test, a year over year comparison? In an A/B test, for example, provide a screen shot of the control copy and the test copy. Who got the various copy versions and how was it delivered? Who provided the copy? If it is it ongoing tracking you would indicate what is being tracked, how it is being tracked and in what variables.

For example:

?Two of our Sr. marketers each provided different benefit copy for the page. Each copy was run in an A/B test against the existing control copy. (The control copy itself had been created by the Web IT staff). Half the people coming to the page got the control; the other half got the page with the test copy in an alternating fashion. At the time of the test, the copy had not been revised for 5 months?.

?The first test copy (call it the M copy) was run from June 1st. through June 29th. The second copy (the N copy) was run from July 1st. through July 29th.?

?The Control copy was as follows:?

Etc.

Results

This section is where you present those numbers, charts, graphs, etc. as appropriate. You should also indicate any anomalies encountered during the test.

For example:

Analysis

Analysis is the meat of the effort. Now that you have the numbers, what do they mean? Present your causal hypothesis ? what happened and why could it be happening? Here is where you draw your conclusions and inferences on user behaviors and the effect on the business. By effect, I mean to include the economic impact.

The first three sections (Purpose, Test Conditions, and Results) exist to present the context for the Analysis section. It is here you present the success or failure of the thing being tested or tracked.

You should make your conclusions explicit for your reader and include all assumptions, definitions, qualifications, etc. so that any objections can be accounted for. You should also be clear about the limits of the analysis, given the available information.

For Example:

?Historically, the click through rate for the page with the control copy can vary plus or minus 5% month over month. However, there has been a downward trend in the conversion rate over the past 6 months. The current trended average is 15% below the previous norm.?

?The percentage difference in click through between the control copy and the N and M copy is not statistically significant. In effect, we are looking at the same results for the three copy versions. We also note that the alternative copy did not diminish results.?

?We can suggest two reasons for the lack of change. One is that in both cases, the alternative benefit copy was not effective. Perhaps better copy would produce better results. The second possibility is that in this context, benefit copy is not useful in producing click-through; it?s not a significant trigger.?

?The test does not provide a definitive answer to which reason is correct, only that the tested copy produced no improvement. However, it is understood from prior testing that task-oriented visitors tend not to read page copy. This is a plausible explanation for why the copy made no difference at all, either positive or negative.?

?Although the control copy was last revised at about the time the decline in registration began, the decline began before the last revision. It is worth pursuing improved click-through rate for the page. However, it is unlikely that this will address the root cause of the decline that initiated the test.?

Lessons Learned

In this section, you can take a step back and abstract the results of the test. What are the fundamental underlying lessons? Can you extrapolate the lessons to other pages, products or sections on your site? Are the results in line with other industry results? If not, how do they differ and why?

For Example:

?Best practice suggests that persuasive benefit copy should produce correspondingly better conversion results. This is a well accepted understanding. The test results are not what we would expect based on that understanding. It is also understood from prior testing that visitors, particularly if they are task oriented, tend not to read Web copy.?

?A tentative conclusion is that the visitors are not reading the copy. We may also want to look at our event registration forms, where we sometimes have a significant amount of benefit copy on the form page itself, to see if that copy is productive.?

Recommendations

Make specific recommendations based on the insights gained from the test. At the end of the day, this is what management pays you for.

You can also perform an opportunity assessment ? what are options for taking advantage of the knowledge in other areas or in new areas of the business?

For Example:

?Because there is no improvement using the new copy, we should continue to use the control copy.?

?It will be useful to conduct an additional test with copy provided by Marketing to help verify that benefit copy is non-productive on this page. While an eye-tracking study would be more definitive, such a test is cost prohibitive in this case.?

?We have not tested usability changes to the form itself. We also recommend testing changes to the form to see if improvements to the click-through rate can be obtained in that context.?

After the Report is Created

First, distribute your report to the decision makers and the affected parties. But don?t stop there. All too often, this is where the work ends, sitting on a desk and never considered.

Meet to discuss the findings. Meeting, while time consuming, is often the most effective means of getting people on the same page and moving forward. It helps people to focus on the lessons and insights to be learned from the report.

Discuss the analysis and recommendations. Then begin to discuss what changes need to be made: labeling, placement, copy, persuasion flow, the call to action, usability, site navigation, product design. Create an implementation plan. List all the tasks, who does them, and when must they be done.

You should then memorialize the information. Put the report in an accessible repository where folks know where to find it. Reports need to be constructed in such a way that they can provide insight a month or a year later to people who were not involved in the original project. It then becomes part of the institutional expertise. The business will not need to keep testing the same things and relearning the same lessons as often happens when knowledge is not shared.

Lastly, the methodology described above should be part of your regular business work flow, if it is not already. Make it a habit. As part of a regular process, it is both easier to do and more effective.

About the Author

Robert Blakeley is the product manager for WebMD?s analytics tool. Mr. Blakeley has worked in the Internet industry since 1993 and has worked with many companies and government agencies. These include the Direct Marketing Association, International Council of Shopping Centers, Atlantic City and the City University of New York. He can be reached at rblakeley@webmd.net. More articles by Robert Blakeley can be found at www.rblakeley.com/webwork/articles.shtml.

Interview Summary:

"Measuring Google Gadget Ads"  What are Google Gadget Ads? What are top key performance indicators for Google Gadget Ads? How are Google Gadget Ads tracked? Interview with Christian Oestlien, Product Manager for Gadget Ads at Google.   Interview date?November 15, 2007 by Kelly Makimaa with the WAA Research Committee. Time--13:02.

MP3 File