Category: Analytics

You must become an accountable content organization

If you’re in health care, like most of our clients at Creekmore Consulting, you are already familiar with the term “accountable care organization.” ACOs have become a hot topic in health care — last year’s health care reform bill really promotes the idea that health care organizations should be reimbursed based on the effectiveness of treatment, not just on the fact that they provide services. Health care providers are now working to figure out the best ways to demonstrate the effectiveness of the care they provide, so that the government and insurance companies will pay them.

We see a related [though thankfully, very, very rarely life-or-death] issue in our content strategy work: The need for accountable content.

Accountable content starts with a business goal. [Just a quick note to say that your problem is not a good definition of your business goal. For instance, your problem is that your help content is hard to keep current and no one uses it. Your business goal is creating useful help content that reduces call volume by XX% this year.]

It’s too simple to say that everything else flows from the business goal, but it’s true. You have to make every decision about your content through the lens of your business goal. Should you write your own content? Should you license it? Should you hire a freelancer or seek a long-term relationship with a content development firm? Who will approve and manage the content? What kind of technology will you use? How will you measure the results? You’ll give better answers to every question with the business goal pasted on the wall.

We’ve worked with many organizations through the years that do not realize how much content they’re already managing, and the overhead they’re already putting into the process. Very few organizations don’t have content. Most don’t have the right kind, or don’t manage what they have well. The real opportunity sometimes comes in the ability to narrow down your content-under-management to only what truly serves your business need.

Is your content accountable? Is it working for you? Or are you working in service of your content?

Required reading: You’re using research wrong

Let me start by saying, My mama raised me right. It just didn’t fully take.

And so I have disrupted more than one meeting with some — shall we say — unpopular assertions about the role of research in marketing and product development. I’m almost always polite*, yet I can’t let pass the opportunity to share my view that research results do not constitute a message from Your Favorite Higher Being Here.

After New Coke, I’m not sure why anyone has to explain this again. There are emotional and group-think elements to human response, and these are difficult to measure with standard business research techniques.

I won’t go on: I’ll just direct you to Ben McAllister‘s great piece in The Atlantic on the dangers of using research to make creative decisions. He agrees there’s an appropriate role for research, but argues that it’s dangerous to assign scientific weight to information that should be viewed as an indicator only.

Thanks to Randall Snare and Jonathan Kahn for linking to this piece and thereby bringing it to my attention.

*About this topic, anyway. No comment about my embarrassing reactions to poor retail customer service.

Confab | Christine Perfetti: Essential Techniques for Measuring Your Content’s Success

Christine Perfetti is talking about how to test and measure your content.

So important: Web analytics tell you WHAT is happening. They do not tell you WHY.

Perfetti is showing really horrifying and totally believable tests….

  • People who got all the way through reserving a room at Disneyland…who wanted a room at Disneyworld.
  • People who were thrilled with the hotel reservation form for another hotel…until they got to the end and discovered it didn’t work the way it appeared, and they didn’t have a reservation set up after all.

Both these examples just make my stomach hurt.

Perfetti’s recommendations on usability testing

Oh, love this: Perfetti just has observes in the room. She says, People know there is someone behind your one-way mirror. Better just to have them in the room and don’t be sneaky. Intros people by first name only, not title. Prefers that observers use paper and pen so as not to have distraction of typing.

Perfetti’s recommended books on usability testing

What you should look for in a test

  • Users going to the FAQ: People aren’t getting what they want
  • Users clicking on Help
  • Users going to a sitemap
  • Users clicking the back button: They’re not seeing what they want
  • Pogosticking: Going back and forth on a list of links, looking for something they never find
  • Going straight to search: People don’t want to search. They haven’t seen their trigger words, so they type them in.

So now we’re going into advanced techniques specifically for content.

5-Second Test: Bring up a page for literally 5 seconds, see if people can figure out how to solve a problem you give them. We’re doing one here about uploading photos…does it seem it will be quick and easy to do? Perfetti shows a page full of text from Photobucket…the room laughs. Is anyone confident they can upload photos? No one.

Now we see 5 seconds of Picasa. This was also text heavy, but also had a big blue button that said, Get Started. Now we’ve seen a Flickr page. It showed a visual representation of the process. Most people agree this is the one that makes it look quick and easy to upload photos.

fivesecondtest.com will let you test pages on their site. Perfetti warns: 5-second tests work “very, very poorly” for home pages — these pages have many priority. Use these on content pages or other single-purpose pages.

Great point: People want to spend all their time testing their home page. This is not usually a good use of time. They want to find the page that is useful to them. Test whether your home page directs people to the content they want, but test the effectiveness of your other pages.

First-click test: Give users a specific task, and see where they click first. Are they headed down the wrong path?

Love this: Users don’t like to choose their role. Perfetti shows the WebEx site where users were befuddled when forced to categorize themselves as individual, small, medium, large enterprise. They redesigned the site with a single “Products” link, and users found that much easier to navigate.

Comprehension test: Use to determine if people understand your complex content. Sometimes Perfetti will use a questionnaire, or sometimes will just ask questions. She shows the content, asks them to read through it, and asks the questions while they can still see the content. These are generally information, but will clearly show if users understand the complex content.

Inherent value test: Helps you figure out if you’re conveying the product value to prospects. In phase 1, bring in your most loyal customers. You ask them to give you a tour of the site and talk through the value of the product or service. Have them share what they find most valuable. You can also ask them to complete tasks, but that’s not the main goal. In phase 2, bring in people unfamiliar with the product. Then, ask the prospects to complete the tasks that the loyal customers do all the time. You will find out if they see the same value.

Catalog-based task testing: Find out what’s important to users. Take any printed catalogs or brochures [or print out same info from your site] and ask users to highlight important content. Then ask them to find the same content on the website.

Some other software and resources that were mentioned:

Confab | Johns Hopkins Testing Content

Hearing Ahava Leibtag and Aaron Watkins talk about Johns Hopkins and how they test content.

Challenge of working on a major academic medical center’s website: Lots and lots of cooks in the kitchen. Branding is difficult to maintain because content creation is widely distributed. Have to balance the interests of users [patients], doctors and executives.

Wow, here’s the first challenge: Had a 6-page website promoting a weight loss program that offered medical support for people who wanted to lose 20-30 pounds, and also a $4000 procedure that insurance wouldn’t cover.

Competing visions of the site:

  • One person wanted 60-page, research-heavy site.
  • One person wanted to answer patient questions.
  • One person wanted to promote services and show ROI.

How do you meet all these needs?

Created a competitive landscape analysis: Discovered no competitors had research-heavy sites, but instead were all consumer-friendly. So they answered that challenge.

Defined the audience: Obese people with complicating disease, obese people with disease influencing obesity, potential patients for weight-loss program.

Can users find, read, understand, act and share your content? [See the link for an expanded discussion on this metric at CMI.]

How do you know the content is effective? How do you know it meets your users’ needs? Do not assume you know the answer. You must test your content.

User Testing
They showed a bunch of user testing videos. If you don’t do user testing, man, you are missing out. People will tell you, they don’t read. They want to come to your website and magically get the answers. And the best part: They did unmoderated user testing about got great results, but realized the people were reviewing the site once they saw the questions.

In their tests, video decreased comprehension of the message. In the health care realm, they think that video is a poor way to transmit facts. Instead, they recommend using video to make an emotional connection, explain visually how a procedure works, or reinforce the brand. Use video for 1 message at a time. Make it simple. Complex topics need written content so people can review it later.

Really wish every visual designer I know could see the real users in these Johns Hopkins usability tests, one after another, saying, the text is too small.

Here’s a great point: People who spent more time on the site did not increase their comprehension of the message.

Confab | Testing Content

This session from Angela Colter is slam-packed full. Her popular article last year on testing content on A List Apart obviously precedes her, and everyone wants the details.

She’s going to talk about

  • Readability Formulas: Counting the aspects of text that can be counted, and applying a metric. The Flesch-Kincaid formula measures the number of words per sentence and the number of syllables per word to give you a projected grade level of the text.
    Upside: People can understand and act on this metric.

    Downside: You’re only counting what can be counted. Doesn’t explain anything about text clarity. And my favorite downside, it can push you toward really crappy writing.

    Great quote from George Klare: 

    Merely shortening words and sentences to improve readability is like holding a lighted match under a thermometer when you want to make your house warmer.Don’t use a readability formula alone — it’s good as a red flag.

  • Usability Tests: Find out what people actually understand, not what they say they understand. Get the right people in the room. What are the critical issues for the user and for the business? You have to get users to think aloud, but you need to know what the correct answer looks like.Well written tests are critical to evaluating your content properly.Usability testing ends opinion-based arguments.
  • Cloze Tests: These are the content usability tests where you remove every 5th word and see if people can fill in the blanks. Use 125 words or so and 25 or so blanks. If they’re getting 60% or more, your content works. If they’re getting 40% of less, your content is too difficult. In between, you may need revision or more instruction.If you test on the appropriate audience, this tells you clearly if the content is appropriate.

Confab | How to Create a Data-Driven Content Strategy

This session is by Elizabeth McGuane and Randall Snare, who together write Mapped.

They’re going to talk about how to use data to improve your content strategy.

  • Analytics
    When you see where traffic is coming from, and where it’s not coming from, you can tell if you have a title/link name problem. Ambiguous titles don’t inspire clicks. Product names may not inspire clicks. People need to be able to make a clear assumption about what they’ll get when they click.Challenge here is that this kind of information is sometimes buried deep in your analytics reports.
  • User testing data
    User testing can give you really concrete information about how your site and your content confuse users.  McGuane tells about how she tested some content on a banking website, for general readability and also understanding of specific terms. Used a “cloze” test — where you remove every 5th word and let your subjects fill in the word they think fits best. The test demonstrated that the tone of the content was wrong and was confusing 90% of the readers.
  • Metadata
    Ah, now they’re loving on metadata. Make my heart sing. McGuane says metadata gives your content longevity, structure and extensibility. You can run into challenges when you’re trying to best organize the page title and keywords against business goals….what’s good for navigation now may not be good later. Have to get creative about how you manage competing needs.
  • My so called data
    How do you analyze things that are subjective, for instance, quality of writing? Use examples…for brand/style/content guides, examples or prototypes tell the story best.McGuane gives an example of a yogurt company that researched not just the company, not just yogurt, but all dairy…and discovered that by far the most common comment about dairy products was how creamy they are. And the yogurt company wasn’t selling any products based on how creamy they are — great opportunity.
  • Useless data
    What data can you throw away? Had a client request to do a GPS on a blimp…when that didn’t work, wanted a car to follow it…too complex. Turned out it was enough to create a map with dates and times you could see the blimp. Don’t create or use data you don’t have to have.

More: Think about priming — if someone says “chicken,” it doesn’t take you long to process “egg,” but if first you hear “goat,” your brain is slow to process “ghost.” You can prime your readers by carefully crafting the content.