Are you happy with the search experiences you have on websites, on your corporate intranet, or at support sites while looking for answers to your problems? My own experience, and my work with clients and their customers, tells me that very few search projects are successful. People can’t find what they need, and they waste a lot of time while failing to find it. Search projects fail for all of the reasons any project fails, but there are a few obstacles that loom especially large for search. Understanding the big mistakes of search can help you avoid them—or at least put a name to your current misery. We’ll go further than naming what ails you by outlining a path to sidestep the big mistakes and offering advice on how to escape the holes you’re already in. Our advice is primarily for project and program leaders. The big problems all belong to you. [Download the complete PDF.]
Culture Sets the Terrain for Search Problems Search problems occur across the project life cycle. Some problems are entirely predictable and mundane, common to all types of projects: project management gone awry, technology instability, failing to reach closure on requirements and so on.What makes search different from other projects and sets the stage for the big problems to come is what amounts to two cultural issues. We’re not going to suggest that you take on your corporate culture. But you need to understand how culture impacts search success because you’ll be struggling against the cultural tide as long as you are working on findability.First, most companies do not understand how to be effective content producers. In fact, most companies don’t believe they are publishers. They think they make diesel engines, software, or chips or sell fashion or building supplies. However, most companies have publishing operations, just as they have HR, accounting, and logistics operations. People have lots of mental models and role models for these core competencies, yet they have little idea how to be content publishers. Of this, there is widespread ignorance and even opposition. Without the habits and structures that support content publishing—and findability—your information collections will not be in tidy shape. In fact, they will be a mess, so people won’t find the information they need (and they won’t understand why). What’s far worse, people do not understand the tasks and roles that must be funded in order to achieve the find-ability that the organization needs and, as a result, attempts to secure funding will fall on stubborn deaf ears.The second issue is as much environmental as cultural. Search is an arena undergoing tremendous change. For the past 5 years, there has been great change in search usage: what people use it for, what questions they expect to answer, what kinds of information they index, what sorts of people use search, and in what context they use it. Expectations have risen steadily as search engines have become more competent. We are on an event boundary, and search engines are about to blossom into every application you buy. This will have a phenomenal impact on user productivity, but it will also drive expectations and demand. The bottom line is that it is impossible to guess who will use the search service you are building and what questions they will expect it to answer. Nailing down requirements becomes extraordinarily difficult: pilgrims in pursuit of the unknowable.
Project and Program Lifecycle Pain Points
Search projects follow a classic life cycle. But if search projects are to deliver great search, they need to become, or be supported by, search programs. Search projects are like planting grass. Search programs are like getting the grass to grow, keeping it mowed, and controlling the weeds. My life cycle addresses the search program life cycle, which typically starts witha successful search implementation project. Following the implementation, the search program kicks in, with the ongoing activities of monitoring and managing quality and findability. The concluding step, expanding the program, occurs when your program is successful. Your monitoring and analysis indicate what needs to be done, and your metrics prove the worth of doing it.In the search project and program life cycle, there are four phases that we observe to be particularly difficult with search, each with its own problems, which are summarized in the accompanying table.The solutions to these problems are not simple or easy to perform. You can’t solve them alone; they require a range of talents and roles. However, we can offer some insights into how to deal with each.
Vision The earliest mistake in your search project or program isvirtually unrecoverable—the failure to establish and communicate the rightvision. If you fail at this point, the project will not succeed, and you willprobably not be able to resuscitate it. In establishing your vision, you mustset a significant, desirable, achievable goal. To communicate it, you mustpresent a simple, familiar, and evocative story that helps anchor people to theproject, tell them what to expect, and help them think through what will happenand what it will mean. Here are some examples:
- <!--[if !supportLists]--><!--[endif]-->Customers will find the best product for theirsituation in less than 5 minutes.
- A customer will be presented an answer to a productusage question in 1 minute. Ninety percent of these usage questions will beanswered online in 2008.
- A customer with a problem will be on the path toresolution in 2 minutes or less from our website. Forty percent of resolutionswill be unassisted in 2008, growing to 80% by 2010.
- <!--[if !supportLists]-->Eightypercent of employees will use the internal knowledgebase at least weekly.
- <!--[if !supportLists]-->Notice themetric attached to each vision. This will come in handy later.
The vision is based on rational decisions, but it takespassionate commitment to communicate the vision and convince people that it isimportant and right. Ideally, the project lead or program head is theproselytizer and the executive sponsor opens the doors for the pitch. Plan todeliver your pitch daily for the first year of your program and weekly for therest of your tenure.
If you’ve already gotten this wrong, this is a devastatingmistake, akin to damaging your brand. The recovery will require beginning freshwith a new project, a new project name, and a new project leader.
Download the Free PDF (including all charts).
Requirements and Selection
Requirements are notoriously difficult to get right, forwell-known reasons. The big mistake with search requirements is thinking too narrowly. Getting the vision right helps set the scope, but our two cultural issues come into play creating such a strong tide you can’t help but want to narrow your profile to swim against it more effectively. The breadth of the real search requirements can be overwhelming, and as a result, the most common mistake is to narrow the requirements too far. This is especially true if the project team’s purview only extends to a small cluster of information with no clout or even access rights to broaden the search index. You build the service based on the scope that you own and end up creating a service that no one wants to use. Still, to be practical, you have to start something that you can finish, and quickly.The best approach to this conundrum, then, is to act locally but think more globally. As long as you have an outlet from the narrow searchs ervice that will allow you to achieve the sophisticated, high volume,integrated search you are highly likely to need, then it is OK to start small. Ask, how does this service become the most sophisticated and scalable service you can imagine? How will you enlist more support for a broader service? Can the technology support the broadest goals? Will the technology integrate with all the other specialized search engines that the enterprise already owns or is about to buy?If you’re already underway with a project that is too narrow, you need to set the stage for an outlet or opening. Consider your seekers and their questions. You build a service for people. Who are they and what are their problems? What information is needed to answer their questions? Who owns the information? Start scoping out what is really needed for your audiences. Declare the requirements, costs, and so on. Tell people the shortcomings of the current project for their constituencies. Calculate budgets. Talk about vision. Start cultivating allies with other information owners and audience advocates to support a follow-up project.Technology selection is the other big mistake at this life cycle phase. It is very easy to buy search technology you’ll have to throw away.People mistakenly buy technology that is too simplistic, perhaps because they’ve taken too narrow a view of requirements. Very few search applicationscan be satisfied with keyword retrieval. Keyword looks great if you’ve got no search at all, but after even a few days of using it, people will be dissatisfied. You will need technology that identifies concepts, extracts metadata, clusters similar items, and supports navigation by successive refinement. Also, it’s important to understand that when you buy "search," you need more than just indexing and retrieval. You need interfaces to manage synonyms, concepts, metadata, reporting, analysis, ranking, promotion, and merchandising. IT can’t build and maintain these interfaces and functions, despite the lure of simple, easy answers like open source Lucene. (For adetailed list of enterprise search requirements, download my Enterprise Search Planning and Evaluation Framework, www.psgroup.com/detail.aspx?ID=724.)If you’ve already got simplistic technology, your only choices are to scrap it or to invest heavily in content. Great organization and outstanding process can make up for lousy search technology. However, it is a good idea to track what it costs you to make content findable because this money might well be better spent on improved search technology. Make sure you don’t end up investing any more money on simplistic technology—for example, getting an enterprise license to a product that doesn’t meet your requirements. Lobby to buy smart search engines for every project.
Findability Policies and Procedures
Information is created and managed across the enterprise. Findability is almost never managed across the enterprise, or managed at all, and this is a big mistake. Again, this is a result of culture. People don’t know that findability needs to be managed, how it is effectively managed, or who should manage it.
The solution to this problem is a network of concerned citizens and impassioned volunteers. This cadre must contain people who understand the users and their complicated questions; the key collections,what they contain, and how they are organized; representatives of the big content producers; collection owners; and advocates for various audiences, such as the voice of the customer director or the usability program manager.
It’s your job to recruit them all. (Yup, dozens, scores, even hundreds of them.) I recommend you meet with each of them, step them through your vision pitch, and then tell them all about the Findability Directorate (or whatever attractive name you devise for the network). The directorate will share project plans and success stories; establish authority files, file-naming conventions, procedures for communicating findability problems to information owners, and policy for when to archive and remove content; develop strategy for handling metadata inconsistencies across collections,and so on. Then,you’ll ask your recruit about her area of expertise, listening carefully for opportunities to help each other. Then you’ll persuade her to join. You’ll have enthusiastic recruits, skeptical recruits, and refuseniks. Focus your efforts on those who will join because a few months down the road, you’ll have the success-driven clout to get the participation you need.
Initially, the group will come together informally, and policies and procedures will be informal as well. At some point, a combination of your persuasion, your recruits’ enthusiasm, and your recruits’ accomplishments will enable you to formalize the function. The recruits’ efforts will become part of their MBOs rather than a hobby that distracts from their "real" work. Getting the findability efforts funded will depend in part on your success with the next big problem: establishing and communicating meaningful metrics.
There are two key mistakes in the phases after search engine technology implementation: failure to establish and communicate meaningful metrics and failure to invest in improving the quality and value of the search service.
These two problems cross all four of the postimplementation steps, and they are intertwined. If you fail to establish meaningful metrics, people don’t know how well the search service is delivering its promise, how much remains to be done, and what value is being (or remains to be) delivered. As a result, attempts to invest in improvement—in the search experience, information quality, breadth and depth of the search service—repeatedly fail.
What are the right metrics? Ideally, your vision defined your metrics for you. In our vision examples, we had metrics for finding answers in a minute; 40% of resolutions unassisted; 90% of questions answered online.You chose this metric because it had significance to the organization. Use it unceasingly. Tell anyone who will listen that only 28% of resolutions are unassisted, and the 12% shortfall cost the company $36.2 million this year. But the successful online resolutions saved $84.6 million, and online resolution increased by 2% last year. And, by the way, here’s the plan for increasing it by 4% next year, which will save $11.85 million. People just adore precision. (I’ve covered this topic in detail in Search Experience Metrics, http:// psgroup.com/detail.aspx?ID=730.) Ideally, as in this example, the metrics themselves signify value. Other examples are revenue per search, conversion from search, and person-hours saved. If you can’t use a metric that contains business value, look for one that is closely associated with business value. For example, successful customer searches could be effective in an organization that placed great value on customer experience. First-touch resolutions mean a lot in customer service. By the way, you will likely need a series of metrics that are specific to applications or audiences in order to establish the value of the search service. Call deflection works in customer service applications, whereas customer abandonment from the product search results page gets attention over in the ecommerce team.It’s also very useful to have a single number that people use to evaluate search overall. For this I recommend tracking searches per visit, with a goal of less than one and a steady downward trend. No one wants to search. We all want to find.Being successful with search projects and programs requiresa bit of swimming against the tide and a lot of working the system. As a leader, it’s hard not to get sucked into the day-to-day demands of your project. What we’ve tried to do is highlight how important the bigger, "softer" problems are. The softer problems— vision, requirements, metrics—become the nuts and bolts problems of not having the assistance, resources, and funding that you need to achieve your goals. Our prescriptions provide concrete tactics for dealing with the problems and for avoiding the mistakes that lead to failed search projects.
Download the Free PDF.
About the Author
SUSAN E. ALDRICH is a senior VP and senior consultantat the Patricia Seybold Group, Inc. Aldrich manages the Search, Navigation, and Discovery Research Practice, with a personal research focus on customer self-service, information management, and technologies and practices for monitoring, measuring, and managing the Quality of Customer ExperienceSM (QCE)