Search solutions have been a mainstay of knowledge management since document management software first entered the scene. During the last decade, the emphasis has shifted from using keywords to find a particular document or list of documents, to solving business problems and answering questions.
In a recent study by Aberdeen Group on the use of search, users’ focus shifted noticeably in the last two years. "The major emphasis from responders in our previous study was a general goal of increasing productivity," says David White, senior research analyst at Aberdeen. "Now, the priority is to support decision-making within specific functional areas, based on understanding information contained in large stores of content."
The need to access content enterprisewide for decision-making has generated continued interest in federated search and has also promoted use of hybrid search in which structured and unstructured information is integrated.
White says, "A much greater percent of our ‘best in class’ top-performing responders are using dashboard technologies to integrate structured and unstructured data than are the laggards in their industry. Also, customer service, R&D and sales received equally high ratings in their use of search technology, indicating the value of search is broadly recognized across functional areas." Organizations are also moving aggressively to tag information, putting metadata around it that allows searching for type of project, key people and other dimensions.
Addressing customer needs
One of the strongest drivers for robust search capability is the need to provide high-quality customer service, including technical support, by accessing whatever information is required to address customer needs. Netezza, an IBM company specializing in data warehouse and analytic technology, wanted to enhance its ability to respond by providing a single view of customer information across several systems used by its support agents. Agents had been using the search tool in SharePoint, but could only search a few content sources. To respond effectively to customer inquiries, agents needed to access numerous sources, including SharePoint, e-mail, an engineering wiki, bug tracking documentation and more.
Netezza selected Customer In-formation Solutions from Coveo, a specialized search application that shows 360-degree views of customer information. Within a month, Netezza was able to reduce the time required to identify known customer issues by 67 percent, because it was much easier to locate the relevant information. The number of duplicate bug submissions, for example, was reduced by 50 percent because customer agents were able to more easily discover previous submissions. Coveo presents search results in a dashboard format that shows software incidents from a variety of perspectives, including by customer account, software release, hardware model and other dimensions.
Being able to find descriptions of similar previous incidents helps on two fronts: for agents assisting customers and for developers who need to stay up-to-date on software issues. "We are not just providing a search function," says Laurent Simoneau, CEO of Coveo. "We are providing resolutions to complex problems and affecting overall business metrics." In the case of customer support, the information may not all be in one place. "Coveo’s Enterprise Search 2.0 platform is the virtual glue over all of the repositories," Simoneau explains, "allowing a unified view of the information no matter where it is."
Providing that unified view involves multiple steps. The first step is to normalize information using a set of rules so that the data becomes comparable no matter where it originates. Some sources may have a different format for names and addresses, for example, or lack metadata tags that are already present in other sources. After normalization, text analytics are used to summarize the contents and provide entity extraction. Processing the text makes it much more valuable than if it were simply indexed and searched via keywords. It also allows for sophisticated inquiries from the search software such as "Did you mean?" or "More like this?" to redirect the search. Coveo is also recognized for its strengths in categorizing and visualizing search results in graphic form.
Coping with volume
Sometimes the sheer volume of information is a potent incentive for developing effective search capability. Tyco Electronics has more than 75,000 employees and manufactures several hundred thousand products that include electronic components, network systems and telecommunications products. The company’s 200 intranet sites, 200 SharePoint sites, forums, blogs and wikis constituted a formidable array of content that needed to be accessed for a wide variety of business uses. Dissatisfied with the limited functionality of its search software, Tyco Electronics began looking for a more effective solution and chose the Velocity Information Optimization Platform from Vivisimo.
Velocity was installed and rolled out to the intranet sites in about 60 days, with other content added in subsequent phases. Tyco Electronics is also making use of the social components of Velocity, including tagging, rating and annotating search results. The number of searches per day increased by a factor of 10, with the majority of users rating the search application as "effective" or "very effective." In addition, besides using Velocity to locate content, Tyco Electronics has indexed employee biographies and contact information to establish an expertise location ability.
Product development, customer service and supplier relationships are among the specific business problems at which search solutions are being directed now. "It is no longer enough to put up a search box," says Stacy Monarko, senior director of product management at Vivisimo. "The push is toward goals such as improving information governance, shortening the training cycle or other business need." Moreover, IT is no longer the primary driver for deploying a search solution. "The business leaders in the organization are increasingly recognizing that they need to make the most of their information," adds Monarko.
Concern about security is sometimes an obstacle to the implementation of search technology. “We know of one company that turned off its search engine because it was getting to unsecure content,” Monarko says. “Without the proper controls, search can violate privacy regulations. Velocity works with the enterprise’s security system, including having control at the document or even at the sub-document level if a given document has different levels of access within it.”
Open source option
Open source software (OSS) is predicted to grow at more than 20 percent per year, according to IDC (idc.com), and has made inroads into many knowledge management markets, including enterprise content management, business process management and business intelligence. LucidWorks Enterprise from Lucid Imagination is an enterprise search platform built on open source Lucene/Solr search technology from Apache. LucidWorks Enterprise is free for development and test; production deployments require a subscription through which a variety of support options are available.
“LucidWorks Enterprise is an open source search platform that can be used out of the box,” says Marc Krellenstein, founder and CTO of Lucid Imagination, “and we provide ongoing support and customization for those organizations that want that service.” The cost of support is comparable to the cost of supporting proprietary products. “The advantage of using open source products comes strongly into play for large applications where licensing fees might become prohibitive for proprietary products,” Krellenstein says. “Considering that LucidWorks measures up to the proprietary products in terms of scalability and accuracy, the cost-effectiveness is a strong benefit.”
The maximum benefits of deploying search technology can best be achieved by careful consideration of the organization’s needs and goals. Although many organizations have search software in place, not everyone is satisfied with its performance, and that is sometimes the result of not having carried out a requirements analysis.
“Even though search is to some degree commoditized,” Krellenstein says, “there is not usually a single right answer to a search. It depends on the data and the user. For any given application, you can segment the data to get a better answer, but the downside is then you have more silos. Finding the right balance can be a challenge.”
Planning for search
One of the biggest obstacles to a successful search implementation is the failure to consider how all the parts of the enterprise work together. “It’s very important to have a knowledgeable project administrator who sees the big picture when you are searching across many repositories,” says Lynda Moulton, senior analyst and consultant at Outsell’s Gilbane Group. “These should be people who know not only the content, but also know the organization’s staff and business processes.”
An early stage in deploying a search application should include an overall assessment of content. “Text analytics can be valuable not just for business problem solving but also for obtaining an overview of information in the enterprise,” Moulton says. “A comprehensive linguistic statistical analysis shows immediately what words and concepts appear frequently in the documents, which can help start a thesaurus and a taxonomy.”
Organizations that were most successful, Moulton reports, are the ones that also went on to slice and dice information across the functional areas to identify specific content and learn how it was being used. That process should be ongoing, to keep pace with the organization as it changes over time. To work well, a semantic search engine with auto-categorization requires a current thesaurus of terminology and concept relationships that are meaningful to the enterprise in which it functions.
Text filtering
The explosion in the volume of enterprise content has had an impact on organizations’ need for search, and also on their need to analyze the content more thoroughly to make sense of it. Longtime search software vendor ISYS developed many documents filters over the years, and decided to offer them as an OEM product to other software companies.
“Document filters identify the file type, identify and extract metadata, and then extract the text itself for deep inspection and indexing,” says Dave Haucke, VP of marketing at ISYS.
Equivio provides analytic solutions for e-discovery. It incorporated ISYS Document Filters into its product suite to enable high-performance text extraction. “The cost of document review is so high that law firms and corporate law departments want to include just the relevant ones,” Haucke says, “but they also need to be sure they don’t miss anything critical.” ISYS Document Filters allows extraction of text from several hundred file formats and types.
Sybase also includes ISYS Document Filters as an available component for its Sybase IQ business intelligence solution. Sybase uses the filters to ingest text from unstructured documents. From there, the customer performs analysis using Sybase’s tools. The extracted text can then be used in Sybase’s applications for e-discovery, fraud detection and forensic analysis.