Like quite a few enabling technologies for knowledge management, search solutions have shown remarkable resilience through the economic downturn. According to IDC, the market grew about 19 percent in 2008, to just over $2 billion, and preliminary figures for 2009 indicate a 6 percent to 8 percent increase. "Several factors are at work," says Sue Feldman, research VP for search and discovery technologies at IDC, "including the ongoing information explosion, compliance requirements and the growth of e-commerce."
Autonomy held the largest share for search and discovery software in 2008, with 14 percent of the market, followed by Microsoft, with 9 percent, according to IDC. Both of those companies showed double-digit growth for their software products in 2008. Google and Endeca are next in sales revenue, and they also showed healthy growth. Interestingly, the "Other" category, which accounted for nearly 60 percent of total revenue, grew at over 20 percent. According to IDC, "other" is made up of the hundreds of privately held companies that do not disclose revenue. Most of them are small, but some have revenue of more than $20 million
A new dimension
Search technology has evolved over the past decade, moving from basic keyword search to more sophisticated enterprise search that includes relevancy ranking, concept searching, clustering and entity extraction. More recently, the emergence of search-based applications has brought a new dimension to search (see chart http://www.kmworld.com/downloads/66062/Search_Market_Map_Chart.pdf). "These applications also employ search as a central component, but they solve a specific business problem or support an information-intensive process," says Feldman, "Search-based applications as a group are growing much faster than enterprise search because their value in accomplishing a task is immediately recognized by business users, because they are quick to deploy, and because they either control costs or increase revenue. Good examples of those are call center support, e-discovery, marketing, sales, R&D research to locate opportunities and e-commerce."
IntraLinks provides a collaboration platform called IntraLinks Exchange that is designed to facilitate information sharing among organizations engaged in a variety of corporate activities, ranging from mergers and acquisitions (M&A) to bankruptcy proceedings. Delivered via a software as a service (SaaS) product, IntraLinks Exchange provides access to documents in many different formats located in a variety of repositories.
More dynamic search
IntraLinks Exchange’s existing search solution had limited ability to deliver the interactive search experience that the company considered ideal. Moreover, the permissioning model maintained information about access in a separate database, which resulted in slow response times to queries and lack of document-level security.
After exploring the options for a new search solution, IntraLinks selected the Active Intelligence Engine (AIE) from Attivio. The unified information access (UIA) technology in AIE allows users to retrieve structured and unstructured data in a single query. In addition, UIA enables complex permissions to be maintained within the Attivio application rather than in a separate database. That affords greater security and reduces response time for queries. "This was really the biggest challenge," says Fahim Siddiqui, executive VP for products and operations at IntraLinks. "Access to documents and even concepts needs to be controlled, and Attivio’s product was the only one that could do what we wanted."
IntraLinks also sought a more interactive search process, including faceted search in which content could be categorized dynamically, and concepts could be extracted as the user conducted the search. "We wanted users to be able to explore the information to find what they needed, and for the system to suggest documents that fit the requirements, rather than just having a system where the users searched through a static file structure," Siddiqui explains. Users see only listings for the information they are allowed to access. Metadata is generated when the files are uploaded, but users can also add their own tags. "We also have alerts to notify users of new or changed documents, since the search engine is continually monitoring the content," Siddiqui adds.
Attivio’s ability to search and use relationships in structured and unstructured information with the same query relies on the company’s JOIN capability. Unlike a typical SQL query, the Attivio query does not need to have a data model set up in advance because it is schema-agnostic. "Users don’t always know ahead of time what they want to find out," says Sid Probstein, CTO of Attivio. "For example, they might want to display purchase data and customer comments for a particular time period." Structured data can be visualized through charts and graphs, and displayed side by side with content such as e-mails.
Guided navigation for e-commerce
Looking for items on retail Web sites has frequently been a difficult and frustrating experience for consumers. More recently, however, guided navigation is being used to provide better responses. Endeca powers the Home Depot Web site, and is effective in presenting an array of options to site visitors. If a prospective purchaser types in "faucet," for example, several major choices are presented even before the user hits Return:
- shop by category,
- shop by brand,
- how to: project guides, and
- how to: buying guides.
Those options, each of which also has subcategories listed, let the user select a general goal and then direct him or her toward more specific information through Endeca’s Guided Navigation. If the user hits Return, the default option is a page of faucets with a video on how to replace a bath faucet, a likely activity for a faucet shopper. Some of the information is drawn from product databases (structured), and some is in the form of documents such as project guides (unstructured).
Taxonomies have traditionally been used to organize and present content, but Guided Navigation has the advantage of being more flexible and better suited to a fast-moving business environment. “With Guided Navigation, a string of attributes such as price, availability and size could be factored together in a search,” says Paul Sonderegger, Endeca’s chief strategist, “rather than the user relying on a set of predetermined categories that might not apply as a situation changes over time.”
Search applications can also be used effectively in design engineering applications. In the telecommunications industry, for example, many factors enter into the selection of components. “The chip might need to be a certain size, and have a specific power consumption,” explains Sonderegger, “but other factors could also enter into the decision, such as inventory levels or preferred suppliers.” If the organization wants to steer engineers toward a product or supplier, the search application could present that option alongside the results. “The point is, the engineer might not have knowledge of all the options,” Sonderegger adds, “but search and navigation together can reveal options that improve a decision-making process.”
What’s relevant
Search results can be better tailored to the task in other ways when a search application is used, rather than a general search tool. Relevance ranking, for example, should be different for a researcher exploring the acquisition of a pharmaceutical company as compared to one who is looking for potential new drug products, even if the key words used are the same. “The different perspectives of users give them a different idea about what is relevant,” says Sonderegger. “Search applications allow relevancy to be calculated differently depending on the task.”
The role of search technology will continue to grow, both to perform simple tasks such as locating a document, and for more advanced and often more critical decision making processes such as discovering trends, tracking opinions or analyzing information across data and content repositories. Those latter areas are already hotbeds of research. “We are starting to see new approaches to enterprise search,” says IDC’s Feldman. “More attention is being paid to reducing the overwhelming amount of information that people must contend with, by using an individual’s job category, relationships with other people or previous queries as filters.”
Over the next five to 10 years, Feldman predicts, computing in general and search in particular will look very different from the way they do now. Search will become ubiquitous and implicit, and will routinely unify access to both structured and unstructured information through high-end information access and management platforms. Those platforms will not only handle all types of formats effectively, but will connect to both internal and external information sources, providing more comprehensive results.
Geospatial search
Geospatial software has been used for decades, but as global positioning system (GPS) technology has become more widely used by consumers, the value of that information has become more broadly recognized. Maps are an effective way for visualizing demographic, environmental and health information. Now, interactive mapping applications are being integrated with document collections so that users can find geographically relevant information.
MetaCarta displays documents side by side with a map (Page 17, KMWorld April 2010) that indicates the location to which the document is related. The MetaCarta Geographic Search and Referencing Platform (GSRP) is a self-contained appliance with information about more than 200 million locations, and connectors for map servers and content repositories. MetaCarta is also integrated with SharePoint and Microsoft’s Virtual Earth. As users view different parts of the map, they can see snippets of information from SharePoint content that relates to those locations.