News   Features   White Papers   Research Reports   Web Events   Conferences  
November 29, 2006

Table of Contents

Featured Content: Enterprise Search Market Trends -- Free Report from Faulkner Information Services
Search, Aggregation, and Syndication Segment Reaches $68 Billion by 2009; According to Report
Toward the transparent enterprise
Recommind gets to the point
Collarity’s Compass Automatically Ranks Information Based on User Interest
Attensity, ClearForest, Inxight Expand Tech Partnerships with Business Objects
AOL Acquires Relegence Corporation
Easing navigation
One-stop business Web
Quintura Launches Find Engine

Featured Content: Enterprise Search Market Trends -- Free Report from Faulkner Information Services


The market for enterprise search technology has changed in key ways in recent years, particularly as more companies have entered the sector. These changes provide potential customers with new ways to implement technology that searches their networks for critical data. The changes also create a more complex array of choices, so understanding the state of the market and its future direction is critical for making the right choices about enterprise search technology.  Click here for your free PDF of this report from ITI's Faulkner Information Services division.

Report Contents

  • Executive Summary
  • Market Dynamics
  • Market Leaders
  • Market Trends
  • Strategic Planning Implications
  • Web Links

Executive Summary

Enterprise search technology enables users to employ a single interface to perform natural language searches across multiple storage sources containing a variety of file types. The market is crowded and quickly changing. Whereas until recently it was dominated by smaller specialty players, large companies have used a mixture of acquisitions, partnerships, and internal development to gain strong positions in the sector.

The volume of data in most corporate networks, combined with the variety of ways the data is stored, means that no search tool will be able to effectively find all relevant data in a typical environment. Although in some respects this limitation makes it possible for specialized vendors to establish niches for themselves in the face of competitive pressures from much larger competitors, it also makes it difficult for the specialists to maintain meaningful technological advantages over new entrants. With these opposing difficulties creating pressure from two sides, many smaller vendors are creating revenue by forming partnerships through which large software providers (e.g., enterprise resource planning, enterprise content management vendors) integrate the smaller company's search technology into their own products.

Organizations that are evaluating enterprise search technology should bear in mind that no single product will likely be able to search all data effectively and provide results in exactly the manner desired. Therefore, it is important to prioritize which types of data are most important and then seek a product that handles that type well. The use of multiple search tools should also be considered.

Market Dynamics

Important corporate data was once largely confined to databases, which are structured and therefore facilitate search. Today, however, a great amount of corporate data is contained in word processing files, spreadsheets, e-mails, and other sources that don't lend themselves to searching. Some organizations attempt to make files easier to locate by tagging them with metadata. Metadata is information about a file that is not in the body of the file. For example, it could designate the subject into which the file should be categorized. Metadata is either produced automatically (for instance, the author's name would be automatically stored) or it is manually entered by a person. In either case, however, metadata cannot account for all the queries that a user could potentially make. Natural language queries are too diverse and unpredictable, and the labor involved with metadata tagging is too intensive to be performed for all corporate data.

Enterprise search technology aims to address this problem by enabling users to search across a network or a collection of networks to find files regardless of their type or their storage medium. Most products aim to enable users to search using natural language queries rather than Boolean searches, the syntax of which confuses some end users.

The goals of reducing the time employees spend looking for information and making better use of existing data for decision support functions have historically been key drivers of search technology. In addition to these broadly-held goals, the specific needs of two sectors have been particularly influential in spurring the growth of enterprise search in recent years:

US Government--The US Electronic Government Act of 2002 compels the public sector to increase the use of the Internet as a means of making information and services available to citizens. The need to electronically organize such data and services and make it easy for citizens to find has made the US government a prime user of search technologies.

E-Commerce--E-commerce companies that aim to let their customers easily search through large product catalogs are another major customer for enterprise search technology.

Market Leaders

The enterprise search market is densely populated, with entrants ranging from multi-billion dollar giants to small, specialty providers. The market is also in flux. Among the four vendors listed below as being the most dominant and influential, two--IBM and Google--have only recently become major players in the sector. IBM gained its spot largely through acquisitions, in particular its acquisition of Iphrase. Google attained its position by internal development, expanding its Web search technology into enterprise search products and services.

Autonomy -- Already one of the main players in the enterprise search technology market, Autonomy strengthened its market position in 2005 by acquiring Verity, itself a major vendor in the sector that expanded its portfolio in recent years by purchasing Inktomi. Autonomy reports that it has integrated its own enterprise platform, IDOL, with Verity's comparable K2 Enterprise. The integrated product is called IDOL K2. IDOL K2 categorizes data, provides collaboration functionality, offers data retrieval, and enables users to personalize search results. The company's IDOL Federator integrates with K2 to facilitate searching multiple sources, including searches using third-party technology.

FAST-- In addition to its FAST Enterprise Search Platform (ESP)--its flagship, general-purpose enterprise search application designed to search large networks for a variety of file types--the company offers two lines of specialty search solutions with more narrowly-defined purposes.

Data Search. The Data Search line of products, which are based on FAST (ESP), can be purchased and used separately or in conjunction with each other.

  • Fast Data Search for Site Search--A search tool that customers can add to their Web sites.
  • Fast Data for Search Compliance--Searches for customer-defined flags that indicate that a document might affect a company's compliance with prevailing regulations. It can be set to search documents in real-time as they are created or distributed.
  • Fast Data Search for eCommerce--A search tool that customers can add to their e-commerce sites. It can adjust search results to place showcased products more prominently.
  • Fast Impulse--Designed to let e-commerce companies better control how their products are presented to users who are browsing the site. It also helps control marketing techniques such as cross-selling and up-selling.
  • Fast Data Search for Intranet--A tool for searching enterprise networks and external sources.
  • Fast Data Search 360--A tool for enterprise searching, designed to be rapidly integrated into a corporate network.

Search Derivative Applications (SDA).  FAST search derivative applications are custom-developed from FAST ESP technology. Others may be developed in the future.

  • FAST Marketrac--A tool for finding information relevant to market intelligence, inside and outside an enterprise.
  • FAST AdVisor--An application for electronic directories (e.g., online yellow pages, business-to-business verticals) that provides tools for users to find content and for advertisers to reach customers.
  • FAST mSearch--Lets manufacturers of mobile devices give their customers a search tool for finding Web sites online via their mobile devices.
  • FAST mSearch for the Enterprise--Enables mobile devices to search enterprise data sources as well as the mobile Web.

In addition, FAST also offers three publishing-oriented SDAs that have search capabilities.

Google -- Google has used its dominant position in the Web search market to quickly establish a strong foothold in the enterprise search sector. It offers dedicated search hardware as well as software that can be installed on desktop systems or servers. The company's enterprise search technology is closely based on its Web search technology, offering ranked results, summaries, and cached pages, among other features familiar to users of its online service. Although its products initially focused on HTTP documents, the company's products are now designed to search a broader range of document types.

Search Appliances. The company offers two product lines of rack-mounted search devices: the Google Search Appliance and the Google Mini. The basic model in the Search Appliance line is the GB-1001, which can search up to 1.5 million documents and handle 300 queries a minute. Multiple units can be clustered to create the GB-5005, which can handle up to 5 million documents a minute, or the GB-8008, which can handle up to 15 million. The Google Mini is a lower-cost rack-mounted device; the base model enables searching of 100,000 documents and can be upgraded to search up to 300,000.

The Mini starts at $2,995, and the GB-1001 starts at $100,000.

Search Software. Google's search software can be downloaded for free. The Google Desktop for Enterprise is designed to enable users to search a corporate network for files, e-mails, cached Web pages, and other types of files. (The company offers a support package for $20,000.) The Google Toolbar for Enterprise, which is currently in beta testing, adds extra buttons and features to Google's browser-based toolbar.

IBM -- IBM is investing heavily in enterprise search, both in terms of developing the technology and in building a product portfolio. The bulk of the development work is being done as part of an initiative called the Unstructured Information Management Architecture (UIMA).

The UIMA framework is the foundation for IBM's Websphere Information Integrator OmniFind Edition, which provides the ability to search intranets, the Web, databases, file systems, and other data repositories. It uses IUMA technology to determine the ranking and relevancy of search results. In late 2005, IBM acquired the business of iPhrase, whose products enable e-commerce companies to let Web users search electronic catalogs and other online information sources. The products gained in this acquisition are now part of IBM's WebSphere suite. For organizations that wish to develop software that has features not offered by OmniFind, IBM offers the UIMA Software Development Kit.

IBM views enterprise search as part of a larger information management program, which also includes technologies such as content and records management. In addition to its acquisition of iPhrase, IBM has acquired several other companies in recent years to advance its information management efforts. These include Ascential, DWL, and SRD in 2005 as well as Venetica and Alphablox in 2004.

Other Noteworthy Vendors

Despite recent merger and acquisition activity, the enterprise search market remains crowded. Small and mid-sized vendors such as Convera, Endeca, Entopia, and InQuira continue to collectively play an important role in shaping the market. Some of these smaller companies may become the acquisition targets of larger organizations that are looking to quickly enter the market or that wish to expand their current search application portfolio. (For perspective on the difference in size between certain vendors in the market, Convera had $25.7 million in revenues in 2005, and IBM had $96.2 billion, although only a fraction of IBM's revenues were from search technology.)

Large vendors whose specialty is not search are also having an impact by adding search technology to their existing products. For instance, Microsoft now has two search offerings: SharePoint Search Services, which is part of its SharePoint Services, and Windows Desktop Search, to which the company has recently added intranet searching, integration with SharePoint, and other features that make it more suitable for enterprise deployment. Hummingbird, Oracle, and SAP also now have search products. Although applications with search technology added as an enhancement may be, in most cases, much less powerful and feature-rich than the leading products on the market, many companies, particularly small to mid-sized ones, will find such offerings suitable, especially when the products are freely offered with other software.

Market Trends

A few years ago, the sector was dominated by small- to mid-sized companies that were primarily focused on enterprise search. In recent years, however, large companies have used a mixture of acquisitions and internal development to quickly and aggressively gain a strong footing in the market. Boutique vendors do not appear to have technological advantages that are strong enough to compensate for their lack of name recognition and marketing power, if they have any technological advantages at all. Consequently, newer entrants into the market such as IBM, Google, and Microsoft are poised to increase their positions within the sector, even if their offerings lack some of the functionality of their competitors.

Although boutique vendors may loss ground, the market will likely still have room for them, at least for the next several years. Enterprise searching still works with only moderate effectiveness, and the continued growth of stored electronic information--particularly audio, video, and image files--makes the task even more difficult. Organizations have varying needs in terms of the types of information for which they must search, the total amount of data to be searched, and where the data is stored. Consequently, there is ample room in the market for niche players. No search product will meet the needs of all customers, or even a large number of customers. Individual products will find success in certain vertical markets, or they will establish themselves as the best tool for certain types of searching.

In addition to several acquisitions, there has also been a significant amount of technology consolidation through partnerships. Many developers of content management, customer relationship management, and other types of enterprise software are integrating third-party search technology into their own products. For instance, portlets from Autonomy integrate with IBM's WebSphere Portal Software. Likewise, EMC's Documentum, one of the leading content management products, can be integrated with search technology from companies such as Convera, Fast Search & Transfer, and Google. Furthermore, Fast and Siebel formed an agreement in 2005 for Fast InStream to be integrated into Siebel's core business applications. Search capabilities can therefore be installed by purchasing a dedicated search product or by implementing another enterprise application that includes searching functions. These types of partnerships offer specialty vendors another opportunity to solidify their place in the market and find additional revenue streams.

It is uncertain whether search products will have more success in the coming years as a standalone technology or as technology that is integrated into other applications. The trend of large software companies adding search capabilities to their products suggests that the integrated approach will become more popular, although in many cases the search technology that is integrated into other applications may be developed by smaller, third-party companies, not developed in-house by the large vendors.

Currently, one of the key trends in Web search technology is verticalization, which refers to search services that focus on a certain topic (e.g., financial services, travel). The push toward verticalization is driven by the need to improve results: searches across the entire Web produce results that are not focused enough for many purposes. Although there is still considerable emphasis on comprehensive, multi-purpose enterprise search applications, there are some vertical offerings, such as FAST's Data Search products. Given the continued difficulty enterprise search technology has in capturing all relevant data and returning results in an effective way, there is a good chance that more verticallyfocused enterprise search applications will be developed.

Strategic Planning Implications

Organizations that are evaluating enterprise search offerings have many choices to consider. Enterprise search is available as software, hardware, or a service. Also, it can be standalone or part of a broader offering. Further complicating the evaluation process, enterprise search can be highly specialized, so a particular search solution may not be capable of sorting through certain types of files or it may not be able to index and present results in the way a certain customer requires.

With these difficulties in mind, prospective customers would be prudent to begin the selection process by identifying what information sources they wish users to be able to search. Although any organization would want all information to be searchable, this is an unrealistic expectation in most cases. The currently available search technology is limited in terms of its ability to effectively cover every source, and the cost of search technology is high. Therefore, the list should be prioritized to target sources that must be searched and leave as optional sources that might be less critical.

Although many search products are intended to be comprehensive, potential customers should remain open to the possibility of using more than one solution to meet their search needs. Although a multivendor approach works against the goal of having a single search interface, and although it adds to expenses, it may be the best option in some cases. The technologies might overlap in some areas, but they may also compensate for each other's deficiencies.

In addition, some search needs are specialized and are best met with specialized tools. For instance, some organizations may be primarily interested in searching through large text documents, such as manuals; for these organizations, a product such as TeraText Solution's TeraText Database System, which is designed specifically for text searching, should be strongly considered. The overall effectiveness of a particular product matters less than the product's ability to meet the specific needs of an individual organization.

A search application's utility is determined not only by the quality of its algorithms but also by its user interface. Therefore, an organization should use a trial implementation to test a product's ability to search the targeted storage sources and return useful results. End users will be the best judges of whether search results are useful, so a handful should be included in the trial implementation.

Enterprise searching is difficult in large part because corporate data is stored in so many sources, in such an unstructured way. Rather than relying on enterprise search technology to alone address this problem, a complimentary approach is to change the way that data is created and stored. Enterprise content management applications aim to manage data from its inception. Although likely there will still be some unstructured data despite the use of content management technology, content management can minimize such data and improve an organization's ability to effectively find key information. There have been some signs that enterprise content management and enterprise search are merging to an extent. For instance, EMC, a content management leader, now includes search capabilities as part of its EMC Documentum content integration offering. The search capabilities include EMC's own technology as well as technology from Google. Also, FAST offers its search technology to providers of content management application and other independent software vendors through a platform called InStream. Partnerships in which enterprise search vendors provide their technology to large enterprise software vendors will become more common in the coming years, providing potential customers with another option for implementing enterprise search.

About the Author

Geoff Keston is a project manager for a leading technology consulting and services company. In this role, he has been responsible for the successful completion of enterprise software implementations, network upgrades, and telephony implementations for major retailers, financial firms, and public institutions. Geoff also writes extensively on issues relating to software, data networking, and ecommerce, as well as on the cultural, economic, and political issues raised by technology. He is a Microsoft Certified Systems Engineer and a Certified Novell Administrator.

Web Links

Click here for your free PDF.

Back to Contents...

Search, Aggregation, and Syndication Segment Reaches $68 Billion by 2009; According to Report

Outsell, Inc., a research and advisory firm for the information industry worldwide, has announced publication of its MarketView report, Search, Aggregation & Syndication: 2006 Market Size, Share, Forecast and Trend Report. Outsell forecasts that this segment will achieve a four-year compound annual growth rate of 17% through 2009, reaching $68 billion. The report provides company information on players in the search, aggregation, and syndication (SAS) sub-segments of the business information industry, such as Google, Yahoo!,, LexisNexis, Factiva, EBSCO Industries, Follett Educational Services, and Ingram Book Group.

Key findings of the report, include: Search research by sub-segment: Top 10 Web search and portal companies include Google, Yahoo!, MSN, AOL,, Infospace,, SINA Corporation, TOM Online, and LYCOS; Top 10 licensed content aggregators and professional services companies include Thomson Corp. (Dialog & Gale), LexisNexis, EBSCO Industries, Factiva, ProQuest, OCLC Online Computer Library Center, Copyright Clearance Center, Books24X7, Flysheet Information Services, and Access Copyright; and Top 10 distribution and subscription services are Follett Educational Services, Dawson Holdings, Royal Swets & Zeitlinger, Ingram Book Group, Source Interlink Companies, Baker & Taylor, Blackwell's Book Services, Publisher's Group West, Maruzen Company, and iGroup.


Back to Contents...

Toward the transparent enterprise

Open Text has introduced Livelink ECM 10, which, it says, lets customers implement a legitimate enterprisewide content management strategy, allowing them to preserve and protect intellectual capital, leverage business content across all applications and effectively address governance and regulatory compliance requirements.

Accomplishing those goals allows users to reap the full benefits of Open Text's new initiative, dubbed Enterprise Transparency, which, the company says, reflects the evolution of content management from simply tracking and controlling information to leveraging it for business advantage--setting it in action to drive business processes, create content-centric business applications, bridge structured and unstructured content repositories, and unleash information workers to make better, faster decisions based on a holistic, centralized view of business content.

Open Text reports that some of what's new in Livelink ECM 10 is available immediately, with the rest being rolled out in 2007.

New features in Livelink ECM 10 include:

Enterprise Library Services--In Livelink ECM 10, seamlessly integrated archival, metadata management, enterprise records management and search capabilities are exposed as Enterprise Library Services, providing organizations with the ability to effectively implement enterprisewide retention strategies. In addition to managing content stored in Livelink ECM repositories, Livelink ECM 10 also provides the ability to manage the metadata and life cycle of content stored in enterprise applications from vendors such as Microsoft, SAP and Oracle, as well as business content stored in SharePoint Server 2003, e-mail, file systems and other repositories.

Bridging business content with enterprise applications--Version 10 allows users to access business content in ERP systems, such as customer information or purchase order documents, from their familiar Microsoft Office Outlook 2003 interface.

Flexibility for provisioning basic content services--Open Text reports organizations have the flexibility to build and deploy solutions on any basic content services offering (such as SharePoint Portal Server 2003), while managing the enterprisewide retention of business content with Enterprise Library Services.

Emphasis on user experience--In addition to Livelink ECM's Web-based user interface, Livelink ECM 10 will feature a new rich client interface, providing seamless access to business content from within leading Microsoft desktop applications.

New Web services APIs: Livelink ECM 10 will provide published Web Services APIs for Enterprise Library Services and Livelink Content Services.

Back to Contents...

Recommind gets to the point

Recommind announces that its flagship information access platform, MindServer 4.2, provides full enterprise search integration with Microsoft Office SharePoint Server 2007.

The integration enables Recommind and Microsoft customers to quickly implement Recommind enterprise search with Microsoft Office SharePoint Server 2007 and establish a powerful information hub. Recommind explains that combining the MindServer platform with SharePoint Server 2007 enables users to search knowledge assets within the SharePoint portal, as well as in the organization's e-mail, document management, enterprise management, human resources, time and billing, CRM and contact management systems.

Recommind explains that MindServer delivers results ranked by relevancy to the user's SharePoint interface. From there, users can refine their searches using Recommind filtering capabilities. When a search result is selected, the user can view text of the results through Document Preview, view additional context for the result and/or browse other related results.

Back to Contents...

Collarity’s Compass Automatically Ranks Information Based on User Interest

Collarity, Inc., a search technology provider, has launched the Collarity Compass, a search technology that automatically ranks information based on the implicit interests of individuals and dynamically created communities of searchers to improve relevancy. The new technology is designed to give publishers, who integrate the Compass into their site, the ability to offer their consumers control to cut through information and find the narrow slice of the web that is personally useful and interesting to them.

The Collarity Compass is designed to enable publishers to deliver more effective site and web search results by providing each visitor with the power to automatically define what is most relevant for them and provide search guidance for future searchers. Every visitor on a publisher's site becomes both an information provider and an information consumer. As site visitors search, the system automatically and anonymously gathers, organizes, and shares search knowledge. Collarity couples content relevancy and site search to improve user satisfaction.

While a site's web visitors search, the Collarity Compass creates a real-time dialog, automatically streaming relevant keywords and URLs based on the knowledge and successful search history of millions of other searchers. Collarity is delivered as a hosted web service overlay to a web publisher's existing information architecture and can be up and running in hours. In addition, there is no up-front licensing fee for publishers with websites supported by advertising. Collarity Compass is a free web service for consumers, and is generally available in beta for consumer and publisher testing.


Back to Contents...

Attensity, ClearForest, Inxight Expand Tech Partnerships with Business Objects

Attensity, a provider of text analytics software solutions, has announced its Exhaustive Extraction technology has been integrated with the BusinessObjects XI Release 2, a business intelligence (BI) platform from Business Objects, a provider of BI solutions. The newly expanded technology partnership between Attensity and Business Objects is part of the Business Objects Open Search Initiative (OSI), a new partner network of text analytics, enterprise search, and consulting companies. Through this technology partnership, business and government users will be able to interact with stores of information--everything from call center notes and customer emails to surveillance reports--to discover facts, trends, and insights without having to be experts in the BI system itself.

ClearForest, a provider of business solutions enabled by text analytics, has also announced a technology partnership with Business Objects, a provider of business intelligence (BI) solutions. As a member of the Business Objects Technology Partner Program, ClearForest will work with Business Objects to create structure from text sources for reporting, contextual search, and faceted navigation within BusinessObjects XI Release 2. The partnership is a result of the new Business Objects Open Search Initiative (OSI), a partner network of text analytics, enterprise search, and consulting companies designed to help customers find answers hidden in textual sources. ClearForest and Business Objects software are available from their respective vendors.

Inxight Software, Inc., a provider of enterprise software solutions for information discovery, has also announced a new technology partnership with Business Objects to provide federated search and text analytics for the BusinessObjects XI platform. In addition, Inxight will participate in the new Business Objects Open Search Initiative, a partner network of text analytics, enterprise search, and consulting companies. Inxight allows users to leverage the power of unstructured text in more than 30 languages to augment the BusinessObjects XI platform in financial services, legal discovery, government, healthcare and other mission-critical operations.


Back to Contents...

AOL Acquires Relegence Corporation

AOL LLC has announced its acquisition of news and information search technology company, The Relegence Corporation of New York, NY. AOL's acquisition agreement with privately-held Relegence was signed and closed on Monday, November 6th, 2006. Financial terms of the deal were not disclosed. Relegence and its 60 employees will continue to be headquartered in New York, with offices in London and Tel Aviv. It will operate as a wholly-owned subsidiary of AOL, and will continue to partner with and serve financial services firms.

Founded in 1999, Relegence sells subscription-based services that offer real-time discovery, notification and delivery of information, connecting customers to cross-media content as it is generated from anywhere across the globe. Using proprietary technology, Relegence simultaneously monitors, indexes, and filters more than 20,000 unique, global sources of live content streams--such as local and international newswires, print media, television and cable networks, regulatory feeds, corporate and information websites, internet bulletin boards--and deliver to a user's desktop only the information that is relevant to his or her individual search criteria. Additionally, Relegence can integrate internal content streams and programming with external content. AOL plans to leverage these capabilities for its ongoing efforts to develop and deploy its publishing and content management platform.


Back to Contents...

Easing navigation

Siderean has released Seamark Navigator 4.1, which features a number of visualization enhancements designed to interactively present relationships, analyses and trends.

The company says Version 4.1 is able to derive more meaning from search results than typical navigation solutions by applying advanced analytics to quickly and easily compute sums, averages, maximums and minimums over numeric facet values, enabling enterprise users to survey and track aggregated results from multiple data repositories at a glance. Further, the company says, it incorporates hierarchical clustering and entity-level security, resulting in enhanced scalability and security capabilities.

New features and functionalities include:

  • new analytics capabilities, including chart support, table support and date/time support;
  • a point-and-click navigation builder for increased flexibility;
  • point-and-click information feed support for RDBMS, LDAP and Web service protocols;
  • standards-based SPARQL support for querying disparate data sources;
  • hierarchical clustering to allow exponential increase in scalability;
  • data provenance and role-based security; and
  • "Tag Library" support for enhancing user customization.

Back to Contents...

One-stop business Web

Northern Light announces a new version of its Business Research Engine.

Users of the Business Research Engine can browse the market intelligence centers (MIC), access featured reports and resources from the Web, and use the Business Web search without limitations, says the company. Full-text search of trade journal and business news databases is available, as is access to a new white paper service. A subscription or day pass is required to read articles from the journals and business news sources and to take advantage of advanced functionality such as search alerts, public alerts and user preferences.

Back to Contents...

Quintura Launches Find Engine

Quintura has launched its visual Find Engine, designed to give users the control to find what they are looking for by using advanced Neural Networking techniques. While viewing the Quintura Cloud, users can visually navigate and refine in order to find relevant information. Users can Save and Share their Find results.


Back to Contents...
[Newsletters] [Home]

Problems with this site? Please contact the webmaster. | About ITI | Privacy Policy