Paralleling politics in the run-up to an election year, enterprise search in 2007 was full of more hype than substance. There were declarations of search being passé, even "dead," as vendors attempted to look new and fresh to their prospects, declaring their products "Information Access Platforms" rather than just plain old search engines. Don’t let yourself be fooled: Underlying search and retrieval technology hasn’t changed much, though it is better at fundamentals like indexing and clustering. Overall, the search marketplace continues to surprise, some­times delight, and often obfuscate prospective customers. Nevertheless, based upon the research we’ve done at CMSWatch for our Enterprise Search Report, some key trends are emerging. 

Navigating Search

The search vendor Endeca may have applied for patents on "guided navigation," but while they popularized it, the reality is that nearly every vendor provides some degree of results categorization, be it clustering, browsable categories, or another approach. Distilling terms for navigation is relatively easy when the source is structured content or when metadata can provide guidance, but it gets far more difficult when unstructured content is in the mix. Vendors frequently differentiate by showcasing auto-extraction facilities that glean facets and terms from your corpus with­out you having to tag content. But based on the customers we’ve talked with in our research for the Enterprise Search Report, this is met with varying degrees of success. You’ll want to test such claims with your own content before proceeding—know that auto-extraction and categorization are still relatively hit-or-miss. The nature of language itself is inherently baffling. Humans often cannot make sense of what something means; software doesn’t do much better. Better to invest in a metadata strategy first and then train your software based on that.

Analyzing Search

What is management without measurement? Seems obvious, but until recently most search tools had no built-in capacity to analyze the rich information found in their logs. Using analytics to improve search is a no­ brainer. You can find out what people are actually looking for, what they’re finding (or not finding), and how often. Unfortunately, even systems with nice query and reporting interfaces often don’t link directly to the administrative panels (e.g., for hit boosting or best bets) to redress the problems you find, but arming yourself with good metrics is a useful start.

Uncrowding Search

Enterprise search vendors are making their default search results look like Google’s, even down to the green titles and blue hyperlinks. Why? Evidence suggests that searchers have more confidence in Google-like results pages. But enterprise search is often more com­plicated than public web search; sometimes employees need more bells and whistles on the results page, such as folders categories,and other data. That’s one of the reasons we use a scenario-based analysis to evaluate search technologies, and you should too. One tool does not fit all. But overall, let’s simplify the search results page. Options are one thing,but to have "facets,""buckets," and "folders" all in one set of results doth not make for a more usable experience.

Socializing Search

Many tools now allow you to rank, discuss, and blog about a search result right next to it, functionality that seeks to catch the trendy Web 2.0 wave. We can’t get too excited about this when many searchers still can’t type a simple keyword and get the results they seek. While using theso-called "wisdom of the masses" to refine relevance rankings can be helpful, it’s no replacement for cleaning up content, providing more semantically rich hooks into content through consistent metadata, and building a better index. Savvy enterprises will focus on getting these search basics right, then maybe turning their attention to social tag clouds … in 2009.

Talking Search/BI Convergence

In 2007 the marketplace saw substantial discussion and speculation about the "convergence" of big-time structured search, usually under the auspices of Business Intelligence (BI) tools, and big-time unstruc­tured search (usually by traditional enterprise search products). At CMS Watch, we think the convergence remains mostly talk. Most enterprise search tools that could retrieve information from unstructured datasets could also access and index databases as well. However, vice versa was not always true: BI vendors—and their customers—aren’t as savvy with unstructured content. For now, the two software segments remain quite distinct, mostly because customers are still trying to solve basic search and BI problems before moving on to more advanced challenges.

Google Continues Long March to the Enterprise

We’ve done a lot of research on the Google Search Appliance, the Trojan horse of the enterprise search world. Google continues to improve its appliance—and intimidate competition—even though (as we elaborate on in our report), the appliance remains somewhat deficient as an enterprise search tool. Long-term plans for the appliance surpass simple search. Google can use it as a platform for other enterprise applications, perhaps some offered on a hosted basis. But it’s a long way from the public web to the enterprise, and customers report that once you reach the limits of the appliance, you’re pretty much stuck. Google found a great, underserved niche for simple search and has exploited it well, but don’t expect plug-and-play magic.

Platforms, Products, and the Application Management Conundrum

One of the great surprises—and, for many customers, disappointments—of enterprise search is how technically challenging it becomes after you get beyond the basics. And this can happen pretty quickly.  Is the fundamental problem with enterprise search a lack of power or a sur­feit ofcomplexity?  The marketplace doesn’t seem to be able to decide. Some vendors, such as IBM, continue to expand on their multifaceted search toolkits. Meanwhile, competitor Oracle is working to simplify its offering and conceal or abstract much of the underlying power. Neither approach is universally ideal, but it is a measure of a maturing market that vendors are tending to go one direction or the other. The good news for you, the buyer, is that you have clear choices: fulfill an immediate business need or develop long-term capacity. That debate is as old as enter­prises adopting software. We make no judgments, but we point out in our report that you have solid alternatives either way.

If search is a software application, then it should be managed like one. This sometimes comes as a surprise to even the most seasoned enterprise IT team who may labor under the misimpression that even a lower-end search product is "install and forget." Even the lowest-end search platforms (even appliances like Google’s) can be configured and extended. But, of course, configurations beg management and testing themselves. In short, enterprise search has the enduring need for proper software development lifecycles and code and configuration management. As search tools get ever more powerful, the more it becomes incumbent on you to manage them with the same care that you manage your other complex, mission-critical applications. Treat search like the serious project it is before you invest in any tool.

About the Author

THERESA REGLI is principal at CMS Watch, a buyer-focused and vendor neutral analyst  firm covering content management, enterprise search, enterprise portals, and web analytics. She is contributing analyst and editor of the Enterprise Search Report 2008, available at www.cmswatch.com, where you can also read more about search trends.

InterSystems Corporation, a global software product company, announced the release of InterSystems DeepSee embedded real-time BI software. DeepSee's design provides application developers with the BI capabilities needed to extend existing applications as well as create new applications that are BI-enabled from the start. DeepSee is the newest offering in the InterSystems product line. The software family also includes the InterSystems CACHÉ high-performance object database, InterSystems HealthShare platform for regional and national electronic health records, and InterSystems Ensemble rapid integration platform.

(www.InterSystems.com)

Moonwalk Inc. announced that it has partnered with Plasmon on a joint solution that provides companies using traditional Hierarchical Storage Management solutions more control of their datasets and storage infrastructure. Moonwalk 6.0 advanced data migration software can move, copy, or archive files from various platforms and file systems to the Plasmon Archive Appliance based on the business value of the unstructured data under storage. The solution allows users to deploy a set of policies for classifying and archiving data, and it makes certain that these centrally configured policies will be distributed and enforced right to the edge of the network. It’s also network-aware and data-centric, so it allows for data management by project, location, user, group, size, name, time and other attributes.

(www.moonwalkinc.com)

Gotuit, a provider of video optimization technologies, announced that it has partnered with Pixsy, a B2B provider of private label video and image search, to deliver enhanced video search and syndication capability. Gotuit customers using the Gotuit Video Metadata Management System (VMMS) now can have mRSS feeds of their Gotuit-powered video library published to and ingested by Pixsy’s Media Search. Gotuit allows video publishers monetize structured metadata by describing each scene within the original source videos. The scene-level metadata can now be syndicated through mRSS feeds to Pixsy for inclusion in their Video Search Playback(VSP) platform.

(www.gotuit.com, www.pixsy.com)

Content Analyst has begun shipping Version 3.3 of CAAT, its search and text analytics software platform. Features in V. 3.3 include new data connectors, processing filters and multiple enhancements to CAAT’s core functionality, says Content Analyst.

The new SharePoint connector will allow CAAT users to ingest, analyze and search the vast stores of unstructured information in SharePoint repositories. New filters available in the CAAT 3.3 release include an OCR filter to help remove noise and improve search results from scanned documents, as well as e-mail filters to programmatically deal with the redundant information contained in many header and footer fields.

Further, the company says, CAAT 3.3 augments its core capabilities by adding full 64-bit support for dtSearch (used for keyword searching) as well as enhanced export capabilities. CAAT 3.3 supports cross-lingual categorization, which the company claims is a first in the industry, offering automated organization of mixed language document collections. CAAT 3.3 also provides users with a categorization "self-test," which automatically analyzes the quality of the examples and enables users to easily fine-tune how CAAT categorizes large document collections.

Mark Logic Corporation announced the launch of MarkLogic Sever 4.0, the newest version of its XML server. MarkLogic 4.0 includes new features that provide geospatial support to build location-based applications, alerting capabilities to push content to users based on saved profiles, and entity enrichment for search and navigation. MarkLogic Server 4.0 New Features built-in support for geospatial data tagging formats like GML, KML, andGeoRSS/Simple, built-in support for entity identification and inline markup, and new content analytics features.

(www.marklogic.com)

The Hyperwords Company has released Hyperwords 5.0. Hyperwords is a Mozilla Firefox web browser Add-On and is fully customizable. When the user selects any word, sentence, or paragraph, a menu appears which allows the user to carry out web-searches, language translations, and unit conversions. Results can be viewed directly in the menu, or opened in a new browser window. Other operations such as language translations and unit conversions can directly modify the displayed webpage.

(www.hyperwords.net)

OrcaTec LLC, a provider of information retrieval software and consulting, announced the release of a beta test version of Truevert, a semantic search engine. The first release of the Truevert semantic search engine focuses on "green" issues related to environmental and sustainability issues. Truevert uses YAHOO's BOSS interface to retrieve relevant documents from the web. Truevert uses OrcaTec's patent pending semantic search to identify and rank the most relevant documents. In addition, Truevert displays a list of the themes for the retrieved documents. Users can organize the returned pages by theme or can conduct new searches for any theme. Truevert also allows users to identify the interesting phrases in document so that they can quickly know what it is about and what other topics they might find interesting.

(www.truevert.com)

Biz360 Inc., a provider of media and market intelligence solutions, launched Opinion Insights, a technology-based monitoring and measurement solution designed for product and marketing decision makers. Opinion Insights captures and analyzes consumer opinion information from thousands of shopping, consumer product, and expert review websites, and summarizes this information into ongoing intelligence at an individual product level. Opinion Insights delivers a critical metrics, alerts, and in-depth analyses including customer advocacy trends, identification of key influencers, competitive gap assessments, price/value analyses, as well as feature-level and source-level analysis.

(www.biz360.com)

ZeBAze Computing has released ZeBAze, a new set of Windows search engines. Powered by artificial intelligence functions, ZeBAze mimics the human capability to use flexibility during the search process. Users can identify desired values for several number and date fields, and define which fields are more important than the others. ZeBAze retrieves the database's rows that are closest to these preferences, ranking them with an overall and flexible understanding of the search criteria. ZeBAze can connect with almost any database or spreadsheet table, and the search results can be displayed on screen or written into the database to be re-used with ZeBAze or another application.

(www.zebaze.com)

EntropySoft, a provider in the Enterprise Content Integration (ECI) market, announced a new OEM agreement with Coveo to integrate a selection of bidirectional connectors from EntropySoft into its flagship G2B Information Access Suite. The OEM deal provides Coveo customers with connectivity for accessing more information across the enterprise.  The new agreement with EntropySoft will allow Coveo to expand its portfolio to include connectors for Alfresco, FileNet P8, Hummingbird DM, Interwoven TeamSite, Microsoft SharePoint 2003, and IBM Lotus Quickplace.

(www.entropysoft.net, www.coveo.com)

The Dow Jones Media Group and Access Innovations have announced a joint marketing agreement to blend machine automated indexing and thesaurus tools for a robust indexing solution.

The companies report that the goal of the agreement is to enhance indexing and indexing database producers, online and Web-based directory publishers, corporate libraries, Web portals, and corporations and associations with large intranets. Both firms say new clients can experience clear and quantifiable increases in knowledge worker production output, as well as lowered staff and operating costs.

Further, they say, the MAI indexing and Synaptica thesaurus solution will help organizations increase their search engine capabilities and effectiveness.

Nstein Technologies has released WCM 4.0, a robust digital publishing solution for newspapers, magazines and online content providers. The new version is designed to increase editorial productivity by providing intuitive tools for the creation and management of complex sites and microsites.

The solution offers a new, role-based interface for editorial teams and includes:

• multisite/multichannel content management;
• out-of-the-box conversion and importation of content from popular XML formats;
• role-based, configurable editorial dashboards that mirror physical workflows;
• automatic search/association of assets;
• on-the-fly content association;
• real-time semantically generated tags and metadata;
• faceted search--advanced XML site search, topic clustering, most rated and most viewed stories; and
• a search engine optimization toolkit--keyword density, "friendly" URLs and optimized HTML structure.

The company says the user interface allows content experts to manage content from creation to distribution. It has built-in tagging and metadata management allowing easy association and bundling of content. WCM’s open design allows integration with virtually any new type of content modules (Facebook, Twitter, forums, blogs, wikis) and other content management systems (editorial systems, ECM, repositories).

WCM 4.0 integrates seamlessly with both TME and Nstein’s digital asset management solution to allow on-the-fly semantic tagging and semantic association of stories. Additionally, the open architecture allows integration with third-party tools, other content management systems and repositories.