Enterprise Search

by Theresa Regli

December 2008

What exactly is enterprise search – and what should you consider when identifying needs and choosing a solution?

Google has led many of us to believe that searching for information is an easy task: type something into a box, and some useful answers come back to you in the results. But that’s only on the Web. Unfortunately, searching for content within your enterprise isn’t nearly as easy.

Searching the web – which is what we all do when we type something into Google or any other internet search engine – is a very different beast from enterprise search. Here, we’ll explore what enterprise search is and things you should consider when looking at search tools for your enterprise.

WEB VS. ENTERPRISE SEARCH

Information retrieval and online search have been available for decades, but until the popularization of the World Wide Web, search-and-retrieval was a discipline with a loyal but relatively small following. Interest in enterprise search spiked largely as part of the internet explosion. “Enterprise search” was an attempt to make certain content available to authorized employees, partners, or contractors to an organization. At the same time came the realization that enterprises could benefit from the increasing amounts of data on personal computers, company websites, and departmental servers.

While website search is intended for use by individuals seeking web content, both within and beyond the enterprise, enterprise search systems are intended for use within an organization by employees seeking information held internally, in a variety of formats and locations.

WHY ENTERPRISE SEARCH?

It’s an oft-cited fact that the volume of digital information doubles every two to three years. In the meantime, the variety of content, both in terms of form (e.g., blogs) and type (such as Flash and other multimedia) expands yearly. Finally, a growing number of people have become everyday searchers, expanding the search community from a former group of specialized information professionals into a ubiquitous group of employees who have diverse needs and expectations.

Enterprise search vendor messaging speaks directly to these challenges, but you should be forewarned that many of these expectations may go unfulfilled. Unfortunately, the ease of use of public search services contributed to a widespread but mistaken perception that search is easy.

However, the typical enterprise search system does not contain “all” of an organization’s content. Limitations include considerations of security, inability to integrate specialized content (such as Flash files or images), difficulty reconciling structured and unstructured content, and the sheer cost, time, and difficulty required to incorporate diverse content repositories.

Nevertheless, enterprise search systems can make immediate and direct contributions to the financial health of an organization. But – and this is an important but – careful planning, project management, content cleanup, and budget controls are needed. Enterprise search, like a handful of other organization-wide applications, is always complex and often more expensive than planned.

CORE FEATURES OF ENTERPRISE SEARCH

Almost any search-and-retrieval system that describes itself as suitable for “enterprise search” offers these core features:

1. The technology to take a document, index the words in that document, and configure that index such that a user can search it.
2. Administrative controls to allow an administrator to index certain documents and not others, and adjust the system so that the speed of indexing, query processing, relevancy ranking, and myriad other attributes can be “tuned.”
3. Logs to record user behavior; for example, the number of queries, the number of“hits” a document received, the most frequently used search terms, and dozens if not hundreds of other measures.

Beyond these three baseline features, the options and variations of features are extensive. In some cases, the add-ons are more complex than the search engine itself.

WHAT SHOULD I CONSIDER WHEN PICKING A SYSTEM?

Contrary to what you may have read, or what’s been “scientifically calculated” to the hundredth decimal point or plotted on a consultant-approved four-quadrant graph, there is no "best" software product. At CMS Watch, we don't believe in leaders and laggards. Neither should you. The best software for you is the one that best matches your needs — your budget, scope, and type of project — in short, the one that fits best for your content and search scenarios.

If you've been looking at search tools at all, you've surely noticed that vendors (and many consultancies, and some analyst firms) tell very different stories. Some try to answer the ubiquitous question, "who's leading?" You should in turn ask, "leading at what?" If one thing has been clearer than all else in our decade of evaluating different content technologies, it's that different scenarios favor different products.

Finding a good software fit with your particular business objectives (in our examples below, an enterprise search engine) involves a few things:

First, do some internal scoping and analysis work before deciding whether you need to buy more technology at all. You may be able to tune or extend existing search technology, or clean up your content to help search work better. There’s a chance that deleting or archiving content, adding meta data, or tweaking your search index might solve your search problem. A little analysis here can save you a lot of time and money later.

Second, outline the scenarios that aren’t currently being fulfilled by your existing technology set. Your CFO can’t pull up all the financial reports from a year ago? Your call center reps can’t answer customer questions in less than 5 minutes? What are those customer questions like, and what does that call rep do to find it? We outline general search scenarios we’ve seen over the years in our Enterprise Search Report. It doesn't matter whether you employ these scenarios or not; the key thing is that you carefully outline what you're trying to achieve with which types of content.

Third, figure out which type of technology will provide the biggest near-term value. Knowing what type of scenario(s) you're addressing allows you to begin to isolate vendors who potentially hit that sweet spot. You need to tell a “testable” story, one you can give to the vendors you’re considering so they can show how they fulfill your needs with their tools.

Next, it’s show time: have a proper “bake-off,” where the vendors show you how their tools can fulfill your needs. They shouldn’t be dazzling you with canned demos, but showing you what they can do with your content and for your situation. This is also the opportunity to know the people from the company who would work for you. Don’t sign a contract until you do. Remember, the vendor selection process is like dating. You want to really get to know who you’re dealing with before you make it a long-term relationship.

Finally, iteratively test before making your final selection. You should never select a product without having the chance to use it and test it yourself. Much like you wouldn’t use a Honda Civic to drive a Formula 1 race course, you wouldn’t use a complex enterprise search engine when all you need is a simple departmental search. Yet most people spend more time selecting a €30,000 car than they do a six-figure or potentially multimillion euro software package. At least people take the car for a test drive. Very few enterprises do this with software. Don’t make the same mistake.