Business Analytics-Why Search is Not Enough

Posted: 2016-09-15

Author: Admin

Due to the popularity of internet searches, many businesses have attempted to adapt the search paradigm to tame their own unruly document clutter. Knowledge workers can be much more effective in performing their jobs, they believe, if all the pertinent information for accomplishing the job is at their fingertips.

Searching on the internet, however, doesn’t necessarily yield the desired information; rather, it provides links to documents that might contain the desired information. Internet search deals with documents which are relatively homogeneous, in the sense that they are all uniquely addressable web pages. Furthermore, the search relies on document tags (keywords), so the relevance of a query’s results depends on how appropriately the tags characterize the document’s contents.

Basic search has evolved and become more sophisticated, particularly in the areas of interpreting search requests, indexing documents, and presenting results as interesting visualizations. The question, however, still remains: are these developments enough to make the knowledge workers more effective at their jobs? Search can certainly save time in getting to the relevant documents, provided the right search query is posed. The question of knowledge worker effectiveness becomes lot more significant when search enters the realm of business analytics.

The Search Process

The search process begins with a user articulating a search request, and ends with the user receiving results delivered by the search mechanism. An auxiliary process continuously digests new documents, making them available for searching.

User Articulation of Search Request

“Has anybody worked with Smith & Company? I would like to know their experience with them” or “what’s the latest return policy for electronic products?” are typical examples of information that a knowledge worker might be seeking in the context of their job. In order for users to express their requests in a natural way, the search system must be able to understand the user’s intent, and then translate that intent into a formal query that can be executed by the underlying search engine. While we are far from achieving this goal, techniques such as auto-suggestion and use of natural language processing technologies have made it easier for users to express their requests.

Presentation of Search Results

Have you ever been frustrated with a search when you’re unable to get the results you’re looking for, despite repeated attempts? After trying one request after another and scanning through many documents, you still feel lost. Perhaps what you’re looking for may not even be there at all, but you have no way of knowing for sure. Other times, you get lucky and find what you want instantly.

Does a smarter presentation of search results improve user search experience? The simplest approach is to present search results in order of their relevance, while advanced techniques involve the inclusion of snippets from documents and other semantic content that may help users avoid a full scan of documents. In some cases, presentation may include data visualizations.

Digesting Documents for Search

A document is as good as lost if it isn’t classified properly. Business documents can be quite large, so for business users, it is not just a matter of getting to the correct document; they want to get to the specific part of the document that contains the relevant information. This means that for a document to be amenable to intelligent search, its contents must also be indexed properly. Alternatively, the parts of a document may be marked up during the digestion phase, and then extracted during the execution of the search query. Full-text search and other text processing techniques have made it possible to reach to the depths of any document.

The Lure of ‘Search’

Part of the attractiveness of ‘search’ is its simple appearance at the user interaction level, and the ease of use for the end user. In contrast, finding information from documents and data through other mechanisms such as BI (business intelligence) appear much more laborious. Thus, it is very tempting to try to extend the search paradigm to the domain of BI and Business Analytics. Considering the skills and time needed to produce a new BI report, search appears to have the advantage. So-called ‘self-service’ BI attempts to eliminate the time lag between when information is needed and when it is delivered, and analytical search plays a pivotal role in the realization of this goal. In traditional BI, there are two roles involved: a BI expert, and the end user, who receives the information gleaned by BI. While search-based self-service relieves some of the pressure on the BI expert, it shifts that responsibility to the end user. In order to be effective, the end users of BI information must have at least a partial understanding of the BI process.

Why Search is Not Enough?

Search is not a panacea for discovering insights from data. It is not a substitute for business analytics, by any stretch of the imagination. Business analytics is all about supporting data-driven decision making; this includes not only searching data for what it explicitly represents (descriptive analytics), but also what it implicitly encodes: knowledge that can be used to predict the future (predictive analytics) and also suggest preventive or coping measures to deal with the predicted future (prescriptive analytics). Furthermore, to assist users in making effective and timely decisions, a business analytics system must reduce the cognitive dissonance between the decision-makers and the business analytics system (cognitive analytics).

When it is well integrated in a decision-making process, search can be very useful–but this integration is not a straightforward plug-and-play activity. A deeper integration of search within the decision-making process is a complex endeavor, and has a profound impact on all aspects of the search process itself.

About the author:

Dr. Rana is founding CTO at diwo. With more than 35 years of experience as serial entrepreneur and executive leadership, he was also the CTO of CommerceOne, and a core team member for developing strategy and launching distributed systems management initiatives at IBM.