What’s Wrong with the BI Dashboard?

Aside from being a Top 50 Thought Leader in Analytics, tell me about your background Neil.

 

I am a mathematician, an actuary in the R&D group at AIG in New York. That’s where I learned about modeling and analytics. After ten years, I started Hired Brains Research, a consulting and implementation firm in Data Architecture, Analytics, AI, Data Science and organizational change. My current portfolio includes data warehouse modernization, AI Last Mile and Complex Supply Chain analytics.

I’ve written more than fifty articles on the subject on Diginomica, and the author of the foundational report for the Society of Actuaries, “Ethical Use of Artificial Intelligence for Actuaries.” With James Taylor, I was the co-author of the first book on Decision Management, “Smart (Enough) Systems” The truth is, I have always been focused on how to help organizations improve their analytical capabilities.

 

In a recent article – Will BI Survive? – you raised some interesting points about BI’s relevance in today’s environment and suggest that the user interface is a significant barrier to adoption. Let’s talk about decision-making, the BI user interface and how AI and NLP are changing the game.  

For starters, you state that with data growing in scale and complexity every day, and the demand for actionable insights getting more urgent, BI dashboards actually make it harder to make data-driven decisions.

 

(Neil) The simple answer is that you can’t solve today’s challenges with yesterday’s tools. Dashboards can make certain insights more accessible and useful for end users, but they have limits. BI dashboards were designed to deal with the reporting backlog in IT and even though it’s added layers of functionality over the years, it’s still primarily a tool to report what’s happened. Every organization needs to monitor, measure and manage past performance to determine whether it’s on track to achieve its goals. There is nothing wrong with that, it is just not enough – especially in light of today’s data challenges.

The primary problem with dashboards is that someone with specialized skills and knowledge has to build them, maintain them and keep them coherent with changes in models and data. Developers create dashboards, pre-defined, subject-oriented visualizations. Decision-makers are dependent on IT or a data scientist to answer daily, ad hoc questions for them. This gap creates a bottleneck that slows time-to-insight for decision-makers and ultimately time-to-value for the business. Typically by the time a report is delivered it’s too late to change the outcome. Organizations today need tools that empower business users with information so they can take constructive action while there is still time to change an outcome. 

 

You cite research that the most common complaint is that BI tools are still not easy to use and that the reason is the interface.

 

(Neil) The GUI is just too limited a metaphor to fully exercise the needed capabilities of analytics. A GUI interface does not translate to ease of use necessarily. Graphical interfaces can be so complicated the users aren’t sure what will happen when they click on something. Or terms are used that are not intuitive. Often,interface designs are so crammed with objects, the screen is a bewildering arrangement of things. But most importantly, a BI GUI requires you to understand its semantics and the underlying structure of data sources: data elements, filters, metrics, dimensions, conditions have to be selected accurately. A really competent BI system is multidimensional and non-linear. How do you represent that with a bunch of buttons? How intuitive can it be if you need a three day class to get started and BICC (BI Competency Center) to build the structures for you.

 

BI platforms and data discovery applications are supposed to launch insight into action, informing decisions at every level of the organization. But they fail to do this. They can not close the decision-making loop. After purchasing BI tools, many organizations are left with costly investments that create inefficiencies, have low adoption and exclude the vast majority of employees who could benefit from those operational insights. Now that’s what I call a lack of ROI. 

 

The path to ROI with BI is better decisions, not better dashboards. What users want is a simple interface that allows for rapid fire question and insight generation. Going forward, nirvana would be an experience that doesn’t require users to exit their workflow and having all the insight you need – including next best actions or recommendations – readily available  in a single interface. Not having to exit the dashboard allows the decision-maker to be more responsive to business challenges and drive better business outcomes.

 

Over the decades, the BI market has gone through several cycles of innovation with the latest focused on the infusion of AI and ML algorithms to automate the analytics workflow in support of decision making and natural language processing (NLP) interface. Do you see automation and conversational interfaces becoming the new interface to enable a wider range of people in an organization to leverage data more effectively for all the big and small decision-making that is part of their respective role?

 

(Neil) Conversational BI uses natural language for getting the analytics in real-time, not just the past, to make timely decisions. It shifts the workload from the user to the software and provides a single place for accessing and working with all the data. A fully functional NLP interface would have to provide a real conversational interface, not one requiring you to learn the precise way to phrase a question or to use pre-assembled phrases. Also, the NLP has to be able to communicate to the backend to resolve your questions. The system will  turn your questions into effective queries. For example, “What’s the historical trend of our domestic car market?” “Normalize that for outlier economic events in time. “Compare that to all cars sold in the US from any source.” “I want that in tabular and chart formats.”

Greater use of AI and machine learning in preparing, cleansing and relating data yields a faster time to value. AI is also used to generate insights without the use of a data scientist and eliminate bias and error from the analysis process. You can go to The Last Mile in AI Deployment for more of my thoughts on AI.

 

In the blog entitled – Want better decision-making?  you say that “informing people with analytics isn’t worth a bucket of spit (as they say in Texas) if you can’t take it all the way to presenting a solution – and business users want to see the insight/answer in context of their particular roles.” Today most analytic tools cannot support this kind of problem-solving.” This seems like the classic “last mile of analytics” challenge, where there is a gap between having insight and being able to take effective action.

 

(Neil) This is the big challenge for BI, to step out of its dashboard role and participate in the decision-making process. The overly-simplified model prevalent in the BI industry is that getting better information will yield better decisions – but it doesn’t. Decision-making is not a linear path, so next generation tools must facilitate fast iteration – this step is critical to developing confidence for a given decision. If you want to use data to run your business you need interactive, navigable data tools that work the way people – across the business – consume and make sense of data. If you – an analyst, a business line manager, an executive – have a question about data, your BI tool should have the answer contextualized, easily consumable and action by any user. But today, most BI tools fall short of this goal. A modern approach empowers everyone in the organization to ask and answer questions related to their role without relying on IT or a team of data scientists. 

 

To wrap up, the business intelligence market is long overdue for a fresh approach that takes advantage of today’s innovations (AI, NLP, cloud-scale data warehouses, data lakes, new pipeline tools) to rapidly turn data into insight and then support a user in taking action that delivers a tangible business outcome. It’s no longer about providing reports or dashboards but accelerating decision-making to drive business value. 

There is an emerging category of analytics called Decision Intelligence that aims to reduce the existing bottleneck or “decision gap” that comes with legacy approaches to business intelligence. I think this is finally the technology that will deliver on the nirvana of BI making it agile, providing the user with a seamless and more natural experience of asking/answering questions in context of the business moment and delivering the right insights and recommendations in a way that informs their decision-making to drive measurable business outcomes. And it would reduce a decision-maker’s reliance on IT teams or data scientists to answer daily, ad hoc questions.