Data In Wisdom Out
I am often asked by the curious, what do we actually mean by “wisdom out”. Is it just a marketing ploy or is there something more to it? How do we get wisdom out of data? And, what is this so-called wisdom good for, anyways?
In this article, I am sharing my thoughts on the concepts of data and wisdom in general, and how it inspired the architecture and foundation of diwo (“data in wisdom out”) platform, in particular.
Being the first employee of a new startup about three years back, I had the unique privilege of gazing into the crystal ball, and project how the enterprise software space might look like in three to five years time when we are fully ready to enter the market. This exercise was essential in setting up a vision and a development road map for the company. Furthermore, it was also important for me personally to take on a challenge that expands the frontiers of my intellectual and entrepreneurial journey.
At that time, “Big Data” was the most trendy topic and IBM was feverishly pushing the concept of cognitive computing fueled by the famous jeopardy challenge win by their meticulously hand-crafted custom made intelligent question answering system, called Watson.
I concluded at the time that being a late entrant in the big data space, it won’t be wise for us to count on riding the big data wave. Big Data meme was subsequently superseded by IOT, big data analytics, and data science memes among others. More recently, both big data and cognitive computing seems to have given way to the latest hype around “Artificial Intelligence”.
Having a comprehensive background in distributed computing, artificial intelligence, and distributed problem solving, cognitive computing meme was a natural hook for me to gravitate toward. Question answering systems like Watson represented a tip of the iceberg. I saw an immense potential in borrowing from the progress in cognitive science and cognitive psychology and applying those concepts to develop higher layers in the enterprise software stack to make it easy for business users to deal with complexity caused by information deluge and to reduce their cognitive burden in their routine decision making activities.
Inspired by systems thinking approach, the first conscious effort was to contrast ourselves away from data first and silo-ed approaches. Instead of data first approach, we subscribed to ‘business first’ approach. A business is in the game of creating and capturing value. Data is useful only and as much as it participates in generating business value.
Data is simply a trace of historical events captured by tracking and measurement mechanisms. Data, in itself, has no intrinsic value but can potentially participate in the creation of value. Unbridled data collection is a foolhardy pursuit and must be tampered by business context. Business goal dependent, perception systems with attention filters must be designed and applied at the very first contact with data to retain only what might be useful and to discard the rest.
To adopt a business first approach in practice, we advocate starting from the top and understand what business value a company is striving to create, how do they go about creating it, and what hurdles they have to overcome in the value creation process.
Instead of developing a raw analytics capability by climbing up on the analytics ladder, from bottom raw data layer to descriptive, predictive, prescriptive analytics, and so on, we took a fundamentally different and an intelligent systems approach.
We imagined an enterprise to be a continuously evolving intelligent system that continuously senses its environment, makes informed choices, and acts upon these choices to influence the environment in its favor in a tangible and quantifiable way.
This cognitive view of enterprise, required that we go beyond the traditional notions of data, information, and knowledge. For this purpose, we introduced the concepts of awareness, intuition, and wisdom in diwo architecture while relating these to the concept of data and knowledge.
Data is the raw information that one receives from external sources. We keep it confined to the data staging area for subsequent reference and delete it when it is no longer relevant.
Knowledge is what is obtained after applying perceptual attention and feature extraction filters and transformers.
Awareness layer is about understanding the value and power of knowledge one has accumulated or has access to. Knowledge is power, they say, but one must be aware of this for the knowledge to be of any value.
A knowledgeable person or a business is one which has access to a lot of knowledge. But one who understands the power of this knowledge is said to be aware. In diwo, we implement the notion of awareness by a time-sensitive opportunity sensing module.
Once we are aware, we move to realize the higher level cognitive capability of wisdom.
Wisdom is knowing when and how to harness the power of knowledge once you are aware of it.
In diwo, this is achieved by an opportunity exploration module which provides a guided cognitive decision-making capability built around the concept of continuous business optimization.
Finally, few word about intuition! By intuition, one generally means unique human ability to arrive at useful decisions without quite knowing how those decisions were arrived at. In that sense of the term, current AI systems (deep learning neural networks) can be called as having intuitive ability. It is not that the AI systems can’t be intuitive, the real issue is that we don’t want AI systems to be intuitive and demand explanation from them to compensate for our lack of trust.
This concludes my brief explanation for the phrase “Data in Wisdom out”.
About the author:
Dr. Rana is founding CTO at diwo. With more than 35 years of experience as serial entrepreneur and executive leadership, he was also the CTO of CommerceOne, and a core team member for developing strategy and launching distributed systems management initiatives at IBM.
Why AI Models Eat BI Dashboards for Lunch
Many organizations devote precious time and resources developing predictive AI/ML models to determine what might happen in the future. The ultimate goal of these models... + 2021-03-26
Why More BI Dashboards Don’t Mean Better Decisions
A typical analytics dashboard, whether it’s viewed as a pivot table in Excel or within a business intelligence (BI) platform, is often the primary source... + 2021-03-17
How banks can leverage AI to better manage pandemic relief programs
The recent massive economic shifts due to COVID-19 have driven up risk for the vast majority of financial services institutions. Customer sentiment, spending and behavior... + 2020-07-28
Financial Services: Preparing for the Looming Credit Crisis
The COVID-19 pandemic has significantly affected financial institutions – slowing the growth of loan originations, increasing credit costs, contracting economic activity, and causing record levels... + 2020-05-15
Can retailers find strategies that balance short-term recovery with long-term sustainability to lead them out of the COVID-19 crisis?
Retail and fashion business leaders are currently focusing on business continuity (“keep the lights on”) and crisis management (“sell the right stuff”), but soon they... + 2020-05-05
Transforming Business Decisioning in the Pandemic era is more crucial than ever.
Our human tendency toward incremental thinking limited us from foreseeing how “a few cases of the flu” would balloon into the impact we see today.... + 2020-05-01
Surviving COVID-19: How can Retailers free up working capital tied-up in inventory?
Among the many sectors affected by COVID-19, retail will be one of the hardest hit. The sudden decline is attributed to country-wide store shut-downs and... + 2020-04-23
Don’t let your shiny AI models lose their luster!
Businesses have made significant investments in building AI/ML models in recent times. While models are increasingly driving significant operational efficiencies and differentiation for businesses, they... + 2020-04-22
AI Modeling in the time of Covid-19
Companies of all sizes are facing unprecedented uncertainty and challenges due to the global impacts of COVID-19. It has created a major systemic shock to... + 2020-04-14
How to derive value from your AI investments with Decision Intelligence
Ultimately, a company’s value is measured by the sum of its decisions. In order to succeed, the organization must make and execute decisions—across all levels... + 2020-01-24
Why isn’t my enterprise getting value from AI at scale?
Why do most AI initiatives for business fizzle out? Why do so many teams’ best efforts to develop or deploy new algorithms or predictive models... + 2019-11-21
Here’s why you’re probably losing the AI race
As the AI arms race is becoming more heated, more organizations are looking to beef up their competitive advantage... + 2019-08-01
Data In Wisdom Out
I am often asked by the curious, what do we actually mean by “wisdom out”. Is it just a marketing ploy or is there something more to it?... + 2018-10-31
Is Data a source of value?
We have always been exposed to natural and man-made events and have wondered and been impacted by their outcomes... + 2018-10-16
The Future of Decision Making: Human-AI Symbiosis
When we have to make an important decision, we face numerous challenges: uncertainty, complex data that’s difficult to interpret, competing priorities... + 2018-09-20
Digital Transformation- a $900B failure this year alone
Digital transformation has become a major priority for most organizations in some form or another, but for many, it’s proving to be quite the challenge... + 2018-09-20
So What’s AI’s Dirty Little Secret?
One could read the massively hyped claims of the trillions in productivity gains and then how far AI still has to go for real-world application... + 2018-09-20
How AI can rescue your BI “situation”
Even with a few credible upstarts in the past couple years, Self-Service BI still seems to be dominated by some large players that also require some very large and "very ongoing" commitments... + 2018-03-20
Business First! – how diwo aspires to flatten the knowledge pyramid
We can all agree on the fact that we are sitting on an unprecedented volume of data, and it is continuing to accumulate exponentially... + 2017-09-18
The Growing Market Impact Of AI
The future is always uncharted territory, and in the hype that currently sorrounds AI, with its ambiguous... + 2017-09-18
AI: How It Will Redefine My Job?
In the past few years, there has been an explosion in innovation and interest around Artificial Intelligence... + 2017-09-18
Wait Less, Sense More, Act Fast!
“If I had more legs, I could walk faster. If I had more hands, I could handle more, and if I had extra senses, I could respond wiser” – did you ever wish that?... + 2017-09-11
Unlocking the holy grail of UX design: bias-free user feedback in real time!
The potential of measuring emotional feedback from users is an exciting proposition in many fields, not least because it offers the possibility of unlocking... + 2017-07-25
What’s the Deal With AI Connectors
Artificial Intelligence (AI) is changing our lives, and these days feels more omnipresent than ever before. From Siri to autonomous cars... + 2017-07-25
Business Analytics-Why Search is Not Enough
Due to the popularity of internet searches, many businesses have attempted to adapt the search paradigm to tame their own unruly document clutter... + 2016-09-15
Drowning in data? Still no excuse for inaction!
Wouldn’t it be fantastic if we had a total understanding of the laws governing the reality around us? Imagine for a... + 2016-09-15