As the amount of data accumulated by businesses continues to grow, one of the often confounding questions asked is “How do we turn it into insights and then profits?” Sooner or later, most businesses find themselves confronted with analysis paralysis and become unable to extract meaningful insights or monetary value from the data that should be fueling their growth.
I see three primary causes for this dynamic: First, businesses that do not have the right data management processes in place for capturing, cleaning, tagging, storing, and analyzing their data, as well as providing its access to those who need it most — their employees — will forever be hampered in their quest to achieve meaningful insights and revenue. Analyzing bad or unclean data leads to misguided business decisions and initiatives that can have more of a detrimental impact to the business than had it not attempted to harness its data at all.
Second, businesses over-invest in technologies that serve only a single purpose, such as marketing analytics, and allow individual business units to purchase their own Big Data analytics solutions. This creates a hodge-podge infrastructure of incompatible technologies that cannot be used across the entire business and that often work at odds with one another. If one or more data-driven business unit excels beyond the rest of the organization or begins to work with cross-purposes from the others, it can unknowingly cause serious disruptions in productivity, service, and performance.
Third, companies may not have a clear vision for what they want to achieve from their data, which leads to wasted resources, project abandonment, and discontent with their investments.
Tackling these three challenges will put you on the right track, but there are many other less obvious factors or misconceptions that can impede data effectiveness and cause analysis paralysis. Here are three guiding principles to keep you moving toward a Big Data solution that works for your entire organization.
1. Let business objectives and outcomes guide your decisions.
The sensible approach for choosing a Big Data solution would be to identify the scope of all the business challenges you wish to address — throughout the entire organization, not just one or two functions — and let that guide your purchasing decisions. Many enterprises quickly lose sight of the forest for the trees, focusing on the promise of what Big Data can do, as opposed to the value-based needs across the organization. Remember that analytics is a means to an end, or in this case, value. The key element to extracting value from data comes from applying analytics to sound business rules that lead to critical human decision making.
Without underlying objectives or practical guidelines that address problems, analytical output simply creates more questions and queries that can tie up both the data scientists and business executives, hampering progress. Take a FICO score, for example. Without a related decision or objective, it’s merely a number — an output from an analytical model. Its value becomes apparent only when an underwriter applies rules created by an organization to meet certain business objectives to help make a decision — in this case whether or not the organization should lend money, how much, and at what rate.
2. Match the right algorithms to the right problems — with realism.
With so many different technologies and flavors of analytics available on the market, we can easily see how companies can over-invest or invest in the wrong solutions. Why purchase deep learning and neural net technologies — as cool as they are — when simple linear regression models will do? And while artificial intelligence and machine learning are also cool to work on, for the average enterprise, the highest reward option may be going from a rules-based system to one that relies on simple regression models for predictions. For example, a marketer who blasts email campaigns based on age, gender and ZIP code using regression models to rank order the prospects and customers may help increase targeting 3–5X.
In our experience, only 10% of use cases find real value in advanced AI or machine learning techniques, while the other 90% can use simple analytical models to get to the desired business outcome. Determining propensity and purchase intent scores, for instance, can be powered by simple models, but more sophisticated examples, such as fraud prevention or insurance claims anomaly detection, would benefit from AI and machine learning applications. So it’s important to think about the types of problems you’re trying to solve and then choose which flavor of analytics is best suited for each.
And finally, even if you believe that you really could benefit from more advanced techniques, you may not have enough data to implement them. Deep learning, for instance, is not possible for the average enterprise. Unlike Facebook and Google, an average enterprise does not have enough streams of data to apply some deep learning algorithms. To explore deep learning, Google fed a billion images of cats into a deep learning algorithm, which used the images to learn how to identify cats on its own. In a commercial setting, however, such rich data sets may not exist, or if they do, may not be easily assembled. For deep learning to work in commercial enterprises, there needs to be an application requiring a large and steady flow of data with a constant feedback loop for self-learning.
3. Consider all costs — not just technology.
The cost to build Big Data systems in-house versus using a platform-as-a-service provider is going to weigh heavily on decision makers. While hard costs are easy to calculate, it’s the softer or otherwise hidden costs associated with your decision that might hurt your bottom line more in the long run. These can include the cost of resources, time, possible missed opportunities for business impact, or worse, dissonant results that degrade processes, require costly fixes, and cause possible revenue losses. Consider the case of insurance companies that constantly audit incoming claims. When the models used to score each claim for anomalies result in a high rate of false positives, it can waste labor reviewing scored claims. In such a situation, upgrading to a sophisticated machine learning model may well be worth the added expense of better technology or reliance on expert resources.
Conclusion
In short, Big Data solutions are generating some of the highest ROIs for businesses since the advent of the Internet twenty years ago. However, a Big Data solution does not always need to be big in scope to deliver big business outcomes. With the proper planning, goal setting, and vetting of technologies, companies can rapidly deliver the desired insights and financial benefits from the data that resides within their organization through careful investments. Failing to do so brings risks beyond run-up expenses in high technology costs, wrong choices of products, or burdened resources and processes; it also creates a lack of, or incapacity of, decision making.
Companies are predisposed to analysis paralysis. They’re averse to risk, facing the unknown, and know the feeling of inertia when change is forced upon their employees all too well. Analysis paralysis in terms of choosing a Big Data solution is an unavoidable symptom of a bigger, unavoidable paradox: To need Big Data analysis means a company is large enough and ingesting enough data to require it. Yet larger companies struggle with this in part because they’re large. It’s never easy to turn a large vessel in a new direction, but large vessels must go toward Big Data analytics to remain competitive. In addition, analysis paralysis is — ironically — caused by the fear of missing out on the promise of new technologies. Lofty expectations, rushed investments, misguided implementations, or incomplete insights actually end up impeding progress and causing companies to miss out on the benefits that ought to be brought by timely and measured implementations of Big Data and predictive analytics.
Want to learn more about how to drive long-term success with analytics? Check out our white paper "Delivering Big Data Success with the Signal Hub Platform."
Laks Srinivasan is co-COO at Opera Solutions.
This article was originally published at InsideBigData.