/ Posts / 7 Reasons Your Machine Learning Project Will Fail
on November 21, 2019
I always tell our consultants that before they do something for a client they should ask “why” 7 times. Why do you want that pie chart? Why does it matter? Why will it help you make a decision? Etcetera. It’s a great way to ensure our projects provide meaningful insights and drive decisions that improve the organizations we do business with.
Given that, I thought it was appropriate to list 7 reasons why your AI/machine learning will fail if you don’t ask your own “7 Whys”.
There’s more to life than time series. Time series forecasts are inherently bad at picking up the impacts of other dimensions. We get better results by clustering then running a time series. Or, better yet, run a 1X, 2X, 5X, 20X demand forecast. X can be hours, days, weeks, etc. The impact of holidays, product type, etc will be much more insightful; plus we’ll be able to forecast with less historical data.
What’s worse than picking the wrong type of model? Trying to do too much with a single model AND trying to do too much, period. Again, there is no one-size-fits-all approach. Trying to forecast apples and cats, you better have different models. Use cluster analysis to group like things first. Trying to forecast sales of every SKU? That’s also a waste of resources. Find the ones with the most impact first -revenue, losses, spoilage, inventory, turnover, etc. Ask yourself why you’re forecasting that product that counts for 0.001% of revenue.
How accurate is your forecast now? Do you think you’re going to go from 80% accuracy to 99%? You’re probably not. Figure out what it really means to have a more accurate forecast. And figure out what a percentage increase in accuracy means to the company. It’s always good to be more accurate but some people think that if they don’t get 99% accuracy then it’s not worth it; that’s almost never true. Figure out what your target level of improvement is and shoot for that.
It is important to gauge the current state of your organization’s data capabilities before planning your machine learning project, too. Machine learning does not happen overnight and sometimes it is better to tackle data preparation and BI tasks before beginning any machine learning projects.
Every company has bad data. Every. Single. One. Fix it. Because the only thing worse than no data is bad data. Missing values. Dirty data. Inconsistent data. These are all problems that we can address but the wrong time to address these problems is when the model is being built. It ties back to expectations; if you give me dirty data then I will give you bad results. There are loads of tools we use to prep data – some you may have and some you may have never heard of. Tools such as Trifacta can be used to clean and transform data, as well as more traditional technologies such as SQL views and python can be used too.
Are you forecasting water sales but you don’t have temperature data? Are you forecasting capital equipment but you don’t have opportunity data? Are you forecasting preventative maintenance but you haven’t had any failures? We need to find a “signal” in the noise. If you can’t give us enough data there might not be a signal. This is why it’s so important to have an understanding of the business: to make sure a potential data set isn’t being overlooked.
Referring back to the notion of bad data, not including data because it is dirty is not a viable adjustment in many cases. Deleted data can have a drastic impact on a model and in many cases just needs some additional cleaning.
So, you’ve created a model that very accurately predicts X. Now what? People keep asking you for more. And more. There’s no way to keep up with the requests from the “business” side of the house. Unless you have unlimited time, unlimited budget, or, the right tools. We’ve found that using a tool like DataRobot can help you build lots of models quickly and keep up with that demand. And you don’t need to hire more and more data scientists to do so.
If it’s not repeatable you won’t get results. One-off predictions will help you understand you business but they won’t help your people do their job better. What you need is a system that integrates your data, makes predictions, and, gets the information into the hands of the people that can take action on it. Imagine if Netflix emailed you once a month with suggested content rather than showing you on-screen what you might like. Also imagine if your doctor had to wait a week to tell you that you were about to have a heart attack. Designing the model and determining how it can go into near real-time production is how you truly build value.
You’ve got the right model. Into the hand of the right people. At the right time. Now your accuracy drifts. What you thought was 92% accurate is now 80% accurate. How do you know? Without a system in place to tell you that things your model is drifting, there is no way for you to know that. Platforms exist to determine model accuracy faster than a data scientist could ever dream of.
by N/A - Dominick Amalraj on October 2, 2020
Learn how DataRobot can accelerate every aspect in the machine learning processView
by John Fitzgerald on August 31, 2020
A fire at a neighbor's house reminded me that the first attempt at solving a problem might not always be the right approach.View
by N/A - Tyler Robinson on August 18, 2020
How to take your Fantasy Football draft to the next level.View
by N/A - Dominick Amalraj on May 26, 2020
Elevate your organization from machine learning capable to machine learning driven.View
by Scott Duthie on May 6, 2020
4 Ways to extend Qlik NPrinting to get more value out of it.View
by N/A - Brian McManamy on April 5, 2020
Are you getting the most out of your Qlik Sense monitoring tools?View
by N/A - Tyler Robinson on March 21, 2020
How can you use data to solve your most critical problems?View
by Wendell Truax on March 16, 2020
Plan your Qlik Sense upgrades more reliably with our extension inventory application.View
by Scott Duthie on January 8, 2020
Pomerol joins forces with non-profit to increase sex trafficking awareness through data analytics.View
by on November 21, 2019
Pomerol Partners and Sense Excel collaborate to “turbo-charge” reporting and analysis for organizationsView
by N/A - Mike Mahoney on November 21, 2019
Couchbase and Pomerol Partners Drive Customer Success with Faster Time to ValueView
by on November 21, 2019
We have partnered with StreamSets to help modernize your data integration efforts.View
by on November 21, 2019
Use these tips to build a self-service analytics platform for your organization.View
by Scott Duthie on November 21, 2019
Pomerol Partners and DataRobot to collaborate on automated machine learning within predictive analyticsView
by on November 21, 2019
Check out the new updates and functionalities for the Qlik Sense February 2019 Release.View
by Kanon Cozad on August 1, 2019
Learn about Big Squid and how Pomerol can help you implement it.View
by John Fitzgerald on December 25, 2016
Leverage K4 Analytics for advanced planning, budgeting, and forecasting from inside your Qlik appsView