So, someone in your organization has put before you, the CEO, what he or she thinks is a great business case for an AI project.
How do you know if it actually is a good business case that you should approve, or one that should be rejected?
Here’s a quick guide that will help you make that important decision. You can share this guide with anyone who is going to be putting AI business cases together for you in the future, to make the review process better and more efficient.
Vision and strategy
First and foremost, does the AI project align with the vision and strategy of the company? It absolutely must. If it doesn’t, the business case before you simply is no good, no matter how cool the technical details or algorithms may sound.
Is it a small step towards big improvements?
AI projects are inherently risky, because you are dealing with randomness – in data, outcomes, predictions, etc. Therefore, it is a good rule to break down your overall AI efforts into small projects. But each small project must fit into the bigger picture of where you want to go with AI strategically. Don’t invest in small AI projects to learn or “try” AI – that is not a path to success. Rather, that can just produce a whole lot of stuff that adds up to nothing.
Instead, do small projects that clearly add up to big steps toward your strategic objectives. That’s a recipe for long-term success.
What value will be created?
The business case must make it clear which values are going to be created. Typically, this falls into increased revenue, reduced cost, or increased customer satisfaction.
It can be tricky to relate the planned outcomes to monetary value. But if this is impossible or any value added is very vague, you are probably better off not approving the business case.
In the best case, the net resulting value can be given as a range from X (worst case) to Y (best case) dollars.
How will success be measured?
Closely related to the previous point, a good business case defines indicators of success. The best are leading indicators, meaning you can keep track ‘online’ if the needle is moving in the right direction. Lagging indicators can be ok too, as long as they directly relate to the values being created.
Which business process is going to be improved?
A good business case is practical and links to specific business processes that it aims to improve. This also makes it more likely that the new AI solution can be effectively put into production “This will improve customer churn prediction” is nowhere near as good as “The will generate alerts when the probability of a specific user churning out significantly exceeds a configurable threshold, and these alerts will automatically be sent to the head of customer success.”
Is it clear which models and algorithms are required?
A good business case presents clear ideas about the general type of mathematical models and algorithms that will be employed. If it does not, that is a warning sign, indicating bigger risks for delays and added cost. You may need to scale back the project and narrow it down to focus more sharply on a specific process and how the solution will improve it.
It is almost always a great idea to start with the simplest conceivable AI model. This creates a baseline against which more advanced models can be compared, to see if they actually perform better. If no such baseline exists, the business case should recommend one be created.
Is it clear which data will be needed, and is it available?
If data requirements are not clearly laid out, that is another big warning sign. In such cases it is likely better to start with a smaller project to investigate the data question separately first, if that is the main hole in the business case.
Leadership
Is the project manager experience with AI projects? If not, seriously consider getting outside help. AI projects are not like other projects, such as software development. A great software development project manager does not necessarily handle AI projects well.
If this skill set does not exist in your company today, you should start building it. It may be a good idea to bring in outside expertise to both help with near-term AI projects, but also to help you build your internal AI capabilities – which you are absolutely going to need, long term.
Resources and capabilities
Is every technical challenge matched to a resource – internal or outsourced – that is able to tackle it? Like with project management, AI projects require particular skills that are hard to come by. You will need machine learning engineers, data engineers, data scientists, etc – depending on the exact nature of the project. Once it goes to production, you will most likely need infrastructure experts with experience productizing AI solutions.
Again, bringing in outside expertise can be a good idea – but I strongly recommend that part of the scope then is training of your own people.
Another resource question is, which other projects or tasks will suffer while this project executes? Unless you have a dedicated team for AI projects, you’ll need to consider which other business functions will struggle as they lend support to the proposed project.
Is your AI organization mature enough?
As stated above, it is better to break AI down into small projects. However, as your AI organization and capabilities mature, the definition of “small” can change. Make sure the proposed project fits with this maturity.
Are the right domain experts in the project group?
To succeed, AI projects need domain experts on the team. For example, if you are developing an AI for optimizing inventory management, the team absolutely needs to include at least one experienced inventory manager. The domain expert needs to understand enough AI to be effective. If that is not the case, then some training is recommended before the project is launched. I recommend that you build an internal training program with the aim of giving a substantial part of your total staff a practical understanding of what AI is and how it can be leveraged for your business domain(s).
How is it going to work in production?
A good business case shows how, if successful, the end results of the projects would be rolled out to production. I’ve seen many AI projects where the models, algorithms, and data would be there for proving some point, but where it would not be of value in production.
For example, maybe you can get historical data for something, and build a model on that. But in production, you would need forecasts for that same data. If those are not available, the project can’t be put into production. So always ask, “how is this actually going to work in production?”
Ethical concerns
The business case should include considerations of GDPR, potential ethical issues, especially if the resulting system might end up showing bias towards certain groups of customers or users. Ideally, testing scenarios are defined that would verify that these issues won’t be a concern.
What will it cost and what is the ROI?
Like with value creation, a good business case gives a range of costs, from the worst to best case. AI projects should report frequently on progress so that significant deviation from the original cost can be detected early. Again, AI projects are inherently risky, so they require extra monitoring.
If surprises don’t pop up, you should be surprised.
Ways to scale back
If your conclusion from the previous steps is that the business case isn’t great, there are two related ways whereby you could turn this into a meaningful project.
- Use a simpler algorithm/model
- Scale back ambitions
Executing a less ambitious project may still yield a lot of highly valuable information about important questions: how good is our organization at executing this type of project, where we able to move the needle at all towards the objectives, and so on. Based on that, you can then re-evaluate the original business case and take another step forward.
If you’d like my help with designing, evaluating or implementing AI projects, get in touch with me. I would love to help you.
You can also schedule a free, no strings attached, consult here, or just call me on +45 51 64 66 91.