These articles about AI and ML project design are intended to address the
real
difficulties
people experience when trying to define and scope their projects.
Project
design issues
are widely regarded as a key cause of project failure. So to have the best chance of
success,
read our tips before you start, and review them regularly during your project.
These articles are organised as a set of questions you should ask yourself about each aspect of your
project. Most importantly, you do not need to be an AI or ML expert to answer the questions.
We also provide a free tool - a kind of Business Model Canvas for AI and ML projects:
Project designer
Using the tool, you can record your answers to these questions.
Who will own, operate and maintain the solution? (Teams or individuals). Can more be done to achieve and sustain adoption of the solution? Who will provide technical support?
There are a number of roles associated with an AI/ML project, even if the result is purely one-off advisory work and isn't a production system.
Maintaining knowledge
Who will be responsible for ensuring knowledge and experience generated during the project is maintained, documented, communicated to those who need it, and not lost?
Adoption
Even after the project, the work (data, or methods) will often have value. Who will take the opportunity to reuse this work in other business areas or projects? This requires ongoing socialization of the project outcomes.
Operation and maintenance
If the project will ultimately lead to a solution in regular use, who will be responsible for operating and maintaining it? People often come to rely on software without necessarily having any guarantees that the service will continue. Who (if anyone) is maintaining the quality of the service or data? Do they have time for that? Who can be called if there is an issue?
Ownership
The project may generate expectations, liabilities and other risks. For example, a business may make strategic decisions based on conclusions from an AI/ML project or solution. Who is responsible for the outputs of the project?
Intellectual Property (IP)
If the business wishes to sell, publish or use the solution in future, are there any legal constraints? Who can sign off on use of this IP?
Who are the users of the solution? How many users are expected? How can you measure uptake and use?
Having developed your project into concrete solution ideas, explore who might be users of a solution and how many users you would expect. These numbers affect technical choices around solution scaling and performance.
In addition, you can monitor uptake and use of the solution, if it is not a mandatory replacement of another process. Voluntary use of any of your project outputs is a measure of success. How can you measure use?
Who - or what - will consume or rely on outputs of the solution?
List the systems, processes and/or users who will consume (use) the outputs of the solution and/or project.
How will the solution be integrated into existing business proccesses and systems? What input and output dependencies will exist?
This step is important to understand both dependencies and integration impacts of the solution or project.
First, look at the input dependencies - anything or anyone that produces input necessary for the system, especially if it continually varies (such as new events). It's also important to capture relatively static inputs such as map, network or inventory data so that it can be updated periodically.
Next, look at the output dependencies - anything or anyone that consumes (uses) the outputs of the solution or project - whether that use is simply new insights or actionable data. Try to include by name any process or system which is affected.
Managing changes to dependencies
Make notes on which dependencies are likely to require changes - either to your solution or project, when new data or data formats become available - and how output dependencies will be affected by changes you make. Are you able to warn of these changes and provide dependant systems time to react?
Consider how to deploy the solution to minimise technical and change risks. Potential strategies include:
Note: Ensure validation is always part of any new model deployment!
How would you rollback and defer deployment if problems are encountered? Plan for regular deployments over the lifetime of a production solution.
Deployment is always a high stakes time for any project, but especially when big changes to key processes are involved, such as new automation or highly data-driven processes. Explicitly outline your deployment strategy now to show you have considered these risks and propose suitable mitigations.
Different deployment strategies can help to mitigate different risks. For example, if your project or solution changes user experience, you may want to use A/B testing or Canary deployment to verify users are happy with the new system.
AI/ML solutions often require regular deployment to manage continunally changing data. This means that you need to have a safe process for repeatedly deploying new models, which may behave in different ways.
The ability to roll-back a deployment without data loss can be crucial. Some deployment strategies offer this; in some systems it may be very difficult to recover user-data from a deployment after roll-back.
Finally, consider what service-level guarantees you are required to provide and how deployment strategy can contribute to, or jeopardize these. For example, if you are required to achieve very high levels of uptime, you may need a deployment strategy which involves no downtime.
How will you document the design, use and other technical details of the solution - where will the documentation live? How will you provide training to users and maintainers? In addition, note the legal status of any Intellectual Property (IP) generated by the project.
The project team will move onto other things, so whatever type of AI/ML project this was, a huge amount of the value is stored in the knowledge that was gained. It's important to plan to retain that knowledge. Documentation is a big part of that.
To make documentation successful, list some key documents that will be produced and maintained (incrementally updated) over the course of the project. Make this documentation part of your project plan and make sure it happens. This might include:
To keep the documentation relevant and accurate, it's important to use a shared, online platform such as a Wiki to produce it collaboratively. The edit process must be hassle-free.
Some technical documentation can be stored in your version control system, but the majority should be accessible to non- programmers. There are a number of free and commercial Wiki solutions, such as Confluence.
Lastly, make a note of the IP constraints on your solution, due to internal choices or due to use of third-party software or data which may create restrictions on commercial use. IP is also part of your project's value - ensure you know who controls it, and what you own!
Make a plan to socialise the benefits of the solution and raise awareness of your successes. How will you build momentum and interest in your project, both now and through to delivery and even after adoption?
Regular communication
Your project will need ongoing support from stakeholders and leaders. It's important to share news about the project - good and bad - so that people are aware of the project and it's potential. You might also discover new opportunities and applications this way! How will you regularly share news? How can you register who's interested?
The value of data
There are also incidental benefits of the project, which you can only exploit if you become aware of them by talking to people. A major one is the value of the data you will produce. You will find it essential to do the hard detective work of cleaning and linking the most definitive dataset possible - many other business functions may well find that extremely useful. Plan to shout about that outcome and let people become aware it's available.
Build investment
When people get regular updates about a project, they become more interested and invested in it. They understand the difficulties and appreciate the work you're doing. They recognise your wins. But when you rarely communicate, they may only hear the bad news, and won't be as forgiving. Build investment through regular communications with all stakeholders.
Looking further afield
Can you communicate your findings or results with other industry groups - perhaps by speaking at an industry conference? That can help your organisation raise their profile, and it'll put you in touch with people who are tackling similar problems. You can also consider academic publications.