Why patience is the first thing organizations need for adoption of AI?
The world of business is rightfully excited about the adoption of AI-based systems that promise to change the way businesses are run by empowering managers to make better and faster decisions, by cutting costs and improving efficiency. AI has numerous applications in a variety of organizations, which can vary from simple rule-based expert systems to much more complex machine learning models that understand various forms of information and make sense of huge sources of data that are impossible or very costly for humans to process. For example, natural language processing models can be trained to read thousands of documents, extract the desired information, and summarize the text. Audio processing models can transcribe recorded meetings, analyze them to identify action items, and improve information management in an organization. Computer vision models can be trained and used to make sense of camera footage and extract useful metadata from them.
Machine Learning 101
When we talk about machine learning, what we really mean is that our most complex algorithms can learn to mimic some of human’s simplest cognitive capabilities. Machines can learn in different ways such as supervised, unsupervised, and reinforcement learning, depending on how the algorithm interacts with the training environment. However, the most disruptive leaps in machine learning have happened in the area of supervised learning where algorithms learn from a huge number of patterns along with what humans understand from those patterns.
The algorithm forms a mathematical mapping from input patterns to human-generated labels. What has been revolutionized in recent years is our capability to feed large numbers of input-output pairs to algorithms. Thanks to hardware, computational advancements and also mathematical novelties that let us represent patterns better for computers to understand we are able to ‘teach’ machines ways to mimic simple human-like cognitive tasks.
One example of supervised learning is object recognition in computer vision, where models are trained to accurately recognize objects in images. The algorithm has to see and learn from millions of images along with labels that describe what humans see. Although supervised learning produced extremely impressive results, supervised models are only as good as the training dataset provided to train them. As an example, the computer vision community owes the huge progress made in object recognition to the Imagenet Dataset. It can take an army of people to collect such a high volume and variety of images and annotate every single image. This is what makes supervised training very costly and hard to customize for certain applications. Without annotated datasets the best algorithms will not know anything. Importantly the quality of the training dataset is what dictates how accurate the trained model is. For example, in the case of object recognition, if the training dataset only includes pictures of animals, the trained model will not be able to recognize plants.
In recent years open source research has produced many pre-trained models and annotated datasets, many of which are publicly available. However; it is very rare for these datasets and trained models to match perfectly with business use cases. As a result some annotated data is always needed to retrain and tweak the already developed models.
For example, analyzing camera footage in retail stores can improve store management. To do so algorithms need to be able to distinguish between customers and staff. Although there are pre-trained models that can identify people in camera feeds, they need to be further trained to distinguish between customers and staff of a specific store. Collecting and labelling a dataset for such tuning can be very costly. Another example is the pre-trained language models that understand general language extremely well, but always need to be fine-tuned with labelled text to perform well in a specific context. An example of such a language model is BERT, trained and released by google.
The Process of Adopting AI Takes Time
The Process of Adopting AI
Although AI has the potential to free up people's time from boring work and make processes much faster the availability of data with useful annotations is what makes the first stages of adoption of AI very slow (frequently even slower than a humans’ own workflow). Only businesses that have collected the right set of annotated data over years of operation are in the position to quickly adopt AI. Where this type of data is not available, organizations should be prepared for the time and cost of building their own data set to achieve their specific aim.
As machine learning practitioners, we always experience a very important challenge to adoption - change management, understanding and utilization. To accomplish these, organizations need to work with machine learning experts to identify the plausible opportunities for adoption of AI in their existing workflows. It is challenging and important to identify the volume of data and the right kind of annotation that is needed to address particular use cases. Once the business problem is translated into a well-defined machine learning problem and the characteristics of the data are figured out, 90% of the remaining work is data preparation, annotation and feature engineering while 10% of the work is algorithm design, training and tuning.
Where to Start
The best starting point for the adoption of AI for every organization is where the task can be achieved with the least amount of annotation, or where programmatic annotation is an option. When large amounts of data and hand-labelling are required, it is important for AI practitioners to educate businesses about the obstacles and costs of data availability and manage their expectations in the early stages of adoption.
The first and most effective step every business can take for systematic adoption of AI is revising their workflow for storing, labelling, and maintaining data. Most companies do not have access to the right tools to annotate data and store it in a way that is easily fed to an algorithm. Even when there is historical data accompanied by human-made decisions there is no guarantee that this annotated dataset can be used to train a machine.
What companies collect and store is usually only the final decision made based on an unknown collection of pieces of information, but what is needed to train a model is the intermediate cognitive tasks that human subjects accomplished to come to that decision. In the example of the recruitment industry, a company might have stored the final decision that was made for each resume, but these labels are not enough to train a machine to read resumes. The data that is needed to train a resume reader should include numerous resumes with important information such as qualifications, degrees, etc, marked on them. The existing workflows of organizations usually do not generate this type of annotated data.
human and machine working together
Another reason that makes adoption of AI slow is the risk of relying on an algorithm on its own. There are both technical and ethical reasons for why businesses should consider AI-based systems as an ensemble of humans and machines for now. From a technical point of view the full replacement of a task in an existing workflow is much more costly and time-consuming than it seems to be. It is always a good idea to start small and achieve full integration of AI through an iterative and incremental process. Removing humans from the workflow makes the risk of using AI high since it is difficult to anticipate all the edge cases in the first rounds of training models.
An iterative process with humans in the loop allows the technical team to understand the details of domain knowledge and improve the models for high accuracy and desired generalization. Also from an ethical point of view, where AI is expected to make critical decisions, in the early stages of adoption an expert should review every decision since the risk of bias and lack of transparency is extremely high. Identifying risks and practical limitations of the trained models is another obstacle for fast adoption of AI that is frequently overlooked.
A final piece of advice
For organizations to be well-prepared for the adoption of AI, business processes should be designed around effective data capture, and workflows should be built to make annotation a part of everyone's work. Most importantly companies should invest in AI literacy so that both decision makers and users understand the potential and limitations of complex AI and are able to use it accurately and in a way that is safe and fair to society.