AutoML is currently one of the hot topics of AI-based solutions. Automated machine learning is a proposed AI-based solution to the growing challenges of using machine learning that automates real-world time-consuming, iterative tasks. The process potentially includes every stage, from beginning with a raw dataset to building a machine learning model ready for deployment.
It's like developing a Machine learning application without disturbing most of the neurons to work.
Just like canned food packages say: open, fry and eat.
How does AutoML work?
This question keeps popping into everyone's mind as this looks like a shortcut for machine learning applications.
In the past few years, the machine learning field has begun to diverge outside of complex models and instead use simpler models that are easier to interpret. Why? As it is hard to know when a model is introducing bias because complex models can be hard to decipher. AutoML came as a rescue to this problem of a complex model by hiding not only the mathematics of the model but also performing the following in the background:
Automated machine learning creates several parallel pipelines to try different algorithm parameters, combined with feature selections, where each iteration produces a model with a training score. The better the score for the metric you want, the better the model considers to "fit" your data. This process will stop once it produces a model that will hit the exit criteria defined in the experiment.
The following diagram illustrates the above process:
What’s new this Automated approach has?
A typical machine learning application uses a set of input data points for training. But an expert should make data amenable for machine learning by applying data pre-processing, feature engineering, feature extraction, and feature selection methods. After this, we have to perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model.
All these challenging steps create hurdles in using machine learning even for known persons, which are simplified and bypassed by AutoML.
This high degree of automation in AutoML even allows non-experts to use machine-learning models and techniques. Automating the process of applying machine learning end-to-end offers the advantages of producing straightforward solutions and models that often outperform hand-designed models while sustaining model quality with large scale, efficiency, and productivity entirely.
Automated machine learning tools Scope: covers every aspect of the machine learning workflow that can be potentially automated.
A few examples of Machine learning workflow that can be automated
Data preparation and ingestion
Data types: Boolean, discrete/continuous numbers, text
Task detection: Binary classification, regression, clustering, ranking, and others
Feature Engineering- this can be time-consuming, but automating some of this can save time.
So Tasks in this step are Feature selection, Preprocessing, Extraction, Skewed data detection, and Missing values detection.
Model selection-Includes were finding the general type of model to use and doing your architecture search to find the specific structure most suitable for a given data task.
Finally, there is the automation of the evaluation step, everything from validation procedures to checking for mistakes, as well as analysis and visualization of the results. And by bypassing all of these tedious and challenging steps, AutoML proves itself to come as a boon for future scientists and developers.
Will AutoML be a Bane for data scientists?
No, It can’t but it proved as a good assistant to them. A data scientist understands and inspects data rather than more on the model selection and hyperparameter tuning than AutoML does.
When You watched Randall Olsen’s (Data scientist and AI researcher) talk at Scipy 2018, The Past, Present, and Future of Automated Machine Learning. He points out that AutoML is not there to replace the Data Scientist, but it is here to improve the productivity of current data scientists and make the field more accessible to those who have not been studying these algorithms for years but still want to pursue and want to explore this field.
Thinking How It's gonna help?
Machine learning professionals, developers and data scientists across industries can use automated ML to:
Implement ML solutions without extensive programming knowledge
Save time and resources
Leverage data science best practices
Provide agile problem-solving
The following diagram will help you to understand better.
Nonetheless, AutoML can still serve as a buddy that can double-check your work and provide lots of heavy lifting for those trying to build an AI application.
Do you still think any Disadvantages exist?
Nowadays, just anyone can claim to be a Machine Learning professional. With larger and larger bunches of people opting to play with Machine Learning, the dangerous aspect is that they are using a drilling machine but do not know the proper ways to do so.
We have seen multiple reports of incorrect/biased predicting models that are continuously circulating in the past few years. If there is bias in the real world, it can easily be increased by not accounting for it properly in the model. That's why If we use AutoML to automate the process fully, then it will cause more harm than good. However, full automation is only the goal of a handful of research companies which can be disastrous too. This bias might be deeply hidden within the guts of the ML portion that will keep creating problems in the future.
Just sit yourself down in front of a computer, make some selections on a few screens, and Avada kedavra, you kill all the challenging hurdles and make a Machine Learning app that does whatever it is you dreamed up. Is life this much easy? I don’t think so. Let’s not set our expectations quite this high that embarking upon using the latest and greatest in Automated Machine Learning makes you an expert in machine learning.
“For men may come and men may go, But I go on forever.”
This means if AutoML is “shallow” and just provides the surface-level accoutrements to make Machine Learning applications, it is dangerous, while if AutoML embraces fully and implements added capabilities to provide insight for the AI Ethics precepts, it is hopefully going to do more good than harm.
At this point, AutoML is still in its infant phase, and many would say that the Machine Learning apps being crafted via AutoML are more prototypes and pilot efforts, not full-fledged ones (this is debatable, of course).
Those that are versed in Machine Learning are already eyeing AutoML with a feeling of uneasiness that the potential dumbing down of Machine Learning is going to be an adverse slippery slope; meanwhile, they welcome well-crafted AutoML that can support professional work on Machine Learning.
Another side that might benefit the community greatly is AutoML, which provides excellent assistance to data scientists.
That’s up to us humans what we're gonna choose to do.
Thanks for reading my Newsletter! Subscribe for free to receive new posts and support my work.
This article was originally published on the company blog.
Intellicy is a consultancy firm specialising in artificial intelligence solutions for organisations seeking to unlock the full potential of their data. They provide a full suite of services, from data engineering and AI consulting to comment moderation and sentiment analysis. Intellicy's team of experts work closely with clients to identify and measure key performance indicators (KPIs) that matter most to their business, ensuring that their solutions generate tangible results. They offer cross-industry expertise and an agile delivery framework that enables them to deliver results quickly and efficiently, often in weeks rather than months. Ultimately, Intellicy helps large enterprises transform their data operations and drive business growth through artificial intelligence and machine learning.
