About AutoML

Automated machine learning (AutoML) is the process of automating the end-to-end process of applying machine learning to real-world problem.
AutoML covers the complete pipeline from the raw dataset to the deployable machine learning model.
The goal of AutoML software is two-fold:
- To enable non-experts to train high quality machine learning models.
- To improve the efficiency of finding optimal solutions to machine learning problems.
Domain knowledge is where Data Scientists over take AutoML.

General Pipeline Involved

Get dataset from different sources.
Consolidate the raw dataset to analyze further.
Provide target variable to predict.
Integrating the data preprocessing steps into the AutoML
- Include the automatic detection of the column types such as continuous, categorical, free text, date, etc.
- Transformation of all the columns into numerical data depending on their type. For example: encoding for categorical columns, processing or deletion of free text columns, transformation of date columns.
- Handling of missing values with different strategies. Example: set missing data to a given value, add a category corresponding to a missing data, data filling using forward or backward values, using different ML algorithm to fill value, and so on.
- Detection and if configured handling anomaly or outliers in the data.
- Detection and handling of skewed data.
- Feature selection is sometimes needed to get rid of columns which do not bring valuable information to solve the problem.
- Redundant or similar feature removal, this will also eliminate bias.
- Data compression and dimensionality reduction methods such as PCA or the use of autoencoder.
- Data normalisation or scaling when there is lots of variation in the data, to make sure the model is not biased.
- Down sampling or up sampling if there is imbalance in the target variable.
- Generation of new features by processing of multiple variables.
- Automated new feature creation, this would be purely with help of domain knowledge.
- Computer vision models and natural language processing models perform additional preprocessing.
  - DNN: select the different topology of neural networks
  - CNN : augmentation, cropping, resize, colour change, noise cancelation, etc
  - NLP : lemmatizations, stemming, word split, remove punctuations, remove stopwords, etc.
List of the models among a predefined catalog to fit the data, and the choice of the train/validate scheme.
- Classification
- Regression
- Segmentation
- Clustering
Hyper parameters of the model are optimised
- Observe the performance of different models with different models with different hyper parameters choices tested by AutoML.
- For DNN based models, additional layers or defined algorithms can be tried as well to find the best model and with weights.
- Transfer learning, in which pretrained models apply already learned weights. Transfer learning helps AutoML apply existing architectures to new problems that require it.
Evaluation metrics
- Selection of evaluation metrics and validation procedures
- The existing metric provided would be used, but usually will have provision for additional metrics may used be compute too.
Domain Knowledge Injection
- Injecting a priori domain knowledge using a fixed data schema brings a lot of interesting capabilities.
- Increase the accuracy of the models by 20-30%.
- Enhances their explainability, as business understands the features used.
- Less computing power is required and training time can be accelerated by 5 to 10%.

Need of AutoML

Automated model selection will act as framework or pipeline
Increase productivity by automating repetitive task. This enables a data scientist to focus more on the problem rather than the models
Automating the ML pipeline also helps to avoid errors that might creep in manually
- Leakage detection
- Misconfiguration detection
AutoML is a step towards democratising machine learning by making the power of ML accessible to everybody.

AutoML Frameworks

MLBox is a powerful automated machine learning Python library.
Auto-Sklearn is an automated machine learning package built on top of Scikit-learn.
TPOT is a Python automated machine learning tool that optimises machine learning pipelines using genetic programming.
H2O is a fully open source, distributed in-memory machine learning platform from the company H2O.ai.
Auto-Keras is an open source software library built by DATA Lab for automated machine learning.
Cloud AutoML is a suite of machine learning products from Google that enables developers with limited machine learning expertise to train high-quality models specific to their business needs by leveraging Google's state-of-the-art transfer learning and Neural Architecture Search technology.
TransmogrifAI is an open source automated machine learning library written in Scala that runs on top of Apache Spark from Salesforce.
Auto-PyTorch is based on the deep learning framework PyTorch and jointly optimizes hyper parameters and the neural architecture.

AutoML Challenges

As we have seen exhaustive list of Advantages, we have series of challenges also. Also not all the frameworks are complete.
Heavy resource utilization: as it requires mode models to run under different parameters, would be very time and resource consuming.
Analysis report: as a ML engineer, they would have done lots of analysis of data during feature selection and feature engineering, these details may not be captured for the ML engineer.
Injection of knowledge to the models: not all the frameworks support this, lets say engineer would want to add some custom logic to the model.
Early stopping or overfitting: in the course of developing solution AutoML need to know when to stop and when it would be over fitted.
ML Pro Engineer is still needed: Auto learning, when to retrain, and many other such steps, is still under ML engineer. AutoML as of now will not replace Data Scientists.
OS support: not all the frameworks support all or different operating systems.
Visualization: good or sufficient visualization is generally not provided by these frameworks.

Conclusion

AutoML steps towards General-AI. It automates monotonous tasks like monitoring, first level analysis, problem detection, or any tasks which are repetitive in nature. An engineer would still be needed to supervise the tasks performed by AutoML.

AutoML should help improve data scientist and employee efficiency, but will not replace them.