Automated machine learning

Short description: Process of automating the application of machine learning

Machine learning and
data mining

Problems

* Classification
* Clustering
* Regression
* Anomaly detection
* AutoML
* Association rules
* Reinforcement learning
* Structured prediction
* Feature engineering
* Feature learning
* Online learning
* Semi-supervised learning
* Unsupervised learning
* Learning to rank
* Grammar induction

Supervised learning
(classification • regression)

* Decision trees
* Ensembles
* Bagging
* Boosting
* Random forest
* k-NN
* Linear regression
* Naive Bayes
* Artificial neural networks
* Logistic regression
* Perceptron
* Relevance vector machine (RVM)
* Support vector machine (SVM)

Clustering

* BIRCH
* CURE
* Hierarchical
* k-means
* Expectation–maximization (EM)

DBSCAN
* OPTICS
* Mean-shift

Dimensionality reduction

* Factor analysis
* CCA
* ICA
* LDA
* NMF
* PCA
* t-SNE

Structured prediction

* Graphical models
* Bayes net
* Conditional random field
* Hidden Markov

Anomaly detection

* k-NN
* Local outlier factor

Artificial neural networks

* Autoencoder
* Deep learning
* DeepDream
* Multilayer perceptron
* RNN
* LSTM
* GRU
* Restricted Boltzmann machine
* GAN
* SOM
* Convolutional neural network
* U-Net

Reinforcement learning

* Q-learning
* SARSA
* Temporal difference (TD)

Theory

* Bias–variance dilemma
* Computational learning theory
* Empirical risk minimization
* Occam learning
* PAC learning
* Statistical learning
* VC theory

Machine-learning venues

* NeurIPS
* ICML
* ML
* JMLR
* ArXiv:cs.LG

Glossary of artificial intelligence

* Glossary of artificial intelligence

Related articles

* List of datasets for machine-learning research
* Outline of machine learning

* v
* t
* e

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. It is the combination of automation and ML.[1]

AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning.[2][3] The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models.[4]

When developing machine learning models, unless we have the required data engineering and machine learning development competencies, as well as the time to train and test multiple models and tune their hyperparameters, it is worth exploring the automatic machine learning (Auto ML) features offered by several cloud-based and cloud-agnostic platforms. These platforms enable the development of machine learning models through a low-code/no-code experience. Previous research developed criteria for comparing different machine learning platforms that generate automatic machine learning models and present their results. The results show that automatic machine learning platforms can provide a rapid pathway for organizations seeking to digitize their businesses. These platforms help deliver results, particularly for time-constrained, resource-constrained projects.[1]

Common techniques used in AutoML include hyperparameter optimization, meta-learning and neural architecture search.

## Contents

* 1 Comparison to the standard approach
* 2 Targets of automation
* 3 Challenges and Limitations
* 4 See also
* 5 References
* 6 Further reading

## Comparison to the standard approach

In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen manually by the machine learning expert.

Each of these steps may be challenging, resulting in significant hurdles to using machine learning. AutoML aims to simplify these steps for non-experts, and to make it easier for them to use machine learning techniques correctly and effectively.

AutoML plays an important role within the broader approach of automating data science, which also includes challenging tasks such as data engineering, data exploration and model interpretation and prediction.[5]

## Targets of automation

Automated machine learning can target various stages of the machine learning process.[3] Steps to automate are:

* Data preparation and ingestion (from raw data and miscellaneous formats)
* Column type detection; e.g., Boolean, discrete numerical, continuous numerical, or text
* Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature
* Task detection; e.g., binary classification, regression, clustering, or ranking
* Feature engineering
* Feature selection
* Feature extraction
* Meta-learning and transfer learning
* Detection and handling of skewed data and/or missing values
* Model selection \- choosing which machine learning algorithm to use, often including multiple competing software implementations
* Ensembling - a form of consensus where using multiple models often gives better results than any single model[6]
* Hyperparameter optimization of the learning algorithm and featurization
* Neural architecture search
* Pipeline selection under time, memory, and complexity constraints
* Selection of evaluation metrics and validation procedures
* Problem checking
* Leakage detection
* Misconfiguration detection
* Analysis of obtained results
* Creating user interfaces and visualizations

## Challenges and Limitations

There are a number of key challenges being tackled around automated machine learning. A big issue surrounding the field is referred to as "development as a cottage industry".[7] This phrase refers to the issue in machine learning where development relies on manual decisions and biases of experts. This is contrasted to the goal of machine learning which is to create systems that can learn and improve from their own usage and analysis of the data. Basically, it's the struggle between how much experts should get involved in the learning of the systems versus how much freedom they should be giving the machines. However, experts and developers must help create and guide these machines to prepare them for their own learning. To create this system, it requires labor intensive work with knowledge of machine learning algorithms and system design.[8]

Additionally, other challenges include meta-learning[9] and computational resource allocation.

## See also

* Artificial intelligence
* Artificial intelligence and elections
* Neural architecture search
* Neuroevolution
* Self-tuning
* Neural Network Intelligence
* ModelOps
* Hyperparameter optimization

## References

1. ↑ Spears, Taylor; Bondo Hansen, Kristian (2023-12-18), "The Use and Promises of Machine Learning in Financial Markets", The Oxford Handbook of the Sociology of Machine Learning (Oxford University Press), doi:10.1093/oxfordhb/9780197653609.013.6, ISBN 978-0-19-765360-9, http://dx.doi.org/10.1093/oxfordhb/9780197653609.013.6, retrieved 2024-06-10
2. ↑ "Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms". KDD '13 Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 2013. pp. 847–855. https://dl.acm.org/citation.cfm?id=2487629.
3. ↑ 3.0 3.1 "AutoML 2014 @ ICML". AutoML 2014 Workshop @ ICML. http://icml2014.automl.org.
4. ↑ Olson, R.S., Urbanowicz, R.J., Andrews, P.C., Lavender, N.A., Kidd, L.C., Moore, J.H. (2016). Automating Biomedical Data Science Through Tree-Based Pipeline Optimization. In: Squillero, G., Burelli, P. (eds) Applications of Evolutionary Computation. EvoApplications 2016. Lecture Notes in Computer Science(), vol 9597. Springer, Cham. doi:10.1007/978-3-319-31204-0_9
5. ↑ De Bie, Tijl; De Raedt, Luc; Hernández-Orallo, José; Hoos, Holger H.; Smyth, Padhraic; Williams, Christopher K. I. (March 2022). "Automating Data Science". Communications of the ACM 65 (3): 76–87. doi:10.1145/3495256.
6. ↑ Erickson, Nick; Mueller, Jonas; Shirkov, Alexander; Zhang, Hang; Larroy, Pedro; Li, Mu; Smola, Alexander (2020-03-13). "AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data". arXiv:2003.06505 [stat.ML].
7. ↑ Hutter, Frank, ed (2019) (in English). Automated Machine Learning: Methods, Systems, Challenges. The Springer Series on Challenges in Machine Learning. Springer Nature. doi:10.1007/978-3-030-05318-5. ISBN 978-3-030-05317-8. https://library.oapen.org/handle/20.500.12657/23012.
8. ↑ Glover, Ellen (2018). "Machine Learning with Python: Clustering". Built in. doi:10.4135/9781526466426. http://dx.doi.org/10.4135/9781526466426.
9. ↑ "Meta Learning Challenges" (in en-US). https://metalearning.chalearn.org/.

## Further reading

* "Open Source AutoML Tools: AutoGluon, TransmogrifAI, Auto-sklearn, and NNI". 2020-06-16. http://www.bizety.com/2020/06/16/open-source-automl-tools-autogluon-transmogrifai-auto-sklearn-and-nni/.
* Ferreira, Luís, et al. "A comparison of AutoML tools for machine learning, deep learning and XGBoost." 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 2021. https://repositorium.sdum.uminho.pt/bitstream/1822/74125/1/automl_ijcnn.pdf
* Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., & Hutter, F. (2015). Efficient and robust automated machine learning. Advances in neural information processing systems, 28. https://proceedings.neurips.cc/paper_files/paper/2015/file/11d0e6287202fced83f79975ec59a3a6-Paper.pdf

* v
* t
* e

Differentiable computing

General|

* Differentiable programming
* Neural Turing machine
* Differentiable neural computer
* Automatic differentiation
* Neuromorphic engineering

Concepts|

* Gradient descent
* Cable theory
* Cluster analysis
* Regression analysis
* Pattern recognition
* Adversarial machine learning
* Computational learning theory

Programming languages|

* Python
* Julia

Application|

* Machine learning
* Artificial neural network
* Scientific computing
* Artificial Intelligence

Hardware|

* TPU
* VPU
* Memristor
* SpiNNaker

Software library|

* TensorFlow
* PyTorch

Implementation|

| Audio-visual|

* AlexNet
* WaveNet
* Human image synthesis
* HWR
* OCR
* Speech synthesis
* Speech recognition
* Facial recognition system

|

Verbal|

* Word2vec
* Transformer
* BERT
* NMT
* Project Debater
* Watson
* GPT-3

Decisional|

* AlphaGo
* Q-learning
* SARSA
* OpenAI Five

People|

* Alex Graves
* Ian Goodfellow
* Yoshua Bengio
* Geoffrey Hinton
* Yann LeCun
* Andrew Ng
* Demis Hassabis

* Portals
* Computer programming
* Technology
* Category
* Artificial neural networks
* Machine learning

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Automated machine learning. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Automated_machine_learning&oldid=4226962"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template