Data augmentation

Short description: Data analysis technique

Machine learning and
data mining

Problems

* Classification
* Clustering
* Regression
* Anomaly detection
* AutoML
* Association rules
* Reinforcement learning
* Structured prediction
* Feature engineering
* Feature learning
* Online learning
* Semi-supervised learning
* Unsupervised learning
* Learning to rank
* Grammar induction

Supervised learning
(classification • regression)

* Decision trees
* Ensembles
* Bagging
* Boosting
* Random forest
* k-NN
* Linear regression
* Naive Bayes
* Artificial neural networks
* Logistic regression
* Perceptron
* Relevance vector machine (RVM)
* Support vector machine (SVM)

Clustering

* BIRCH
* CURE
* Hierarchical
* k-means
* Expectation–maximization (EM)

DBSCAN
* OPTICS
* Mean-shift

Dimensionality reduction

* Factor analysis
* CCA
* ICA
* LDA
* NMF
* PCA
* t-SNE

Structured prediction

* Graphical models
* Bayes net
* Conditional random field
* Hidden Markov

Anomaly detection

* k-NN
* Local outlier factor

Artificial neural networks

* Autoencoder
* Deep learning
* DeepDream
* Multilayer perceptron
* RNN
* LSTM
* GRU
* Restricted Boltzmann machine
* GAN
* SOM
* Convolutional neural network
* U-Net

Reinforcement learning

* Q-learning
* SARSA
* Temporal difference (TD)

Theory

* Bias–variance dilemma
* Computational learning theory
* Empirical risk minimization
* Occam learning
* PAC learning
* Statistical learning
* VC theory

Machine-learning venues

* NeurIPS
* ICML
* ML
* JMLR
* ArXiv:cs.LG

Glossary of artificial intelligence

* Glossary of artificial intelligence

Related articles

* List of datasets for machine-learning research
* Outline of machine learning

* v
* t
* e

Data augmentation is a technique in machine learning used to reduce overfitting when training a machine learning model,[1] achieved by training models on several slightly-modified copies of existing data.

## Contents

* 1 Synthetic oversampling techniques for traditional machine learning
* 2 Data augmentation for image classification
* 3 Data augmentation for signal processing
* 3.1 Biological signals
* 3.2 Mechanical signals
* 4 See also
* 5 References

## Synthetic oversampling techniques for traditional machine learning

## Data augmentation for image classification

When convolutional neural networks grew larger in mid-1990s, there often was not enough available data to train them, especially considering that some part of the overall dataset should be spared for later testing. It was proposed to perturb existing data with affine transformations to create new examples with the same labels,[2] which were complemented by so-called elastic distortions in 2003,[3] and the technique was widely used as of 2010s.[4] Data augmentation can enhance CNN performance and acts as a countermeasure against CNN profiling attacks.[5]

## Data augmentation for signal processing

Residual or block bootstrap can be used for time series augmentation.

### Biological signals

Synthetic data augmentation is of paramount importance for machine learning classification, particularly for biological data, which tend to be high dimensional and scarce. The applications of robotic control and augmentation in disabled and able-bodied subjects still rely mainly on subject-specific analyses. Data scarcity is notable in signal processing problems such as for Parkinson's Disease Electromyography signals, which are difficult to source - Zanini, et al. noted that it is possible to use a generative adversarial network (in particular, a DCGAN) to perform style transfer in order to generate synthetic electromyographic signals that corresponded to those exhibited by sufferers of Parkinson's Disease.[6]

The approaches are also important in electroencephalography (brainwaves). Wang, et al. explored the idea of using deep convolutional neural networks for EEG-Based Emotion Recognition, results show that emotion recognition was improved when data augmentation was used.[7]

A common approach is to generate synthetic signals by re-arranging components of real data. Lotte[8] proposed a method of "Artificial Trial Generation Based on Analogy" where three data examples [math]\displaystyle{ x_{1}, x_{2}, x_{3} }[/math] provide examples and an artificial [math]\displaystyle{ x_{synthetic} }[/math] is formed which is to [math]\displaystyle{ x_{3} }[/math] what [math]\displaystyle{ x_{2} }[/math] is to [math]\displaystyle{ x_{1} }[/math]. A transformation is applied to [math]\displaystyle{ x_{1} }[/math] to make it more similar to [math]\displaystyle{ x_{2} }[/math], the same transformation is then applied to [math]\displaystyle{ x_{3} }[/math] which generates [math]\displaystyle{ x_{synthetic} }[/math]. This approach was shown to improve performance of a Linear Discriminant Analysis classifier on three different datasets.

Current research shows great impact can be derived from relatively simple techniques. For example, Freer[9] observed that introducing noise into gathered data to form additional data points improved the learning ability of several models which otherwise performed relatively poorly. Tsinganos et al.[10] studied the approaches of magnitude warping, wavelet decomposition, and synthetic surface EMG models (generative approaches) for hand gesture recognition, finding classification performance increases of up to +16% when augmented data was introduced during training. More recently, data augmentation studies have begun to focus on the field of deep learning, more specifically on the ability of generative models to create artificial data which is then introduced during the classification model training process. In 2018, Luo et al.[11] observed that useful EEG signal data could be generated by Conditional Wasserstein Generative Adversarial Networks (GANs) which was then introduced to the training set in a classical train-test learning framework. The authors found classification performance was improved when such techniques were introduced.

### Mechanical signals

The prediction of mechanical signals based on data augmentation brings a new generation of technological innovations, such as new energy dispatch, 5G communication field, and robotics control engineering.[12] In 2022, Yang et al.[12] integrate constraints, optimization and control into a deep network framework based on data augmentation and data pruning with spatio-temporal data correlation, and improve the interpretability, safety and controllability of deep learning in real industrial projects through explicit mathematical programming equations and analytical solutions.

## See also

* Oversampling and undersampling in data analysis
* Generative adversarial network
* Variational autoencoder
* Data pre-processing
* Convolutional neural network
* Regularization (mathematics)
* Data preparation
* Data fusion

## References

1. ↑ Shorten, Connor; Khoshgoftaar, Taghi M. (2019). "A survey on Image Data Augmentation for Deep Learning". Mathematics and Computers in Simulation (springer) 6: 60. doi:10.1186/s40537-019-0197-0.
2. ↑ Yann Lecun (1995) (Conference paper). Learning algorithms for classification: A comparison on handwritten digit recognition. World Scientific. pp. 261–276. https://nyuscholars.nyu.edu/en/publications/learning-algorithms-for-classification-a-comparison-on-handwritte. Retrieved 14 May 2023.
3. ↑ Simard, P.Y.; Steinkraus, D.; Platt, J.C. (2003). "Best practices for convolutional neural networks applied to visual document analysis". Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings.. 1. pp. 958–963. doi:10.1109/ICDAR.2003.1227801. ISBN 0-7695-1960-1.
4. ↑ Hinton, Geoffrey E.; Srivastava, Nitish; Krizhevsky, Alex; Sutskever, Ilya; Salakhutdinov, Ruslan R. (2012). "Improving neural networks by preventing co-adaptation of feature detectors". arXiv:1207.0580 [cs.NE].
5. ↑ Cagli, Eleonora; Dumas, Cécile; Prouff, Emmanuel (2017). Fischer, Wieland; Homma, Naofumi. eds. "Convolutional Neural Networks with Data Augmentation Against Jitter-Based Countermeasures" (in en). Cryptographic Hardware and Embedded Systems – CHES 2017. Lecture Notes in Computer Science (Cham: Springer International Publishing): 45–68. doi:10.1007/978-3-319-66787-4_3. ISBN 978-3-319-66787-4. https://link.springer.com/chapter/10.1007/978-3-319-66787-4_3.
6. ↑ Anicet Zanini, Rafael; Luna Colombini, Esther (2020). "Parkinson's Disease EMG Data Augmentation and Simulation with DCGANs and Style Transfer". Sensors 20 (9): 2605. doi:10.3390/s20092605. ISSN 1424-8220. PMID 32375217. Bibcode: 2020Senso..20.2605A.
7. ↑ Wang, Fang; Zhong, Sheng-hua; Peng, Jianfeng; Jiang, Jianmin; Liu, Yan (2018). "Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks". MultiMedia Modeling. Lecture Notes in Computer Science. 10705. pp. 82–93. doi:10.1007/978-3-319-73600-6_8. ISBN 978-3-319-73599-3.
8. ↑ Lotte, Fabien (2015). "Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer Interfaces". Proceedings of the IEEE 103 (6): 871–890. doi:10.1109/JPROC.2015.2404941. ISSN 0018-9219. https://hal.inria.fr/hal-01159171/file/lotte_sigProcCalibReduction-final.pdf.
9. ↑ Freer, Daniel; Yang, Guang-Zhong (2020). "Data augmentation for self-paced motor imagery classification with C-LSTM". Journal of Neural Engineering 17 (1): 016041. doi:10.1088/1741-2552/ab57c0. ISSN 1741-2552. PMID 31726440. Bibcode: 2020JNEng..17a6041F.
10. ↑ Tsinganos, Panagiotis; Cornelis, Bruno; Cornelis, Jan; Jansen, Bart; Skodras, Athanassios (2020). "Data Augmentation of Surface Electromyography for Hand Gesture Recognition". Sensors 20 (17): 4892. doi:10.3390/s20174892. ISSN 1424-8220. PMID 32872508. Bibcode: 2020Senso..20.4892T.
11. ↑ Luo, Yun; Lu, Bao-Liang (2018). "EEG Data Augmentation for Emotion Recognition Using a Conditional Wasserstein GAN". 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2018. pp. 2535–2538. doi:10.1109/EMBC.2018.8512865. ISBN 978-1-5386-3646-6.
12. ↑ 12.0 12.1 Yang, Yang (2022). "Wind speed forecasting with correlation network pruning and augmentation: A two-phase deep learning method". Renewable Energy 198 (1): 267–282. doi:10.1016/j.renene.2022.07.125. ISSN 0960-1481.

* v
* t
* e

Differentiable computing

General|

* Differentiable programming
* Neural Turing machine
* Differentiable neural computer
* Automatic differentiation
* Neuromorphic engineering

Concepts|

* Gradient descent
* Cable theory
* Cluster analysis
* Regression analysis
* Pattern recognition
* Adversarial machine learning
* Computational learning theory

Programming languages|

* Python
* Julia

Application|

* Machine learning
* Artificial neural network
* Scientific computing
* Artificial Intelligence

Hardware|

* TPU
* VPU
* Memristor
* SpiNNaker

Software library|

* TensorFlow
* PyTorch

Implementation|

| Audio-visual|

* AlexNet
* WaveNet
* Human image synthesis
* HWR
* OCR
* Speech synthesis
* Speech recognition
* Facial recognition system

|

Verbal|

* Word2vec
* Transformer
* BERT
* NMT
* Project Debater
* Watson
* GPT-3

Decisional|

* AlphaGo
* Q-learning
* SARSA
* OpenAI Five

People|

* Alex Graves
* Ian Goodfellow
* Yoshua Bengio
* Geoffrey Hinton
* Yann LeCun
* Andrew Ng
* Demis Hassabis

* Portals
* Computer programming
* Technology
* Category
* Artificial neural networks
* Machine learning

* v
* t
* e

Data

* Analysis
* Archaeology
* Cleansing
* Collection
* Compression
* Corruption
* Curation
* Degradation
* Editing
* ETL
* Farming
* Format management
* Fusion
* Integration
* Integrity
* Library
* Loss
* Management
* Migration
* Mining
* Pre-processing
* Preservation
* Protection (privacy)
* Recovery
* Reduction
* Retention
* Quality
* Science
* Scraping
* Scrubbing
* Security
* Stewardship
* Storage
* Validation
* Warehouse
* Wrangling/munging

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Data augmentation. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Data_augmentation&oldid=3372029"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template