Member-only story
Improving NLP Models with Data Noising | Towards AI
How does Data Noising Help to Improve your NLP Model?
The objective of injecting data noising
Introducing data noising to neural network aims at improving model generalization and performance. Xie et al. proposes several ways to generate more training via unigram noising and blank noising for discrete sequence level settings such as language modeling. In other words, it is a way to perform data augmentation on NLP.
This story goes though Data Noising as Smoothing in Neural Network Language Models (Xie et al., 2017). It includes two parts which are smoothing in language models (LM) and method of data augmentation.
Data Noising Method
Unigram noising method is replacing target word by other word according to unigram frequency distribution. Similar to Unsupervised Data Augmentation (Xie et al., 2019), word count (as known as word frequency) is calculated before data augmentation while dictionary is built from training data. Here is the example of unigram noising.
# Original
text = 'The quick…
