Social Media Text Normalization Script

This is a simple normalizer that performs (i) preprocessing of text (ii) lexicon-based normalization

The script implements the techniques presented in the first 2 steps of the pipeline described in the following paper:

Sarker A and Gonzalez G. A Customizable Pipeline for Social Media Text Normalization. [to appear].

Please cite the above paper when using the system.

The resources used by the system can be found in: