NLP for Dummies

by Dylan Sam, '21

Have you ever wondered how computers seem to understand tweets, books, or song lyrics? A small subset of artificial intelligence known as Natural Language Processing (NLP) makes many of these tasks possible. Computers can develop probabilistic models to understand the structure of language and how it is used. This process allows computers to understand natural language, which has many different, powerful applications.
NLP is the process of creating machine learning models to understand or operate on text data. A computer receives large bodies of text, which are also known as corpuses, and performs many interesting tasks on them. 

For example, programs can generate sentiment values, which are scores ranging from -1 to 1, to denote the overall negativity or positivity of the text. Through a process known as topic modelling, computers can extract the main topics and keywords from a body of text. Programs can also generate new articles from past knowledge and training data. Because NLP allows computers to better understand text, it has almost universal application to anything that relates to words. Thus, NLP is commonly used in research in many different fields.

Large amounts of useful text data are stored at clinics, where clinician notes contain information about patients, health, and treatments. In Measuring Processes of Care in Palliative Surgery: A Novel Approach Using Natural Language Processing, E. Lilley  mentions possible applications of NLP in palliative surgery and evaluating the care of patients. The researchers created a NLP algorithm that extracted data from the Research Patient Data Registry, which is a collection of data from hospital systems created by the RISC. Utilizing this data, the algorithm compared the model’s evaluation with manual assessment for 20 randomly selected patients. The model developed “high sensitivities and specificities that ranged from 93.8% to 100%”[1], which is the ability to correctly identify positive results and negative results. This high percentage shows the ability of the NLP model to correctly identify patient results and statuses. Furthermore, the model runs at a much higher speed efficiency; manual assessment for a single patient took 20 hours while the model completed the same task in 26 seconds. NLP serves as an accurate and fast method to evaluate and create benchmarks for patient care and palliative surgery.

​Another interdisciplinary usage of NLP was in a research project regarding analysis of disaster recovery trends. Researchers at the University of Washington developed an NLP pipeline to evaluate how different recovery propositions are expressed within news articles and other texts that describe disaster recovery. The NLP program helps visualize the appearance of common issues during disaster situations over time, such as power systems, frustrated residents, mental health issues, etc. When the researchers asked the public if the development of NLP methods to analyze disaster recovery was useful, “85% responded positively” [2]. This overwhelmingly positive result shows the applications of NLP to the real world and to victims of disaster relief. Its ability to process and visualize large amounts of text data allows it to harness news articles and journals and present them concisely to people. NLP has the ability to help people visualize and understand the trends in disaster recovery and many other similar topics.

These two publications only scratch the surface of the possibilities for NLP. New NLP models and algorithms are released daily that positively affect medicine, linguistics, politics, and so much more. NLP is an incredibly powerful field of artificial intelligence that will begin and continue to transform how language is understood.

Citations:

1. Lilley E., Lindvall C., Lillemoe K. “Measuring Processes of Care in Palliative Surgery: A Novel Approach Using Natural Language Processing.” Annals of Surgery. 2018; 267(5), 823-835.
​
2. Lin L., Miles S., Smith N. “Natural Language Processing for Analyzing Disaster Recovery Trends Expressed in Large Text Corpora.” 2018; 1-8.