Natural language processing or NLP is a branch of artificial intelligence that has many important implications on the ways that computers and humans interact. Human language, developed over thousands and thousands of years, has become a nuanced form of communication that carries a wealth of information that often transcends the words alone. NLP will become an important technology in bridging the gap between human communication and digital data. Here are 5 ways that natural language processing will be used in the years to come.
* Machine Translation
As the world’s information is online, the task of making that data accessible becomes increasingly important. The challenge of making the world’s information accessible to everyone, across language barriers, has simply outgrown the capacity for human translation. Innovative companies like Duolingo are looking to recruit large amounts of people to contribute, by coinciding translation efforts with learning a new language. But machine translation offers an even more scalable alternative to harmonizing the world’s information. Google is a company at the forefront of machine translation, using a proprietary statistical engine for its Google translate service. The challenge with machine translation technologies is not in translating words, but in preserving the meaning of sentences, a complex technological issue that is at the heart of NLP.
- Fighting Spam
Spam filters have become important as the first line of defense against the ever-increasing problem of unwanted email. But almost everyone that uses email extensively has experienced agony over unwanted emails that are still received, or important emails that have been accidentally caught in the filter. The false-positive and false-negative issues of spam filters are at the heart of NLP technology, again boiling down to the challenge of extracting meaning from strings of text. A technology that has received much attention is Bayesian spam filtering, a statistical technique in which the incidence of words in an email is measured against its typical occurrence in a corpus of spam and non-spam emails.
* Information Extraction
Many important decisions in financial markets are increasingly moving away from human oversight and control. Algorithmic trading is becoming more popular, a form of financial investing that is entirely controlled by technology. But many of these financial decisions are impacted by news, by journalism which is still presented predominantly in English. A major task, then, of NLP has become taking these plaintext announcements, and extracting the pertinent info in a format that can be factored into algorithmic trading decisions. For example, news of a merger between companies can have a big impact on trading decisions, and the speed at which the particulars of the merger, players, prices, who acquires who can be incorporated into a trading algorithm can have profit implications in the millions of dollars.
* Summarization
Information overload is a real phenomenon in our digital age, and already our access to knowledge and information far exceeds our capacity to understand it. This is a trend that shows no sign of slowing down, and so an ability to summarize the meaning of documents and information is becoming increasingly important. This is important not just in allowing us the ability to recognize and absorb the pertinent information from vast amounts of data. Another desired outcome is to understand deeper emotional meanings, for example, based on aggregated data from social media, can a company determine the general sentiment for its latest product offering? This branch of NLP will become increasingly useful as a valuable marketing asset.
- Question Answering
Search engines put the world’s wealth of information at our fingertips, but are still generally quite primitive when it comes to actually answering specific questions posed by humans. Google has seen the frustration this has caused in users, who often need to try a number of different search results to find the answer they are looking for. A big focus of Google’s efforts in NLP has been to recognize natural language questions, extract the meaning, and provide the answer, and the evolution of Google’s results page has shown this focus. Though certainly improving, this remains a major challenge for search engines, and one of the main applications of natural language processing research.