| 2020-12-04 11:41 |
Introduction: Despite the downturn in IT spending this year, it is interesting that the NLP budget has increased across the board, and the reported NLP technology budget has increased by 10-30% compared to last year.
Translator: AI Research Institute ( Icarus, )
Bilingual original link: The State of Enterprise NLP in 2020
2020 will be a unique year for public health, professional life, economy and almost all other aspects of daily life. Although some opportunities are disappearing, others are transforming their business models, and companies that have not been affected are rare. Nevertheless, there are still some industries that are booming, not just virtual meetings or healthcare.
Natural language processing (NLP) is one of these areas. In fact, according to MarketsandMarkets™ research, the NLP market is expected to grow from 10.2 billion USD in 2019 to 26.4 billion USD in 2024. Using use cases to help patients and practitioners in a healthcare environment, simplify customer service queries, and even help shoppers virtually, there are several growth factors driving the growth of NLP technology. Whether you are a novice or an experienced data scientist, NLP can help users work faster, smarter, and more accurately.
To understand the development status of NLP in the next few years, we first need to understand the current status of NLP-from challenges, successes, and most common use cases. To this end, John Snow Labs has cooperated with Gradient Flow and recently released a new research report that discusses the use of NLP in different industries, different regions, and different application levels. Knowledge is power. The goal of this survey is to help IT leaders realize the full potential of NLP by understanding how organizations use NLP technology.
This global survey asked nearly 600 respondents from more than 50 countries to fully understand the adoption and implementation of NLP in 2020. The following key survey results will help establish a benchmark for the industry and predict the direction of our NLP development in the next year.
NLP spending is on the rise : Despite the downturn in IT spending this year, it is interesting that the NLP budget has increased across the board, and the reported NLP technology budget has increased by 10-30% compared to last year. This is especially important considering that the survey was conducted at the height of the global COVID-19 pandemic, when global IT spending was declining (Gartner). Fifty-three percent of respondents are technology leaders, and they stated that their NLP budget has increased by at least 10% compared to 2019, and 31% of them stated that their budget has increased by at least 30% compared to the previous year. The same trend applies to large companies (companies with more than 5,000 employees), where 61% of respondents indicated that their budgets will increase in 2020.
The use of cloud computing brings challenges. 77% of respondents said that they use at least one of the four listed NLP cloud services-Google, AWS, Azure or IBM. Although cloud-based services are popular, respondents believe that cost is the main challenge they face when using NLP cloud services. In addition, people are also worried about scalability, because many NLP applications rely on language usage in specific areas, and cloud providers are slow to serve these market needs. Nonetheless, 53% of respondents said they used at least one of the two NLP libraries, namely Spark NLP and spaCy. This is a more accurate and cost-effective choice. It is not surprising to make this choice.
Accuracy is very important and very challenging. More than 40% of respondents pointed out that accuracy is their most important criterion for evaluating NLP libraries. This is especially important considering the use of NLP in critical applications, such as electronic health records or the detection of adverse drug events in medical settings. On the other hand, accuracy is also the challenge most frequently mentioned by all interviewees. However, when looking at technology leaders, this situation changes slightly. Integration issues, language support, and scalability and accuracy are juxtaposed to become urgent challenges. Fortunately, areas such as language support are improving dramatically. Companies such as Google and Facebook are releasing pre-trained embeddings in more than 150 languages. The NLP library is also following up.
Classification and NER are the main use cases. The four most popular applications of NLP are document classification, named entity recognition (NER), sentiment analysis, and knowledge graphs. Respondents from the healthcare field believe that de-identification is another common NLP use case. Automated NLP was once an extremely manual and labor-intensive process, and now it has greatly reduced this burden. NER and classification are two other NLP use cases where medical institutions see great value. For example, these applications can help medical professionals quickly and accurately identify patients’ adverse drug events (ADE), improve medical services, and reduce the burden and cost of the medical system.
Data Sources. Data from files (such as pdf, txt, docx, etc.) and databases are among the top data sources used in NLP projects (61%). From legal contracts and news articles to medical records and SEC documents, these input files are usually stored in PDF format. Although deep learning models have improved in the past few years, there are many difficulties and data quality issues when extracting text from PDF. Interestingly, there are some differences in data sources between companies that are still exploring NLP and those that have gone further along the adoption curve. Respondents in the exploratory stage reported using audio data (29%) higher than those who were more advanced (22%).
According to the growth trajectory of NLP in the past year, it is clear that its momentum will continue until 2021. With time and upcoming technological enhancements, it will be interesting how adoption and use cases develop. NLP has the ability to change the way we work, giving and receiving medical care, shopping, and the interface with customer service. Although some of these cases may be more influential than others, they will all shape our better work and lifestyle.