作者：Valerio Arvizzigno , Dimitri Marx , Francesco Di Stefano
This is a comprehensive guide to supporting faster drug innovation and discovery in the pharmaceutical industry through generative AI/LLM, custom models, and the Elasticsearch Relevance Engine (ESRE). Faster drug discovery leading to promising drug candidates is a major goal of the pharmaceutical industry. To support this goal, the industry must find better ways to leverage public and proprietary data quickly and securely.
According to the WIPO , R&D patent analysis can help researchers and innovators:
- Avoid duplication of research and development work
- Determine the patentability of their inventions
- Avoid infringing the patents of other inventors
- Estimate the value of their or other inventors' patents
- Utilize technology from patent applications that were never granted, invalid in some countries, or patents that are no longer valid
- Gain intelligence on innovation activities and future directions of business competitors
- Improve business decision planning, such as licensing, technology partnerships, and mergers and acquisitions
- Identify key trends in specific technology areas of public interest, such as those related to health or the environment, and provide a basis for policy planning
With the advent of generative artificial intelligence (GAI), the possibility of achieving this goal has never been closer. However, there are also challenges. One of these is how to incorporate a company's private, proprietary data into the functionality of GAI.
The Elasticsearch Relevance Engine™ (ESRE™ ) combined with generative AI question answering capabilities can help R&D teams develop powerful solutions. ESRE is designed to power AI-based search applications. It is used to apply semantic search with excellent relevance out of the box (no domain adaptation required), perform vector search, integrate with external large language model (LLM), implement hybrid search and use third-party or your own transformer models .
In this blog, you'll learn how ESRE and GAI can be used to enable R&D organizations to efficiently interact with data and make better decisions in the drug discovery process. The challenges of privacy and lack of context will be addressed.
While our blog post targets the pharmaceutical industry, the methods outlined are valid for any organization engaged in R&D.
contextualized challenges
In the field of intellectual property and innovation, researchers, professionals and analysts face some major challenges. Let's explore some of them.
Fragmented Information Landscape
Data related to patents and related fields is distributed across multiple sources, each with its own structure, format, and access methods. The patents themselves are often published by different national and international patent offices, making it cumbersome to gather comprehensive datasets for analysis. In addition, information related to prior art, patent litigation, and scientific literature can be found in various databases, publications, and repositories. The lack of a centralized and standardized system can lead to inefficiencies in data discovery, collection and integration.
The Power of Obfuscation
Some inventors may employ a strategic approach to obfuscate inventions in patent documents. Using ambiguous language, complex wording, and intentional ambiguity can allow them to obscure the inner workings and technical details of their creations. This willful darkness serves multiple purposes. First, it prevents competitors from easily copying the invention based solely on the patent description. Second, it provides inventors with the flexibility to adapt their claims to different backgrounds and emerging technologies. Finally, it introduces an element of uncertainty, adding complexity to potential infringers and increasing the likelihood of successful litigation.
hallucinations and general background
One of the challenges facing GAI is the phenomenon of hallucinations. Hallucinations are when an AI model generates output that sounds confident but is wrong, especially when it is asked domain-specific questions.
Here are the key points to understand about hallucinations:
limited contextual understanding
AI models like LLMs (Large Language Models) are trained on large amounts of data from various sources, but they may still lack comprehensive knowledge and understanding of specific domains such as healthcare and pharmaceuticals. When confronted with domain-specific problems or complex scenarios, models may provide plausible-sounding responses that are actually incorrect or misleading.
Dependence on training data
The quality and diversity of training data play a crucial role in the performance of AI models. If the training data does not adequately cover all relevant aspects or contains biases, the model may generate the illusion that it is consistent with patterns present in the training data but does not accurately reflect reality.
overconfidence in response
AI models, including LLMs, have shown a tendency to provide confident answers even when they are uncertain or when the question is beyond their knowledge. This overconfidence can lead to providing misleading or incorrect information to users, especially when they are unaware of the limitations of the model.
seek solutions
Data source integration
Elastic® offers flexible data ingestion options, enabling R&D teams to seamlessly ingest data from a variety of sources. It supports structured data formats such as JSON and CSV, as well as unstructured data such as text documents, image embeddings (dense vectors), and audio files. Elastic's rich ingestion APIs such as Bulk API, Agents, and Logstash® enable efficient and automated data ingestion processes.
In addition to the flexible data ingestion options mentioned above, Elastic also provides powerful capabilities for integrating website data through web crawling. With the help of web crawlers, R&D teams can automatically extract and ingest data from websites, expanding the range of information sources that can be analyzed. For a great example of a web crawler, read this blog post .
Situational visualization
Kibana® turns patent datasets into visually appealing and interactive insights. With its intuitive interface and diverse visualization options, researchers can easily explore and analyze patent trends, patterns and relationships. From dynamic charts and maps to powerful filtering capabilities, Kibana enables researchers to extract valuable knowledge, identify key players, and make data-driven decisions. Its collaboration capabilities enable seamless sharing and discussion of insights, facilitating cross-functional collaboration and accelerating research outcomes.
Deobfuscate novel aspects
Elasticsearch's significant text aggregation is a powerful feature that allows you to identify significant terms within a set of documents or specific fields. It helps you discover meaningful insights by highlighting statistically significant terms and distinguishing them from common or background terms.
Significant text aggregation works by comparing the term frequency of each term in the dataset with the expected frequency based on the background corpus. A background corpus is typically a larger collection or representative sample of documents used as a reference for determining which terms are common and which are important.
Below is an example of a patent dataset. The query might look like this:
GET patent-publications-1/_search
{
"size" : 0,
"query": {
"match": {
"abstract_localized.text": {
"query": "covid"
}
}
},
"aggs": {
"hidden_terms": {
"significant_text": {
"field": "claims_localized_html.text"
}
}
}
}The responses revealed a set of "hidden" terms used in the patent:
"aggregations": {
"hidden_terms": {
"doc_count": 218,
"bg_count": 14899225,
"buckets": [
{
"key": "covid",
"doc_count": 32,
"score": 3776.7204780742363,
"bg_count": 85
},
{
"key": "remdesivir",
"doc_count": 3,
"score": 564.3033204275735,
"bg_count": 5
},
{
"key": "coronavirus",
"doc_count": 7,
"score": 150.57538997839688,
"bg_count": 102
},
{
"key": "cov",
"doc_count": 6,
"score": 142.83756108838813,
"bg_count": 79
},
{
"key": "sars",
"doc_count": 7,
"score": 137.1282917578487,
"bg_count": 112
}
]
}
}With these terms, researchers can refine their investigations.
Furthermore, we will show how to integrate this powerful functionality with LLM.
A bridge between private data and generic LLMs
A Researcher's Best Friend: PatChat
For prior art searches and other functionality, our demo application, PatChat, demonstrates how researchers can interact with patents, scientific literature, clinical studies, litigation, and proprietary information. The approach in the demo shows how the company can address the above-mentioned challenges regarding illusions and general context. Elastic bridges this in a very elegant way, backed by ESRE features.
This application demonstrates an intuitive and efficient way to explore complex information environments using GAI. Researchers can ask specific questions, receive real-time and context-aware insights, and easily navigate data. The app's user-friendly interface, interactive visualizations, and domain customization help increase research productivity, facilitate discovery, and drive innovation across disciplines. Experience a game-changing approach to extracting valuable knowledge from disparate information sources with PatChat.
WARNING NOTE : Please consider this application a quick finish, no production use has been made so far.
vector search
Let's talk about some basics first. Before querying the LLM, we leverage vector search and a custom model to retrieve the context from the Elasticsearch service. Below is an example query used in the _search API call.
First, we build a classic query to search, filter and even aggregate specific fields:
query = {
"bool": {
"must": [
{
"match": {
"abstract_localized.text": {
"query": query_text,
"boost": 1
}
}
}
],
"filter": [
{
"exists": {
"field": "abstract-vector.predicted_value"
}
}
]
}
}Second, we define a KNN query based on the model:
knn = {
"field": "abstract-vector.predicted_value",
"k": 3,
"num_candidates": 20,
"query_vector_builder": {
"text_embedding": {
"model_id": "sentence-transformers__all-distilroberta-v1",
"model_text": query_text
}
},
"boost": 24
}Here are the key elements and what they mean:
1) query : This is a boolean query that searches the abstract_localized.text field for the provided query_text using the match clause. Its value is raised to 1, indicating its importance. The filter clause ensures that only documents where the abstract-vector.predicted_value field is present are considered.
2) knn : This is a K Nearest Neighbor (KNN) query for vector similarity search. It specifies the field abstract-vector.predicted_value as the vector field to search in. The parameters k and num_candidates determine the number of nearest neighbors and the number of candidates to be considered, respectively. All patent data ingested by Elastic in previous stages is processed by a Transformer model to enrich its vector representation.
In the mapping, the field vector is defined as dense_vector :
"vector": {
"type": "dense_vector",
"dims": 768,
"index": true,
"similarity": "cosine"
}3) query_vector_builder : This part aims to create dynamic text embeddings for user's queries, which are then compared with stored data vectors to find semantic matches. It specifies the model used for inference and the text to build the query vector. In this example, use the model with ID sentence-transformers__all-distilroberta-v1 and pass the user-prompted query as the model_text input.
4) boost : This parameter assigns a boost value of 24 to the KNN query, indicating its importance in the overall relevance score of the document.
Now we need to tie the two queries (match and knn queries) together:
resp = es.search(
index=index,
query=query,
knn=knn,
fields=fields,
size=1,
source=False
)This is all great, but how do we simplify it and enable a series of multiple interactions with the LLM?
A chain within a chain: LangChain
For the introduction of LangChain and the concept of specific SequentialChain, please refer to this document .
Here is a quick summary from the LangChain website:
LangChain is a framework for developing applications powered by language models. It supports the following applications:
- Data-aware: connecting language models to other data sources
- Agentic: Allows the language model to interact with its environment
According to the World Intellectual Property Organization, there are now several strategies for conducting prior art searches. For the purpose of this blog post, we will review some steps of the following strategies and how they are implemented through ESRE and LangChain:
- Identify concepts related to innovation
- Determine keywords
- Determine the IPC symbol
- […]
1. Define a chain to identify key concepts of innovation
This feature prompts you to identify key concepts in a patent-related context. It includes a template for asking questions about context and requires a language model (LLM) as input. This function returns an LLMChain object that is used to generate responses related to identifying key concepts in the patent given the context.
def step_1_identify_key_concepts_chain():
key_concepts_question = """
You are a scientist or patent layer.
It is your job to find prior art in patents.
The following context describes patents consisting of title, abstract, publication number and some other information: {context}
Identify the key concepts of the patent.
When providing an answer, put the publication number followed by the title first.
Ignore any citation publication numbers.
When asked: {question}
Your answer to the questions using only information in the context is: """
key_concepts_prompt =
PromptTemplate(template=key_concepts_question, input_variables=["context", "question"])
return LLMChain(prompt=key_concepts_prompt, llm=llm,
output_key="key_concepts")2. Define a chain to determine the keywords in the patent (for query generation)
This function prompts to generate keywords for further prior art research based on the given key concepts of the patent. It requires a language model (LLM) as input. This function returns an LLMChain object that is used to generate responses related to identifying patent research keywords based on the provided key concepts.
def step_2_identify_keywords_chain():
keywords_question = """
Given the key concepts of a patent, generate keywords for further prior art research.
Use synonyms and related keywords based on your knowledge.
Here are the identified key concepts: {key_concepts}
"""
keywords_prompt = PromptTemplate(input_variables =
['key_concepts'], template=keywords_question)
return LLMChain(prompt=keywords_prompt, llm=llm,
output_key="keywords")Now, let's use Elasticsearch's vector search capabilities to query similar documents based on the question and enter them into a sequential chain.
def ask_question(question):
similar_docs = es_db.similarity_search(question)
context= similar_docs[0].page_content
return chain({"question": question, "context": context})In response, you should be able to see something like this:
{"key_concepts": "\n The information processing method includes receiving a selection of a garment from a user, calculating a price of the garment on the basis of a type of the selected garment and parameters relating to a design and materials set for the garment, causing a display part to display the calculated price, and changing the parameters in conjunction with an input from the user. The display controlling includes causing the display part to display the price that reflects the changed parameters in response to the change of the parameters. The information processing method further includes generating an appearance image of the garment on the basis of the parameters, wherein the display controlling includes causing the display part to display the appearance image that reflects the changed parameters in response to an input from the user. Additionally, the price calculating includes calculating a price of the garment further on the basis of a parameter relating to a manufacturing process set for the garment.", "keywords": "\nKeywords:\nInformation processing, garment selection, price calculation, parameters, design, materials, display part, input, appearance image, manufacturing process."}3. Determine the IPC symbol
As a reminder, we will use keywords (LLM generated either contextually or via significant text aggregation, or both, or otherwise) to filter patents and then identify relevant IPC symbols. There are various ways to do this. One very compelling approach is using Kibana's ability to visualize data, as shown in this screenshot.
In Kibana, we use these terms to query datasets:
"information processing" or "garment selection" or "price calculation" or "display part" or "appearance image" or "manufacturing process"gamification shift
When curious researchers seek insights from the PatChat app, they can unlock delightful surprises by asking about innovations. With a mischievous grin, PatChat transforms into a digital Shakespearean poet, mesmerizing researchers with Romeo and Juliet-style answers. Each response is a whimsical amalgamation of academic knowledge and the poetic genius of the bard himself. From iambic pentameter to poetic metaphor, researchers are drawn into a world where academia and theater intertwine. It's a charming twist that adds a touch of humor and creativity to the quest for knowledge, showcasing PatChat's ability to surprise and delight. So, if you have the guts, feel free to ask PatChat about Shakespeare's innovations.
Imagine a researcher seeking patent-related information, but instead of a traditional response, they decide to unleash the hidden comedian in PatChat. With a twinkle in its virtual eye, PatChat transforms into a stand-up comedian ready to surprise researchers with a patent-related quip. PatChat deftly incorporates jokes and puns into patent discourse, and each answer becomes a hilarious amalgamation of wit, puns, and comedic timing.
For fans of Star Trek and admirers of Klingon culture and language, PatChat responds with the following:
getting Started
PatChat is the foundation upon which to build your own patent discovery solution. Feel free to experiment with the code , incorporate new features, and customize the application to your specific requirements. Unleash your creativity and open up the infinite possibilities of conversational patent exploration!
1. Create an Elastic cloud deployment
Follow the steps described in this blog post to create a resilient deployment in the cloud.
2. Load the model into Elasticsearch
Elasticsearch is perfectly capable of accepting models created by third parties and applying them to the data. For the patent use case, we will leverage the full distilarberta model. For loading the model the Python client Eland will be used . Here is the Python script to load the customer model into Elasticsearch:
from pathlib import Path
from eland.ml.pytorch import PyTorchModel
from eland.ml.pytorch.transformers import TransformerModel
from elasticsearch import Elasticsearch
from elasticsearch.client import MlClient
import os
import time
import requests
import shutil
# Run pip -q install eland elasticsearch sentence_transformers transformers torch==2.0.1
es_cloud_id = os.environ.get("ELASTICSEARCH_CLOUD_ID")
es_user = os.environ.get("ELASTICSEARCH_USERNAME")
es_pass = os.environ.get("ELASTICSEARCH_PASSWORD")
kibana_endpoint = os.environ.get("KIBANA_ENDPOINT")
es = Elasticsearch(cloud_id=es_cloud_id, basic_auth=(es_user, es_pass), timeout=180)
# Set the model name from Hugging Face and task type
# hf_model_id = 'anferico/bert-for-patents'
hf_model_id = 'sentence-transformers/all-distilroberta-v1'
tm = TransformerModel(hf_model_id, "text_embedding")
# Set the model ID as it is named in Elasticsearch
es_model_id = tm.elasticsearch_model_id()
# Download the model from Hugging Face
tmp_path = "../tmp/models"
Path(tmp_path).mkdir(parents=True, exist_ok=True)
model_path, config, vocab_path = tm.save(tmp_path)
# Load the model into Elasticsearch
ptm = PyTorchModel(es, es_model_id)
ptm.import_model(model_path=model_path, config_path=None, vocab_path=vocab_path, config=config)
# Start the model
while not MlClient.start_trained_model_deployment(es, model_id=es_model_id):
time.sleep(5)
# Synchronize the saved object (ML model) in Kibana to make it usable
api_endpoint = f"{kibana_endpoint}/api/ml/saved_objects/sync"
auth = (es_user, es_pass)
requests.get(api_endpoint, auth=auth)
# Cleanup
shutil.rmtree(tmp_path)3. Inference pipeline creation
Issue this command in Kibana's development console:
PUT _ingest/pipeline/patents-embeddings-inference-pipeline
{
"description": "Text embedding pipeline",
"processors": [
{
"inference": {
"model_id": "sentence-transformers__all-distilroberta-v1",
"target_field": "text_embedding",
"field_map": {
"text": "text_field"
}
}
},
{
"set":{
"field": "vector",
"copy_from": "text_embedding.predicted_value"
}
},
{
"remove": {
"field": "text_embedding"
}
}
],
"on_failure": [
{
"set": {
"description": "Index document to 'failed-<index>'",
"field": "_index",
"value": "failed-{
{
{_index}}}"
}
},
{
"set": {
"description": "Set error message",
"field": "ingest.failure",
"value": "{
{_ingest.on_failure_message}}"
}
}
]
}4. Create an index template
For simplicity, we will use a simplified schema. In the second part of the blog post, we will discuss the actual pattern of patents.
PUT _index_template/patents-embeddings-index-template
{
"index_patterns": ["patents-embeddings-*"],
"template": {
"settings": {
"index": {
"refresh_interval": "5s",
"number_of_replicas": 1,
"number_of_shards": 5,
"default_pipeline": "patents-embeddings-inference-pipeline"
}
},
"mappings": {
"properties": {
"vector": {
"type": "dense_vector",
"index": true,
"dims": 768,
"similarity": "cosine"
},
"text": {
"type": "text"
},
"metadata": {
"type": "object"
}
}
}
}
}Using the default_pipeline key, we specify the ingestion pipeline to use for preprocessing when indexing new documents matching this index template. The patent-embeddings-inference pipeline will perform inference through our chosen transformer model to enrich documents with vectors.
5. Create a test document
POST patents-embeddings-1/_doc
{
"text": """Patent title: Support brace accommodating a medical device
Patent publication number: US-2019308028-A1
Patent claims:What is claimed is:
1 . A support brace for a body part of a user comprising:
one or stretchable sections configured to accommodate the body part; and one or more attachment devices, each configured to receive a pain relief or rehabilitation assistance device.
2 . The support brace of claim 1 wherein:
the support brace is in the form of a sleeve.
3 . The support brace of claim 2 wherein:
the body part is the knee; and
the sleeve is configured to surround the knee of the user.
4 . The support brace of claim 1 wherein:
each of the one or more attachment devices is a pocket; and
each pocket is sewn to an interior surface of the support brace.
5 . The support brace of claim 4 wherein:
the pain relief or rehabilitation assistance device is an electromagnetic pulse therapy device.
6 . The support brace of claim 5 wherein:
each of the pockets are sewn such that the top to bottom is in the weft direction.
7 . The support brace of claim 5 wherein:
each of the pockets are sewn such that the top to bottom is in the warp direction.
8 . The support brace of claim 5 further comprising:
a slit located in each of the pockets sized to receive the electromagnetic pulse therapy device.
9 . The support brace of claim 8 wherein:
the one or more attachment devices comprises at least four pockets;
each of the at least four pockets extend along the support brace in a linear fashion; and
each of the at least four pockets are positioned to align with one or more areas of common injury or pain for the body part.
10 . The support brace of claim 1 further comprising:
one or more devices for securing the support brace to the body part.
11 . The support brace of claim 11 wherein:
each of the one or more devices for securing the support brace to the body part are stretchable bands.
12 . A support brace for a body part of a user comprising:
one or stretchable sections configured to accommodate the body part; multiple pockets sewn to an interior surface of the support brace; and a series of slits, each slit being located in one of the multiple pockets; wherein each pocket is configured to receive an electromagnetic pulse therapy device.
13 . The support brace of claim 12 wherein:
the stretchable sections are comprised of a four-way stretch material.
14 . The support brace of claim 12 wherein:
the multiple pockets comprise at least three pockets;
each of the at least three pockets extend along the interior surface in a linear fashion; and
each of the at least three pockets are positioned to align with one or more areas of common injury or pain for the body part.
15 . The support brace of claim 14 wherein:
the body part is the knee; and
the pockets are positioned to be aligned with the anterior cruciate ligament (ACL).
16 . The support brace of claim 14 wherein:
the body part is the knee; and
the pockets are positioned to be aligned with the posterior cruciate ligament (PCL).
17 . The support brace of claim 14 wherein:
the body part is the knee; and
the pockets are positioned to be aligned with the medial collateral ligament (MLC).
18 . The support brace of claim 14 wherein:
the body part is the knee; and
the pockets are positioned to be aligned with the lateral collateral ligament (LCL).
19 . The support brace of claim 14 wherein:
the body part is the knee; and
the pockets are positioned to be aligned with the lateral meniscus or the medial meniscus.
20 . The support brace for a knee of a user comprising:
a sleeve comprising:
one or stretchable sections configured to accommodate the knee of the user, wherein each of the stretchable sections are comprised of a four-way stretch material;
an upper stretchable band configured to be secured to the upper leg; and
a lower stretchable band configured to be secured to the lower leg;
a series of pockets sewn along an interior surface of the sleeve in a substantially linear fashion, wherein each of the series of pockets is configured to receive an electromagnetic pulse therapy device; and a slit located in each of the pockets, wherein the slit is configured to permit the electromagnetic pulse therapy device to be inserted into the pocket.""",
"metadata": {
"a": "b"
}
}6. Verification Test Documentation
GET patents-embeddings-1/_searchIn response, you should be able to see indexed documents with computed vectors.
"hits": [
{
"_index": "patents-embeddings-1",
"_id": "cwbTJ4kBBgs5oH35bQx1",
"_score": 1,
"_source": {
"metadata": {
"a": "b"
},
"vector": [
-0.0031025379430502653,
-0.019980276003479958,
-0.001701259519904852,
-0.02280360274016857,
-0.06336194276809692,
-0.040410757064819336,
-0.029582839459180832,
0.07966472208499908,
0.056912459433078766,
-0.06189639866352081,
-0.060898181051015854,
-0.0719321221113205,
[...]7. Launch the PatChat application
Don't forget to set the following environment variables before starting the application. In the demo application, we use OpenAI as an example. Any other LLM service can also be used. Please change the code accordingly.
export ELASTICSEARCH_CLOUD_ID=[YOUR_CLOUD_ID]
export ELASTICSEARCH_USERNAME=[YOUR_USER]
export ELASTICSEARCH_PASSWORD=[YOUR_PWD]
export OPENAI_API=[YOUR_OPENAI_ID]Now start the PatChat application as follows:
streamlit run patchat_gpt.pyexploration continues
After setting up the basic elements in this blog post, we'll build our story into a large-scale environment. Specifically, we will explore how to automate the deployment and extraction of a patent dataset containing over 100 million patents from the BigQuery service. In addition, we will explore how to integrate with other data sources using Elastic crawling capabilities. This is the type of deployment we'll be building in part two.
In addition to the experiments described in this blog post, there are other assets you may want to explore. For example, to build your own ElasticDocs GPT experience, sign up for an Elastic trial account , then check out this sample code repository to get started.
If you want to try out the idea of searching for relevance, try these two:
- [Blog] Deploying NLP Text Embedding and Vector Search with Elasticsearch
- [Blog] Using Elastic to Implement Image Similarity Search
The release and timing of any features or functionality described in this article is at the sole discretion of Elastic. Any features or functionality not currently available may not be delivered on time or at all.
In this blog post, we may have used third-party generative artificial intelligence tools that are owned and operated by their respective owners. Elastic has no control over third-party tools, and we are not responsible for their content, operation, or use, nor shall we be liable for any loss or damage that may arise from your use of such tools. Exercise caution when using artificial intelligence tools with personal, sensitive or confidential information.
Any data you submit may be used for artificial intelligence training or other purposes. There can be no guarantee that information you provide will be secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative artificial intelligence tool before using it.
Elastic, Elasticsearch and related marks are trademarks, logos or registered trademarks of Elasticsearch NV. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.
原文：How to implement similarity image search | Elastic.co | Elastic Blog