By Dain Perkins
Over the past few years we have been discussing the benefits of an open and transparent approach to security , i.e. providing the public with access to details of our detection and prevention capabilities, code, documentation, etc., will enhance what we are able to offer our customers security function. In this blog, we'll explore some of the latest ways our Open Security initiative has impacted the Elastic Security community, particularly the ever-present topics around generative AI and large language models (LLM), such as those provided by OpenAI and Microsoft .
Elastic AI Assistant
Users of Endgame may be familiar with the AI-assisted chatbot Artemis . Artemis provides Endgame users with an interactive chat experience that makes it easy for security analysts of all experience levels to perform guided investigations into endpoint data.
Elastic AI Assistant , provided with Elastic Stack version 8.8.1 , builds on the capabilities of Artemis and integrates the power of LLMs (Large Language Models) directly into analyst workflows. This gives every analyst the tools to more effectively triage, analyze and remediate in any environment.
How does this relate to open security? What really sets us apart from the competition is the public availability of all Elastic Security content, and how existing and new LLMs can leverage this knowledge to make Elastic AI Assistant an effective tool for modernizing security operations.
Data, detection, and artificial intelligence — oh my!
Let's start with the data
Elastic Security is built on Elastic Common Schema (ECS), an open source architecture that defines a common set of fields and data types for logs and metrics in Elasticsearch®. In short, the source ip is always source.ip, the process name is always process.name, etc.
In April 2023, Elastic contributed ECS to Open Telemetry and is working on joint development of a common schema. By contributing ECS to OpenTelemetry, we are working to create a mature common architecture for metrics, logs, traces, and security events. Together with OTel we will continue to develop and support this common schema.
Contributing to ECS is open to everyone, which is actually how I started working with Elastic five years ago. This open spirit encourages users, customers, and developers—all with different domain expertise and needs—to contribute their experiences to the areas they are most passionate about.
Because ECS is completely open, LLMs trained on publicly available data tend to have a good understanding of how Elastic Security stores and references data. For example, prompting the Elastic AI Assistant to help with queries to quickly analyze nginx network traffic provides novice users with the exact syntax needed, as well as a full description of the ECS fields used in the query.
Looks like we have another mystery
Analyzing the never-ending flood of suspicious events can be a daunting task for the most experienced analysts, while new hires often lack the security experience and institutional knowledge of the business environment needed to quickly triage and respond to threats. Elastic AI Assistant helps novice and seasoned analysts triage incidents faster by using information from our public detection repository, context for alert rules, risk ratings, and MITER ATT&CK® policy and technical information. The summary can even include recommendations for investigation based on the specific context of the alert.
In the screenshot below, the Elastic AI Assistant summarizes the "Suspicious DGA DNS Request" alert and provides initial investigation recommendations to analyze the potential impact of this alert.
Does it also cut and dice?
Well, no, but it can help:
- Write detection rules
- Simplify SIEM migration with fast and accurate rule conversion from other query languages
- Workflow suggestions for things like custom dashboards or ingestion pipelines
- Provides recommendations on which proxies to use to ingest a particular source
Customizable quick prompts enable our users to save and reuse the prompts that provide the most effective responses. In the example below, Elastic AI Assistant builds Event Query Language (EQL) rules based on the general use case of detecting "data exfiltration attempts on linux systems".
Generative AI and the Power of Open Security
Half the battle with generative AI is making sure it's trained on the right data. Our commitment to open security makes it as easy as possible for our customers to harness the power of Elastic AI Assistant -generated AI in their daily operations , empowering both novice and experienced users.
Open Security means more to us than public GitHub repositories. The combination of a standardized common taxonomy of events and alerts, public availability across all of Elastic Security, and the power of generative AI enables new levels of optimization and efficiency. Whether it's accelerating architecture and migration to reduce time to ROI, providing every analyst with an environment that accelerates triage and reduces MTTR, or empowers novice and expert users alike to simplify day-to-day operations; Open Security is a key component in modernizing security operations with Elastic.
starting from today
Elastic AI Assistant is now available to all users. For more information on how to integrate it with your model of choice and start harnessing the power of generative AI, read our documentation .
If you want to try it out for yourself, visit cloud.elastic.co and sign up for a free 14-day trial.
The release and timing of any features or functionality described in this article is at the sole discretion of Elastic. Any features or functionality not currently available may not be delivered on time or at all.
In this blog post, we may use or reference third-party generative artificial intelligence tools that are owned and operated by their respective owners. Elastic has no control over third-party tools, and we are not responsible for their content, operation, or use, nor shall we be liable for any loss or damage that may arise from your use of such tools. Exercise caution when using artificial intelligence tools with personal, sensitive or confidential information. Any data you submit may be used for artificial intelligence training or other purposes. There can be no guarantee that information you provide will be secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative artificial intelligence tool before using it.
Elastic, Elasticsearch, ESRE, Elasticsearch Relevance Engine, and related marks are trademarks, logos, or registered trademarks of Elasticsearch NV in the US and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.