Cloud-inspired operating models, advanced flash technologies, cybersecurity and data insights are some of the hottest storage trends for enterprises in 2023. Infrastructure and operations leaders must prioritize these technologies and storage platforms to stay ahead of business needs.
overview
Chance
-
STaaS, container-native storage, and network storage for ransomware protection, detection, and recovery are increasingly being deployed to meet cloud-like operating models and ransomware protection requirements.
-
Advances in flash technology, including the use of QLC-based flash, TCP/IP-based NVMe, and Captive NVMe drives with onboard computing, are extending the applicability of flash to extreme performance and cost-sensitive use cases.
-
Modern unstructured data storage solutions enable better business insights and analytical workflows by supporting benefits such as file and object services, global namespaces, and metadata-based data classification on a single platform.
suggestion
-
Modernize your enterprise storage by investing in networked storage, staas, and container-native storage capabilities to address network security, cloud-like, and cloud-native operational needs.
-
Scale flash-based storage deployments by leveraging modern flash storage and NVMe over TCP for superior application performance, rack density, and power savings.
-
Eliminate unstructured data silos and enable faster analytical workflows and better data management by consolidating all unstructured data into a single platform with a global namespace.
Strategic Planning Assumptions
By 2028, consumption-based STaaS will replace more than 35% of enterprise storage capex, up from less than 10% in 2023.
By 2028, 100% of storage products will include network storage capabilities, up from 10% in early 2023, with a focus on active defense capabilities beyond recovering from cyber incidents.
Enterprises will use QLC in 25 percent of SSD flash media by 2027, up from 5 percent in late 2022.
By 2028, 70 percent of file and object data will be deployed on unified unstructured data storage platforms, up from 35 percent in early 2023.
By 2028, the volume of unstructured data in large enterprises' on-premises, edge and public cloud locations will be between 2023 and
By 2027, 60% of I&O leaders will have implemented hybrid cloud file deployments, up from 20% in early 2023.
By 2027, at least 40% of organizations will have deployed data storage management solutions for classification, insight and optimization, up from 15% in early 2023.
By 2027, 25% of enterprise organizations will deploy NVMe-oF as a storage networking protocol, up from less than 10% in mid-2023.
Preemptive NVMe SSDs will replace more than 30% of field-deployed capacity by 2026, up from less than 5% by mid-2023.
what you need to know
Organizations looking to modernize their storage infrastructure are experiencing some common challenges: some are unique to block or primary storage deployments, some are unique to file and object storage deployments, and some are common to all storage types. shared. Challenges come from managing explosive growth of enterprise data, business needs for an on-premises cloud consumption model to mitigate the risk of cyber attacks, or utilizing the latest storage technologies such as QLC or NVMe over fabric, which can achieve higher cost and/or performance . Modern storage products provide solutions to these challenges while continuing to deliver ever-increasing performance and lower total cost of ownership (TOC). These modern products and technologies enable I&O leaders to proactively respond to business needs by building flexible and agile storage platforms. Figure 1 summarizes the key trends under each storage class.
Figure 1: Top enterprise storage trends for 2023
Common Storage Trends:
The first trend: storage as a service , which brings the cloud consumption model to on-premises storage, thereby preserving enterprise feature sets, performance, and availability without the burden of lifecycle management or upfront capital expenditures.
The second trend: The low cost, energy efficiency and cooling of QLC flash further narrows the gap between flash and HDD, providing a cost-effective path to eliminate hybrid or HDD-based systems.
The third trend: Modern network storage technology further improves the prevention, detection and identification of network attacks on production storage and provides fast recovery from immutable snapshots or backup copies.
Development trend of unstructured data storage:
The first trend: Modern solutions provide a common platform for running both file and object data services .
Second trend: Application of data storage management solutions for data classification and creation of custom metadata for IT and/or business insights, as well as storage optimization, data life cycle enforcement, security risk reduction and faster data Workflow and other results.
The third trend: global namespace solutions provide a common view of all file or object data across geographic locations (including cloud and edge) - through centralized storage and edge caching devices or virtual global file systems .
Block Storage Trends:
The first trend: storage is to use NVMe-oF. NVMe-oF brings the benefits of microsecond latency to external storage over standard TCP/IP networks and provides a more scalable architecture for use in distributed hybrid platforms.
The second trend: Cloud-native container storage combined with data protection, data management, and integration with container orchestration tools such as Kubernetes, container-native storage is becoming the storage of choice for large-scale container deployments.
The third trend: captive SSDs offload I/O functions from the storage controller computer to free up resources to accelerate storage applications, including efficient use of flash memory capacity to reduce overall cost.
The trend briefs below explain each top-level trend, what it means, and the specific actions I&O guides must take when innovating modern storage technologies.
Trend overview
Gartner's top enterprise data storage trends for 2023
| Common |
unstructured data storage |
block storage |
Source: Gartner (June 2023)
Common Trend: Storage as a Service
Analysis by Jeff Vogel
SPA: Consumption-based STaaS will replace more than 35 percent of enterprise storage capex by 2028, up from less than 10 percent in 2023.
illustrate:
Managed storage as a service (STaaS) is a service in which a storage provider funds customers through a subscription and provides customers with access to a data storage platform for consumption. STaaS is a way for businesses to manage storage capacity and workloads without the upfront capital of storage hardware and software or the overhead of staff time. The service can be delivered on-premises or co-exist across a hybrid platform in an infrastructure dedicated to a single customer, or delivered from the public cloud as a subscription-purchased shared service billed under a pay-per-use license.
Why Trend:
Managed storage as a service (STaaS) eliminates upfront asset capital (Capex) costs, improves return on assets by maximizing asset utilization to match workload demands, and simplifies overall management. It also reduces storage usage costs by leveraging the vendor's investments in automation, AIOps and support. Managed STaaS is delivered by the storage provider's software and/or appliances, bringing enterprise feature sets, availability and performance -- while providing a cloud-like consumption model.
Influence:
Eliminates storage capex, streamlines budgeting and procurement processes, eliminates the impact of IT refresh/refresh cycles, and mitigates vendor AIOps management tools by increasing expertise and talent concerns
action:
-
Offerings from STaaS vendors before the next storage refresh, refresh, or refresh cycle to determine which one is right for your environment and replace Capex workload infrastructure.
Further reading:
Quick answer: How can I use storage as a service to reduce IT spending?
Infographic: 6 Ways to Procure and Deploy Enterprise Storage
Common Trend: Network Storage
Analysis by Julia Palmer
SPA: By 2028, 100% of storage products will include networked storage capabilities, focusing on active defense beyond cyber incident recovery, up from 10% in early 2023.
Description: Network storage solutions provide proactive technologies to identify, protect, detect, respond to and recover from ransomware and other cyber-attacks on structured and unstructured enterprise data storage solutions.
Why a trend: Ransom and other cyber-attacks have become a significant threat to businesses of all sizes. As valuable data is stored by enterprise storage systems for customers, storage devices are becoming prime targets for such attacks. Ransom or insider cyber attacks have become so prevalent that I&O leaders and storage professionals are being forced to take responsibility for data network security and adopt new proactive approaches to defense. Initially, storage leaders were not equipped to handle data security, but the new National Institute of Standards and Technology (NIST)-created Cyber Security Framework (CSF) provides organizations with information on how to identify, protect, detect, respond to, and recover from Attack. In October 2020, it released Special Publication (SP) 800-209, Security Guidelines for Storage Infrastructure, which includes comprehensive security recommendations for storage infrastructure. The growing focus on storage security has led to the emergence of new networked storage products and additional data services designed to identify, detect, protect, respond to, and recover from ransomware attacks.
Impact: Network recovery defense should be multi-layered and should not be limited to endpoint protection or backup recovery. Even today, most organizations rely on backups to recover from ransom attacks. Recovery can be devastatingly slow, is not 100% reliable and will not protect you from data extortion, exfiltration or advanced attacks. A new approach to proactive defense must include data network storage capabilities to effectively identify, protect, detect, respond to, and recover from any network anomalies or events on devices responsible for data services and data storage. Most major storage vendors are actively developing networked storage capabilities, which may be included in the storage system or enabled as a separate product. Additionally, most innovative startups are releasing products that can support heterogeneous solutions to protect enterprise data for block, file, and object storage.
action:
-
I&O leaders must evaluate network storage solutions as a new proactive defense mechanism to protect their most critical data. A cybersecurity assessment of storage must focus on the identification (such as vulnerability testing), protection (such as zero trust architecture and logging), detection (such as abnormal IO and data detection), response (such as alerting and blocking) and Advanced features for recovery such as forensic analysis and smart recovery.
Further reading:
The Cybersecurity Leader's Guide to Ransomware
2022 Storage Strategy Roadmap
Common trend: QLC all-flash replaces hybrid storage
Analysis by Jeff Vogel
SPA: Enterprises will use QLC in 25% of SSD flash media by 2027, up from 5% by the end of 2022.
Description: Quad-level cell (QLC)-based storage arrays, with software-enhanced ASIC or FPGA logic (to overcome lifetime limitations, enhance endurance, and improve performance), are now deployed for general-purpose block storage use cases. They are also deployed in less performance-critical backup and disaster recovery use cases. Replacement of TLC media device arrays. QLC-based storage arrays are increasingly replacing HDD disk arrays in file and object use cases such as analytics, backup, and disaster recovery, where the higher cost of flash over hybrid arrays is justified The performance reason for is simple and straightforward. QLC based arrays are becoming more and more popular for very intensive multi-byte use cases.
Why the trend: To take advantage of lower-cost flash media for a wider range of applications, storage vendors are using software and custom logic to overcome the limitations of QLC media. Compared to tri-level cell (TLC)-based arrays, the cost advantages of QLC, combined with enhanced durability and performance, provide sufficient long-term benefits for enterprises (e.g., rapid restoration of backups in case of recovery from ransomware incidents) data).
Impact: Arrays of QLC devices have the potential to replace an increasing number of TLC-based flash arrays and hybrid HDD arrays, as growth in unstructured data drives demand for higher-density SSD drives. The net impact on the cost of traditional hybrid HDD arrays is a decline in market share, while the growth of unstructured data is accelerating. This will lower the average selling price of general-purpose block storage arrays while increasing the average selling price of object storage arrays.
action:
-
Customers should reevaluate their application infrastructure environments based on use case and workload requirements and determine the overall price and performance benefits of using QLC-based storage versus hybrid HDD and TLC-based storage.
Further reading:
Magic Quadrant for Primary Storage
2022 Storage Strategy Roadmap
Unstructured Data Storage Trends: A Single Platform for Files and Objects
Chandra Mukhyala's analysis
SPA: By 2028, 70% of file and object data will be deployed on a unified unstructured data storage platform, up from 35% in early 2023.
Description: A modern unstructured data storage platform can support file and object workloads from a single platform, use a common key-value store, and run file and object services, providing read and write multi-protocol access through NFS, SMB, and S3 protocols. The storage layer, data services and protocol access layer are all tightly integrated and extensible without requiring any external gateways or third-party products.
Why Trend: Modern application workflows ingest, process, analyze, share and persist data through multiple protocols and rely on file and object services. End-user organizations find it difficult to partition workloads into file-only or object-only storage silos. A single platform for file and object data enables the consolidation of all unstructured data workloads, which simplifies not only storage operations but also storage sourcing activities.
Issues involved :
A unified and single file and object data platform will speed up modern application workflows. Speed occurs when applications can freely access data in any protocol, eliminating the need to copy data. I&O leadership will also be able to eliminate separate or specialized storage products that only offer object or file storage (except for niche use cases). For previous generation products, procurement activities for file and object storage products were separated. A modern, unified, single platform for unstructured data storage simplifies this into a single procurement activity.
action:
-
Identify standalone NAS, file servers, and object storage in production and build a list of requirements for a consolidated platform
-
Identify a single platform solution based on a common data repository for all unstructured data and allow multi-protocol access without relying on external gateways
-
Prioritize solutions that can be deployed anywhere: on-premises, public cloud or edge
-
Choose a solution with the enterprise feature set and APIs required for unstructured data workloads
Further reading:
Distributed File System and Object Storage Magic Quadrant
Unstructured Data Storage Trends: Hybrid Cloud File Data Services
Chandra Mukhyala's analysis
SPA: By 2027, 60% of I&O leaders will have implemented hybrid cloud file deployments, up from 20% in early 2023.
Description : The hybrid cloud file data service provides data access and data management across edge, cloud, and core data center locations through a single global namespace.
Why it's a trend: More and more businesses are creating, ingesting and accessing data at the edge, in factories, field offices and retail locations. Data services that analyze or augment data typically exist in public clouds, but workers collaborating on data are spread across many geographic locations, increasing the need for a single global namespace.
Impact: Hybrid cloud file services will eliminate the need to replicate data to multiple locations, either for data manipulation, enhancement, or simply to provide access to users of collaborative data. This will consolidate unstructured data into one copy, enabling centralized management around the protection and security of the underlying data, simplifying operations while consolidating use cases.
action:
-
Decide whether you want to keep your existing storage in various locations while still having global access, or prefer centralized storage and access from anywhere.
-
Invest in a hybrid cloud file solution to access unstructured data across edge locations, cloud and on-premises from a single global namespace.
Further reading:
Modernize file storage and data services for a hybrid cloud future
Hybrid Cloud Storage Market Guide
Unstructured Data Storage Trends: Data Storage Management Services
Chandra Mukhyala's analysis
SPA: By 2027, at least 40% of organizations will deploy data storage management solutions for classification, insight and optimization, up from 15% in early 2023.
Description: The Data Storage Management service (DSM) classifies data based on attached metadata and optionally a parses file or object content to identify specific information or patterns. Date C Slack or classification can help improve outcomes such as IT and business storage optimization, data lifecycle enforcement, security risk reduction, and faster data workflows. Data classification and insights solutions are typically vendor-agnostic Anything that can be accessed via a file or object protocol such as NFS, SMB or S3. This includes data in SaaS offerings such as Box, Dropbox, Google, and Microsoft 365. The solution can be a SaaS-based product or service on-premises software that can run on Any server or hypervisor. The underlying data itself can be on-premises or in a public cloud.
Why the trend : A growing number of I&O leaders are focused on understanding and effectively managing all the data their organization stores across premises, cloud and edge locations. Doing nothing exposes too many risks in terms of security and regulatory compliance. With the amount of unstructured data expected to triple over the next five years, I&O company leaders are looking for solutions to address storage costs. Additionally, they need solutions to ensure data protection and lifecycle management, but also to make data more productive to achieve business outcomes. This is a complex problem of adding additional capacity, performance, or protection compared to the core storage task, and these are all well-known solutions to the problem.
Influence:
Typical outcomes include cost optimization, which aligns storage costs with data value; data governance, which ensures the correct protection and retention policies apply to sensitive data; dATA security, which enables the correct level of privilege and access level control and leverages data classification and enhanced analytics workflows for labeling data with custom metadata.
action:
-
Create an inventory of all unstructured data sources within your organization, whether they are on-premises, in the public cloud, or on the web Also cloud offices like OneDrive, Dropbox, and Box.
-
Work with the business to understand how sensitive data is defined and how its lifecycle should be managed (such as storage location and retention policies).
-
Retention policies should be modernized or updated to address the multiple sources of information in use and enable specific actionable outcomes to be enforced within the DSM solution.
-
Create a list of must-answer questions, including from the business, to understand all the data your organization stores.
-
Work with analytics teams to speed up workflows by leveraging standard metadata, or integrating custom metadata within the data.
Further reading:
Hype Cycle Technology for Storage and Data Protection, 2022
Block Storage Trends: NVMe on Fabric
Analysis by Jeff Vogel
SPA: By 2027, 25 percent of enterprise organizations will deploy NVMe-of as a storage networking protocol, up from less than 10 percent by mid-2023.
Description: Networking for Non-Volatile Memory Representation (NVMe-of) is a networking protocol that takes advantage of the parallel access and low-latency properties of NVMe Peripheral Component Interconnect Express (PCIe) devices. NVMe - Tunnels NVMe commands to remote subsystems. The specification defines a protocol interface and is designed for use with high-performance Fabric technologies -- including Remote Directory Memory Access (RDMA) over Fiber Channel, Infiniband with ROCE v2, iWARP or TCP, or Ethernet. This enables front-end interfaces into storage systems, scaling to large numbers of NVMe devices, and extending distances within the data center to access NVMe-enabled subsystems. NVMe-of significantly improves data center network latency.
Why trend: NVMe-oF products address low-latency application needs when combined with NVMe drives. While infrastructure changes and upgrades are required, the clear advantages these technologies can offer are attracting high-performance requirements and using scalable architectures that can leverage the underlying network capabilities combined with NVMe flash media. The architecture provided by the NVMe-oF protocol scales and enhances storage capabilities in distributed disaggregated platforms.
Impact: The NVMe-of protocol accelerates the adoption of next-generation storage architectures such as block storage-computing to scale capacity and independent computing, use of software-defined storage, and hyper-converged and composable infrastructure. Of the NVMe-of options, NVMe-tcp benefits the most as field. The cost and simplicity of Ethernet is comparable to iSCSI and low-end Fiber Channel SAN bandwidth requirements, and even lower speeds of 16Gbps. In addition, NVMe-oF can also be scaled out to high capacity levels through high availability features, and from a central Managed from one location to serve dozens of computing clients.
Action: Determine the scalability and performance of NVMe-based solutions and workloads that justify the additional cost of such deployments. Use it for high-performance applications such as AI/ML, HPC, in-memory databases, transaction processing or as an alternative to iSCSI environments where managed STaaS can be used. Identify potential storage platform, network interface controller, host bus adapter, and network fabric vendors to verify that interoperability testing has been performed and provide references.
Further reading:
Hype Cycle Technology for Storage and Data Protection, 2022
2022 Storage Strategy Roadmap
Block Storage Trends: Capturing NVMe SSDs
Analysis by Jeff Vogel
SPA: Captive NVMe SSDs will replace more than 30% of field-deployed capacity by 2026, up from less than 5% by mid-2023.
Description: The mandatory NVMe solid-state drive integrates the processor core, which can greatly improve the onboard computing power, thereby handling I/O functions at the hard disk level. Captive NVMe drives are increasingly replacing standard SSD drives as a way to provide customers with storage-system-level capabilities that improve application functionality across a wide range of storage-system advantages. Proprietary NVMe SSD drive vendors design and build storage arrays starting with NAND, along with software to optimize the storage environment. The primary function of a mandatory NVMe SSD drive is to offload input/output (I/O) from the storage controller CPU.
Why the trend: Storage is increasingly seen as a strategic lever to address and enable key storage system features. Capturing NVMe drives leverages the vendor's ability to procure and integrate compute and analytics capabilities to increase the relative intelligence needed to optimize drives across many key operating factors. Future improvements significantly reduce hardware management and support costs and enhance data management through the use of advanced AIOPS capabilities and drive intelligence. Captive NVMe drives pave the way for more efficient and scalable storage system architectures, such as software-defined disaggregated storage-compute, leading to the adoption of NVMe-TCP.
Impact: Using dedicated NVMe SSD drives offers many potentially beneficial benefits, from enhanced storage operations and cost savings to a more network-resilient and smarter data storage serving environment. Vendor-specific approaches to NVMe drives require extensive and costly in-house engineering and procurement expertise that must be leveraged across the portfolio. To compete effectively with vertically integrated NAND/SSD suppliers, next-generation NAND technology and sufficient scale must be strategically acquired. Drawbacks that limit NVMe SSD drives include the high risk associated with single-source supply, including availability at the storage system level and switching costs when supply changes.
Dedicated NVMe SSD drives use special logic and complex AI/ML algorithms, which have the following advantages:
-
Cost advantage of using low-loss cost NAND flash such as QLC
-
Higher SSD drive density than standard SSD drives
-
Additional computing power for better SSD drive capabilities and performance
-
Computing capabilities such as efficient compression and deduplication
-
Granular flash management for enhanced durability and improved resiliency through limited over-provisioning
-
More efficient and flexible scale-out block storage architecture
-
Advanced cyber resilience capabilities such as filtering, searching, and media-level scanning
-
Real-time statistics of entropy changes of data stored on drives (increased security)
-
Improved NVMe application-aware end-to-end security features
-
Better power usage to offset CO2 emissions, and smart power and cooling
Action: The client should:
-
Assess the role of mandatory NVMe drives in storage systems and determine whether these benefits are critical to the overall vendor selection process.
-
Taking a proactive approach, evaluate how each vendor meets the tactical and strategic requirements for NVMe SSDs, and how those vendors' strategies address future storage system trends.
-
Conduct detailed due diligence on those risk and dependency factors that could negatively impact the procurement and ongoing supply requirements of critical application infrastructure requirements.
-
Ask if a proprietary NVMe SSD vendor has a multi-year roadmap and commitment, as well as a strong procurement team.
Further reading:
Emerging Technologies Hitting the Radar: Semiconductors and Electronics
Predictive Analytics: NAND Flash, Worldwide
Forecast: Global NAND Flash Market Statistics, Supply and Demand 2020-2027, 1Q23 Update
Block Storage Trends : Container Native Storage
Analysis by Julia Palmer
SpA: By 2027, 80% of Kubernetes deployments will require advanced features for persistent container storage, up from 30% in early 2023.
Description: Container-native storage is purpose-built to support container workloads, with a focus on addressing the unique cloud-native scale, granularity, and performance requirements while providing deep integration with the Kubernetes container management system. Container-native storage is designed to align with microservices architectural principles and meet the needs of cloud-native data services.
Why the trend: As Kubernetes and container adoption grow, I&O leaders are expressing interest in container-native data services that support stateful applications that require persistent container-native data services. Data services for containers are often deployed in public clouds or integrated as part of a service platform. However, when deploying on-premises, I&O teams need to understand how to design and deploy flexible, scalable, and elastic storage services to meet the unique needs of container-native applications.
Impact: While containers are built to be stateless, the number of deployments of stateful applications requiring persistent data is increasing. A good starting point for initial Kubernetes deployments is the traditional storage approach, where the Container Storage Interface (CSI) abstracts the underlying storage platform. However, legacy storage platforms may not fully meet all the needs of container-based application platforms. The container-native storage approach is designed to align with microservices architectural principles and meet the requirements of cloud-native data services.
Action: Change the traditional storage solution selection to meet the requirements of cloud-native data services while selecting a storage solution for Kubernetes deployments. The maximum requirements are:
-
software-defined and hardware-agnostic
-
Programmable, manageable as infrastructure-as-code, driven by APIs, and supports advanced granular data services
-
Based on a distributed architecture, it can be deployed at any scale
-
Interoperable, certified and fully integrated with a broad range of Kubernetes distributions
-
Follow a simple and predictable licensing model across environments
Further reading:
A CTO's Guide to the Cloud-Native Container Ecosystem
Cloud native infrastructure solution path using Kubernetes
Emerging Technologies Hitting the Radar: Cloud Native
evidence
Ask end-user customers to describe their storage challenges and desired outcomes.
Ask storage vendors to review product plans.
Introduce current capabilities to storage vendors.