In this issue, Jesse wants to continue to bring you to the world of the Internet of Things, to have a look at data storage and the Internet of Things, and to introduce more hardware-level storage in the Internet of Things scenario. Without further ado, let us take a look Contents of this issue.
This article only represents personal opinions, if there is any bias, please forgive me~
The Internet of Things (IoT) includes all kinds of physical objects or devices that can be connected to the internet, from factory machinery, home appliances, and cars to mobile devices like smartphones and smartwatches. The Internet of Things today refers more specifically to interconnected devices that combine sensors, processing power, software, and exchange data with other devices. The Internet of Things is transforming various industries and applications, and the data generated by these devices can significantly improve business outcomes following the data analysis process.
Traditionally, connectivity has mainly relied on Wi-Fi wireless networks, but now types of network platforms such as 5G have gradually been able to process larger data sets at higher speeds and reliability. This will significantly boost IoT applications, resulting in massive amounts of data generated by end devices. Managing the data generated by IoT devices is key to delivering business intelligence and a better user experience. Therefore, we need to understand the flow of data in the world of IoT in depth.
Data Flow in an IoT World
Generally speaking, the IoT system architecture is divided into 3 layers. The first is the data source layer. IoT collects data from smart devices, environmental sensors, smartphones, smart vehicles, and various sensors. The data can then be sent over the network using common standard protocols such as MQTT, CoAP, and HTTP to the edge gateway and then to the cloud. The second is the data storage layer. This layer stores data collected from sensors and devices at the edge or cloud for long-term or short-term applications. Edge gateways provide functions such as sensor data aggregation, data preprocessing, and secure connection to the cloud. In the cloud, there are various database management systems built for IoT applications. These systems can store and manage these massive data for further application. Finally, there is the data analysis and application layer. Most organizations can use the cloud to run the applications needed to process data generated by devices. This layer analyzes data using artificial intelligence, machine learning, and underlying computing techniques to generate useful information. This data can help us provide actionable insights, but also drive business intelligence, optimize operations, attract more customers, automate control processes, and help businesses make the best decisions based on the results extracted from the data analysis layer.
From data acquisition, transmission, storage, calculation, analysis to application, data storage is only a part of the IoT ecosystem; however, it is still a very challenging technology. Storage solutions for IoT must ensure data integrity, reliability and security. What's more, storage solutions must adapt to the various environments of end devices, edge gateways, and data centers.
Key Storage Technologies for the Internet of Things
Moving data from endpoint devices to edge gateways and the cloud is a popular option in some cases. IoT devices usually have some ability to store and pre-process data. For IoT devices in production sites or transportation facilities, not only the output data are different, but also the sources are complex. Fast access speed is required, the accuracy of stored data is extremely high, and the installation environment of IoT devices is relatively harsh. NAND flash memory, which has gained a firm foothold in the consumer electronics market, is one of the storage solutions for IoT devices. However, many challenges remain.
First, the environment is the first challenge for IoT edge device storage. In the automotive world, vehicle-to-everything (V2X) is a perfect example of the concept of IoT. The car of the future will need to exchange information gathered by all of its sensors and exchange information with other vehicles, pedestrians and roadside units (RSUs). Storage devices in cars need to function properly in extreme temperatures. Therefore, AECQ-100 certification is required. Other applications such as factory automation require shock certification (MIL-STD 810G) to ensure storage performance in extreme environments.
Second, efficient data compression ratio is also one of the challenges. Because the collaboration between the cloud and the edge is very important, and when it comes to collaboration, we need to use network transmission, which consumes bandwidth, and a high data compression ratio means saving bandwidth consumption.
Third, reliability is a key factor to consider when applying storage solutions in various IoT applications. Typically, IoT devices are embedded systems designed for specific tasks, rather than servers or PCs designed for scalability and easy upgrades. That said, your storage should have a longer lifespan in IoT devices than products in the consumer market. There are several tricks to achieving high reliability, such as using industry-standard NAND flash, enabling pseudo-SLC mode, selecting high-quality components and fixing BOM and conformal coatings. Testing and verification are also necessary steps to deliver a high-reliability product, and the details and quality of these steps vary from vendor to vendor.
Fourth, data integrity is important. Especially timing data must be accurate for IoT applications. For example, acquiring telemetry data from a vehicle may produce different results when analyzed if the sequence of the data is not exactly consistent and accurate. Power-down protection, end-to-end data protection, ECC and other technologies are used to ensure the data integrity of storage devices. Experience designing these features is critical to the outcome.
Finally, security is an important issue when adopting IoT technology. Most IoT devices have limited computing power available. These restrictions prevent them from using basic security features, such as implementing firewalls or strong password systems to encrypt their data and communications with other devices. Storage solution providers can implement features such as write protection, secure erase, AES encryption, and TCG Opal 2.0 to ensure that private data generated by IoT devices remains safe.
Introduction to CnosDB
CnosDB is an open source distributed time series database with high performance and high usability, which has been officially released and fully open sourced.
Welcome to pay attention to our code warehouse, one-click three links: https://github.com/cnosdb/cnosdb