Fully compatible with DynamoDB protocol! GaussDB (for Cassandra) breathes new life into NoSQL

Abstract: DynamoDB is a managed NoSQL database service that supports multiple data models and is widely used in e-commerce, social media, games, IoT and other scenarios.

This article is shared from Huawei Cloud Community " Fully Compatible with DynamoDB Protocol!" GaussDB (for Cassandra) breathes new life into NoSQL ", author: GaussDB database.

DynamoDB is a managed NoSQL database service that supports multiple data models and is widely used in e-commerce, social media, games, IoT and other scenarios. In order to further meet some potential needs of DynamoDB users and provide customers with more choices, Huawei Cloud's distributed NoSQL database service GaussDB (for Cassandra) has launched a feature compatible with DynamoDB to help users migrate from DynamoDB to GaussDB (for Cassandra) more smoothly. ).

Reasons to use GaussDB(for Cassandra) DynamoDB

Why choose to migrate to GaussDB (for Cassandra)?

1. Low cost of use

With the development of business, the amount of enterprise data continues to increase, and the cost of accumulating DynamoDB read and write requests is also increasing day by day. However, GaussDB (for Cassandra) has a high cost performance, which can not only provide higher throughput, but also reduce the cost of using DynamoDB by half, especially when the amount of data is large, the cost reduction is more obvious.

2. More flexible and open deployment methods

DynamoDB is a fully managed service, and users need to follow certain deployment methods and rules. In contrast, GaussDB (for Cassandra) can flexibly deploy services according to the specific needs of users. Customers can choose to deploy on private clouds or public clouds instead of being locked on expensive platforms. Regardless of the deployment environment, GaussDB (for Cassandra) can guarantee the stability and efficiency of the service.

3. Smooth Migration

In addition to the above two points, GaussDB (for Cassandra) is fully compatible with DynamoDB's syntax and data model, so migrating from DynamoDB to GaussDB (for Cassandra) will become very easy, and the application does not require any modification, and the migration work will be easier Efficient and convenient, and more friendly to business.

How is GaussDB (for Cassandra) compatible with DynamoDB?

HUAWEI CLOUD GaussDB (for Cassandra) has added a protocol compatible with DynamoDB on the basis of being compatible with the Cassandra protocol. Therefore, users can run DynamoDB applications on GaussDB (for Cassandra), while maintaining the original CQL protocol. The overall structure of the service is shown in the figure below.

As can be seen from the figure, the Dynamo Rest Server module also adds a data model mapping module and an expression syntax semantic analysis module while receiving read and write requests initiated by users.

After enabling this feature on port 8000 (DynamoDB SDK default), first, each node will receive requests from the DynamoDB client on this port, and these HTTP requests will be parsed in JSON format, as well as syntax and semantic analysis of various expressions, Then it is converted through the data mapping model, and then the request is converted into a request of the underlying Storage Layer protocol, and finally the data is saved in the storage pool. Here are some key points:

Table mapping: DynamoDB tables will be mapped to GaussDB (for Cassandra) tables, and each table will be stored in a separate KeySpace. Whenever a user creates a corresponding DynamoDB table (CreateTable request), GaussDB (for Cassandra) will initialize the Keyspace and create the corresponding table.

Primary key mapping: DynamoDB primary keys (hash keys and sort keys) are known types and will be used as partition keys and cluster keys for GaussDB (for Cassandra) tables.

RMW request: DynamoDB provides some read-before-write requests (RMW requests), such as writing with conditional expressions, modification with conditional expressions, or writing or modification that needs to return the old value, these requests will LWT (Lightweight Transaction) is used to isolate each other, so for RMW requests, GaussDB (for Cassandra) can be completely consistent with DynamoDB.

Expressions: In DynamoDB, update expressions, conditional expressions, and projection expressions need to be parsed. Although regular expressions and handwritten recursive descent parsers can be used to parse these expressions, considering the diversity of different expressions, GaussDB (for Cassandra) will use ANTLR4 to generate parser code, which can be defined by the context-free Grammar rules, automatically generate parser code, so as to support DynamoDB expressions more flexibly.

Index: In essence, the index of DynamoDB is the same as the materialized view of GaussDB (for Cassandra). GaussDB (for Cassandra) can perfectly support local secondary index (LSI) and global secondary index at the interface level only by adding support for nested types. Level Index (GSI).

How to use GaussDB(for Cassandra) compatible DynamoDB features?

After purchasing a GaussDB(for Cassandra) instance, if you need to use features compatible with DynamoDB, you can submit a ticket and contact HUAWEI CLOUD O&M personnel to enable this feature.

It is recommended to use the boto3 client to connect to GaussDB (for Casssandra). boto3 is a Python client library for connecting to DynamoDB and some other services. Execute the following command to install boto3:

sudo pip install --upgrade boto3

Here's a simple example of writing and reading using boto3:

# 创建Client
client = boto3.resource('dynamodb', endpoint_url='url',
                        region_name='None', 
                        aws_access_key_id='key_id', 
                        aws_secret_access_key='access_key')
# 插入项目
response = client.put_item(
    TableName='TableName',
    Item={
        'PartitionKey': {'S': 'key1'},
        'SortKey': {'S': 'key2'},
        'attribute_name': {'S': 'attribute_value'}
    }
)
# 获取表中的项目
response = client.get_item(
    TableName='TableName',
    Key={
        'PartitionKey': {'S': 'key1'},
        'SortKey': {'S': 'key2'}
    }
)

It should be noted that the endpointurl, awsaccesskeyid and awssecretaccess_key here are the corresponding node address, user name and password of GaussDB (for Cassandra). In addition, GaussDB (for Cassandra) is completely consistent with DynamoDB at the syntax level.

Summarize

In terms of compatibility with the DynamoDB protocol, GaussDB (for Cassandra) strictly follows the syntax and data model of DynamoDB. Therefore, DynamoDB applications can be easily migrated to GaussDB (for Cassandra); in addition, GaussDB (for Cassandra) also provides a variety of automatic management and operation and maintenance functions, such as automatic backup, fault detection and fault tolerance, etc., which help users more Easily manage and maintain databases.

As a high-performance, low-cost, and open distributed NoSQL database, GaussDB (for Cassandra) can meet various data storage needs of users. It not only provides a reliable alternative for users who use DynamoDB, but also allows Users enjoy the advantages of high performance, high reliability and high expansion of GaussDB (for Cassandra).

Extra

On July 7th, Huawei Developer Conference 2023 (Cloud) will kick off, and will open branch venues in more than 30 domestic cities and more than 10 overseas countries. You are sincerely invited to participate in this unmissable annual developer event. Let's start a journey of discovery together!

We will join hands with developers, customers, and partners to present Huawei Cloud series products and services and rich innovation practices, and discuss with you AI, big data, database, PaaS, aPaaS, media services, cloud native, security, Internet of Things , blockchain, open source and other technical topics, to carry out comprehensive and in-depth exchanges.

The conference will bring together global scientists, industry leaders, technical experts, and community leaders, and set up more than 200 special events for developers, providing opportunities for global developers to communicate and cooperate face-to-face, and discuss technological innovation and business development together.

Conference official website: https://developer.huaweicloud.com/HDC.Cloud2023.html

Buy tickets for the conference: https://www.vmall.com/product/10086352254099.html?cid= 211761

Click to participate in the developer community activities, watch the technology master show, play with the technology dream factory, and have a chance to win a 4,000 yuan developer gift bag!

Welcome to follow the "Huawei Cloud Developer Alliance" official account to get the conference agenda, exciting activities and cutting-edge dry goods.

Click to follow and learn about Huawei Cloud's fresh technologies for the first time~