Cloud object storage can be used as a powerful and very cheap database

Do you believe you can store hundreds of millions of records and read and write billions of records for as little as $5 per month with a fully managed, massively scalable, highly available, and inexpensive serverless database? If your database needs can be met by a very simple key-value store, then yes. The solution is to use cheap cloud object storage.

Amazon released their S3 (Simple Storage Service) object storage product in 2006. Since then, all other major cloud service providers, and many others, have offered object storage solutions.
Object storage can be used to store photos, videos, backups, etc., but it can also be used to store data. Data can be stored in any format you want such as JSON, BSON, XML, CSV, Protocol Buffers, MessagePack, plain text or any other data serialization format.
Objects are stored in buckets. Buckets can usually be created in data centers in specific regions of the world. This allows you to bring your data closer to the users who will be accessing the data. With some providers, the bucket can be configured as a website, allowing you to use object storage as inexpensive website hosting.

Price

The main advantage of object storage over serverless databases is price. Storage costs are typically 1 cent to 2 cents per GB, while outbound network costs for the cheapest option are typically 1 cent per GB.
Many object storage products also charge for read and write API requests. If you're using object storage as your database solution, this can add up significantly to your costs if you receive hundreds of millions or billions of requests per month, so it's best to avoid products that charge for read and write requests. There are many solutions that don't charge for requests. In the table below, the Read and Write columns represent cents per million requests. Products with a "-" have no request fee.

For comparison, here are the same pricing for some serverless databases. (See "Serverless databases for mobile or web applications.") Budget object storage offerings are available at a fraction of the price of serverless databases.

The table below shows the monthly costs for some object storage services and databases with different storage amounts and read and write requests. The record size is assumed to be 500, except for the last two columns, which have a record size of 100 small and 4000 large. This shows that pricing doesn't really change much until you start storing and reading and writing hundreds of millions of records. Beyond that, however, the product you choose can make a big difference.

access data

Most object storage products are compatible with Amazon's S3 REST API. The two main exceptions are Google and Azure. S3 compatibility means you can use tools and SDKs built for S3. You can also easily switch providers to avoid provider lock-in, or use multiple providers simply by changing the access URL.

The simple S3 REST API is ideal for mobile and web applications, providing direct access to data without a separate API server.
Objects are stored in buckets in a flat structure, without the hierarchy in the file system. However, you can simulate this by using a path separator, usually "/". For example, "photos/photo1.jpg" creates a simulated "photos" folder that contains "photo1.jpg".
The S3 API allows you to read, write, and delete objects. Objects are accessed by their unique key, probably something like a filename. Queries such as "SELECT * FROM Employees WHERE City='San Diego'" cannot be made. This can greatly limit how you can use object storage as a database, but in many applications all you need is to access data by id.

Record updates are also not possible with object storage. Any changes to a record are done by rewriting the entire record.

Buckets and objects have access permissions to limit who can read and write. Buckets and/or objects can be granted public access to allow anyone to read or write.

performance

Object storage performance is generally very good, even compared to regular databases. I verified the response time of some object storage products and found that the time was 50ms - 300ms. I found similar results with serverless databases. Although databases can typically access data in less than 10 milliseconds, when requests and responses travel over the Internet, access times increase significantly.

rate limit

Note that providers usually have a limit on the number of requests that can be made per second. The limit may be high enough that you won't be affected. Not all providers specify their rate limits. Google is the only company that claims they will scale to meet your request rate.
AWS can read 5,500 objects and write 3,500 objects per second per prefix (folder). If you need more requests, split the data into folders.

backup

Database backups are often performed to avoid data loss in case of server failure or to recover data from user or application errors.
Database durability is a measure of how well a provider can guarantee that your data will not be lost, and is usually expressed as a percentage that includes many 9s. For example, 10 nines will be 99.999999% durable. Among the major vendors, AWS and Google got 11 nines, and Azure got 12. Enhance durability by replicating data across data centers located in different geographic regions. Due to high durability, it may not be necessary to back up object store data.
Almost every object storage provider allows you to version objects. This means that every time an object is replaced with an update, the old object retains a version number, allowing it to go back to a previous version. This provides automatic backup in case of user or application error.

Product Details

AWS S3

AWS S3 is the gold standard against which all other object storage offerings are compared. Pricing for S3 is similar to similar offerings from other vendors.

Pros: Part of huge AWS system; stable and secure; very high rate limits; many data centers around the world
Cons: Request Fees

AWS Lightsail

Lightsail, a division of AWS, has servers and some other cloud offerings priced to compete with budget cloud providers like DigitalOcean. Lightstail products are also generally easier to set up and use than regular AWS products.
In July 2021, Lightsail announced an inexpensive object storage product based on AWS S3. This is a game changer in a way. You seem to get all the benefits of S3 at a lower price. Bundles are $1, 3 or 5 per month with 5, 100 or 250 GB of storage and 25, 250 or 500 GB of network transfer. Storage and networking beyond these limits are charged at normal S3 rates. However, there are no request fees within or outside of the bundle limit.
The $5 plan makes S3 priced similarly to the budget plans offered by DigitalOcean, Linode, and Vultr. However, unlike plans from other budget providers, Lightsail bundles appear to be limited to a single bucket. If you need to set up multiple buckets around the world, a good pricing strategy might be to set up multiple $1 buckets, then increase to $3 or $5 buckets as needed. Lightsail allows you to switch pricing plans once during your billing period.

Pros: Same as S3, but much cheaper; no application fee; $1 lower minimum price

Azure Blob Storage

Azure object storage is called blob storage.

Pros: Part of huge Azure system; stable and secure; very high rate limits; many data centers around the world
Cons: request fee; not compatible with S3

Google Cloud Storage

Google Cloud Storage is Google's object storage product.

Pros: Part of huge Google Cloud system, stable and secure, unlimited rate limits, many data centers around the world, free option in 3 regions (5GB storage, 5,000 writes/month, 50,000 reads/month , 1GB internet/month)
Cons: always free after throttling request fees; highest network rates of any provider; not s3 compatible

Firebase

Firebase, a Google company, offers a number of cloud products at reasonable prices that are generally easy to set up and use. With plenty of free options, many companies can use Firebase Cloud Storage for free.

Pros: Same as Google, but with better free options (20K writes/day, 50K reads/day, 1GB network/day)

Oracle Cloud

Oracle Cloud Object Storage is priced more favorably than other large providers.

Pros: Value for write requests; no network fees for the first 10TB/month, very cheap after that
Cons: request fees; high storage prices

alibaba

Alibaba's Object Storage Service (OSS) is a very cost-effective choice. There is a pay-as-you-go plan or one with storage and network limits for a little cost savings. There are 21 data centers in China, 10 of them.

Pros: Many data centers around the world; good pricing with no request fees unless you hit 500 million requests per month

DigitalOcean

DigitalOcean is a smaller cloud company that offers servers and databases that are easy to deploy quickly and at a lower price than the larger cloud providers. Their object storage, called Spaces, was the first to offer 250GB of storage, 1TB outbound transfers, and no request fees for an astoundingly low $5/month. Any storage or network that exceeds the limit is charged at a very lossy price. This fee structure is ideal for using object storage as a very cheap database.
One problem with Spaces is rate limiting. Each space (bucket) is limited to approximately 200 requests/sec. If this is a problem, you can work around it by using multiple spaces. You can create up to 100 spaces on the same account. (I've seen some users have problems accessing Spaces at rates well below 200/sec.)

Pros: Incredible value
Cons: Rate limiting may limit some users; only 5 data centers

Linode

Linode, similar to DigitalOcean, offers cloud services at a high price for ease of deployment. Their object storage offerings are priced the same as DigitalOcean.

Pros: Incredible value
Cons: Only 4 data centers

Vultr

Vultr is similar to DigitalOcean and Linode. Their object storage offerings are priced the same as the other two.

Pros: Incredible value
Cons: Only 1 data center in NJ

Exoscale

Exoscale's object storage is competitively priced across 6 data centers in Europe.

Pros: Value for money
Cons: Data centers are only in Europe

Scaleway

Scaleway's object storage has 3 data centers in Europe at very competitive prices. You get 75GB of storage and free internet per month.

Pros: Very good pricing and free tier
Cons: Data centers are only in Europe

Wasabi

Wasabi only does one thing, and that is object storage. They offer the best prices of any other provider. There are no charges for reading, writing or networking. The only cost is the storage cost of $0.006/GB, which is very cheap.
However, the pricing FAQ reveals some details that might make Wasabi unsuitable for use as a database. The minimum duration of an object is 90 days. If an object is deleted before 90 days, it is still counted as storage for the full 90 days. What happens if an object is updated multiple times within a 90 day period doesn't seem to be documented anywhere.
Then something like: "If your monthly egress data transfers are greater than your active storage, then your storage use case is not eligible for Wasabi's free egress policy." And: "If your use case regularly exceeds our free API requests policy guidelines, we reserve the right to limit or suspend your service."

Pros: Best price overall
Cons: Restrictions may prohibit using Wasabi as a database

Backblaze B2

Backblaze for computer backup and B2 object storage. B2 has the lowest storage price at 0.005GB/month and a very low download price at 0.01GB/month. However, it does charge a typical $0.40 per million for read requests. This might be a good option if you're not handling hundreds of millions or billions of requests per month.

Advantages: good price
Cons: read request fee; only 3 data centers; no choice of data center

To you

Tebi is a very new organization that only does object storage. Their main function is to replicate to other servers in real time to bring the data closer to the user. Unfortunately, they currently only have 4 data centers.

Pros: Great price; 25GB of storage and 250GB of internet are free
Cons: Only 4 data centers; very new so not time tested

note

If you're using object storage as a database solution, I'd love to hear your opinion. I've only done this so far on a web app using Cloud Storage for Firebase, and it works perfectly. I'm currently evaluating object storage options for a product that requires a considerable amount of storage space, potentially hundreds of millions of requests per month.

This article: https://architect.pub/how-create-very-inexpensive-serverless-database
Discussion: Knowledge Planet [Chief Architect Circle] or add WeChat trumpet [ca_cto] or add QQ group [792862318]
No public	[jiagoushipro] [Super Architect] Wonderful graphic and detailed explanation of architecture methodology, architecture practice, technical principles, and technical trends. We are waiting for you, please scan and pay attention.
WeChat trumpet	[ca_cea] Community of 50,000 people, discussing: enterprise architecture, cloud computing, big data, data science, Internet of Things, artificial intelligence, security, full-stack development, DevOps, digitalization.
QQ group	[285069459] In-depth exchange of enterprise architecture, business architecture, application architecture, data architecture, technical architecture, integration architecture, security architecture. And various emerging technologies such as big data, cloud computing, Internet of Things, artificial intelligence, etc. Join the QQ group to share valuable reports and dry goods.
video number	[Super Architect] Quickly understand the basic concepts, models, methods, and experiences related to architecture in 1 minute. 1 minute a day, the structure is familiar.
knowledge planet	[Chief Architect Circle] Ask big names, get in touch with them, or get private information sharing.
Himalayas	[Super Architect] Learn about the latest black technology information and architecture experience on the road or in the car.	[Intelligent moments, Mr. Architecture will talk to you about black technology]
knowledge planet	Meet more friends, workplace and technical chat.	Knowledge Planet【Workplace and Technology】
LinkedIn	Harry	https://www.linkedin.com/in/architect-harry/
LinkedIn group	LinkedIn Architecture Group	https://www.linkedin.com/groups/14209750/
Weibo‍‍	【Super Architect】	smart moment‍
Bilibili	【Super Architect】
Tik Tok	【cea_cio】Super Architect
quick worker	【cea_cio_cto】Super Architect
little red book	[cea_csa_cto] Super Architect
website	CIO (Chief Information Officer)	https://cio.ceo
website	CIOs, CTOs and CDOs	https://cioctocdo.com
website	Architect practical sharing	https://architect.pub
website	Programmer cloud development sharing	https://pgmr.cloud
website	Chief Architect Community	https://jiagoushi.pro
website	Application development and development platform	https://apaas.dev
website	Development Information Network	https://xinxi.dev
website	super architect	https://jiagou.dev
website	Enterprise technical training	https://peixun.dev
website	Programmer's Book	https://pgmr.pub
website	developer chat	https://blog.developer.chat
website	CPO Collection	https://cpo.work
website	chief security officer	https://cso.pub ‍
website	CIO cool	https://cio.cool
website	CDO information	https://cdo.fyi
website	CXO information	https://cxo.pub

Thank you for your attention, forwarding, likes and watching.

[Database] How to create a very cheap serverless database