Father of Ruby on Rails: "Even Amazon can't understand serverless or microservices."
A case study from the Amazon Prime Video team has created quite a stir in the developer community.
In this case, the Prime Video team migrated a monitoring system from a microservices architecture to a monolithic architecture and avoided using expensive services such as AWS Step Functions and Lambda serverless functions, and the cost reduction brought about by this move The effect was evaluated.
Their requirement was to use a monitoring tool to identify quality issues with "video streams viewed by users", which had to be highly scalable since there were "thousands of concurrent streams". The team initially built a distributed component solution orchestrated by AWS Step Functions, a serverless orchestration service based on state machines and tasks. It turns out that Step Functions is actually the bottleneck.
"For every second of video streaming, our service needs to perform multiple state transitions, so the account limit is quickly reached. Beyond that, AWS Step Functions bills users per state transition ’.” There’s another “cost issue” in that there are a lot of first-level calls to S3 (for temporary storage of captured video frames).
Initial Architecture for Prime Video Surveillance Application, Visibly Costly and Poorly Scalable
"We realized that a distributed architecture didn't benefit us much in this particular scenario, so we consolidated all components into a single process," thus removing the reliance on S3, the case article reads . "We also implemented orchestration to control components within a single instance." Now, the solution runs on EC2 (Elastic Compute Cloud) and ECS (Elastic Container Service), which has "a lightweight orchestration layer to distribute user ask".
The case article concludes: "Microservices and serverless components can be used in large-scale scenarios, but whether they must be used instead of monoliths needs to be analyzed on a case-by-case basis. Migrating services to monoliths allows us to base Facility costs have been reduced by over 90% and we have improved our ability to scale. ” The article also mentions their cost reduction through the EC2 Savings Program, which means that even internal AWS users are billed in a similar model to external users.
1 Opinions Differ: What's Wrong With Our Architecture Design?
The article was originally published in March, but it wasn't until this month that it caught the attention of the entire engineering and technology community.
A comment on Hacker News said: "I'm amazed that this article exists, and even a little bit dumbfounded".
In our opinion, AWS has always had a habit of touting microservices and serverless architecture as best practices for "modern" applications. For example, the "AWS Well-Architected" document has some suggestions under "Reliability":
"Build highly scalable and reliable workloads using Service-Oriented Architecture (SOA) or Microservice Architecture. Service-Oriented Architecture is the practice of making software components reusable through service interfaces. Microservice Architecture takes components further Smaller and simpler."
In the "AWS Prescriptive Guidance" document on modernizing .NET applications, Amazon lays out the benefits of microservices, including faster innovation, high availability and reliability, greater agility and on-demand scalability, modern CI/CD (continuous integration and deployment) pipelines, and strong module boundaries, though it also lists "operational complexity" as a drawback.
However, this case article seems to confirm some of the developer's suspicions. One of the suspicions is that the solutions recommended by AWS might not be the most cost-effective, since they always involve using multiple expensive services. Another suspicion is that the advantages of microservices over monolithic applications seem to be overstated.
Ruby on Rails author David Heinemeier Hansson, who has been advocating for less use of cloud services, commented on the Amazon case study that it " summarizes the theoretical microservices craze that once swept the tech industry . Now, The practical results of these theories are finally in. It is clear that in practice, microservices can introduce unnecessary complexity to a system. And serverless only makes it worse. One team, one application, but Replacing method calls and module separation with network calls and service partitioning is almost insane in either case."
"Even Amazon can't understand serverless or microservices."
In 2020, Sam Newman, author of "Building Microservices" and "Monolith to Microservices," among other books, told a developer conference that "microservices should not be the default choice " and gave software architects some advice on how to adopt microservices. Things to consider before architecting: "Have you done a value chain analysis? Have you seen where the bottlenecks are? Have you tried modularization? Microservices should be considered last."
Now, Newman has commented on the case article on Twitter: "This article is really more about the pricing model for functions vs. long-running virtual machines. It's still perfectly logical architecturally driven, but Lessons learned from this case may have relatively narrow applicability." He went on to write: "The reason why people don't talk publicly about moving away from microservices is that it might make some people think 'they're doing it wrong' .It is right to change your mind as the situation changes.”
Discussions about this article can be seen everywhere, it has provoked a lot of thinking, and there have been a lot of interesting comments and analysis on platforms such as Hacker News, Twitter, and Reddit.
As one netizen commented on Hacker News: "This is a good lesson, a good story, and it is very important that it comes from the Amazon internal team."
Lambros Petrou, a senior software engineer at DataDog, tweeted about the problems with the initial architectural design of the case: "The design in the Prime Video case article is problematic. Misuse of services doesn't solve architectural problems, it just It will bring out the problem." Former CTO Steve Chambers agrees somewhat, saying, "Basically, they're using the same architecture (now) and they're just stuffing components into containers so they don't run on different Expensive calls and data transfers over the network between cloud services... This is clearly an optimization!"
One engineer argued on this case on Reddit: "Microservices have overhead. What should have been simple inter-process communication, or a memory call between two parts of the system, becomes HTTPS, OAuth, JSON encoding/decoding , and these short conversations need to happen every time. When your system is broken down into 500,000 parts, every communication needs to do these things, and you pay for every transaction, the cost and complexity increases .Tearing down a monolith requires replacing the entire application in one shot, which means developers need to do some testing. But DevOps means no more testing, so in production there will be failures, and the only thing we can do is Making the functional blocks smaller makes it possible to find and fix problems quickly."
But overall, this case article isn't necessarily bad news for AWS. On the one hand, it seems like a departure from the best practices the cloud computing giant preaches, but on the other hand, it's also a refreshing take on how to keep costs down through simple architecture. Unlike many promotional case studies, this one looks to be really useful for AWS customers.
2 See what Amazon's CTO and former vice president of cloud architecture strategy say
This incident was widely circulated by everyone, and even alarmed Amazon CTO Dr. Werner Vogels.
He said that building an evolvable software system is a strategy, and we must re-examine our architecture with an open mind.
"Software architecture is not like the architecture of bridges and houses. Bridges are difficult to change after they are built, but software is different. Once the software is running, we can have a deeper understanding of our workloads, and then choose an architecture that can evolve, Make changes without impacting the customer experience. My rule of thumb is that with every order of magnitude of growth you should revisit your architecture and determine if it can still support the next order of magnitude of growth.”
“Prime Video is a good example. There is no one-size-fits-all approach (architecture design). We always push our engineers to find the best solution, and we don’t mandate a specific architectural style.”“I Just want to reiterate that there is no single architectural pattern that fits all situations... Monoliths are not dying (quite the opposite), and evolvable architectures are playing an increasingly important role in the ever-changing technology landscape."
After Dr. Werner Vogels expressed his opinion, Adrian Cockcroft, the former vice president of sustainable development architecture of Amazon Cloud Technology and former vice president of cloud architecture strategy, who announced his retirement last year, couldn't help but come out to comment.
"While there has been a mountain of opinions and opinions flooding the internet about the Prime Video case, most of them miss the point."
The Prime Video team simply followed a "Serverless First" principle: first try to use Step Functions and Lambda for rapid builds. Building a prototype in days or weeks with a serverless approach is the point. Then refactor when you need to deal with high traffic. There is a difference between "Serverless First" and "Serverless Only", "I don't advocate Serverless Only".
He believes that the key reason why this case has caused such a big reaction is that "when microservices are over-marketed, everyone thinks that the Prime Video team has returned to a monolithic architecture."
"I do think microservices are overhyped as the answer to every problem. And it's probably because vendors want to sell Kubernetes with a simple marketing message that cloud-native microservices need to be implemented using Kubernetes. There was a lot of backlash against this way of passing information. Kubernetes complexity comes at a price, and unless your team and size are very large, you don't need it. (Suggest reading Sam Newman's "Building Microservices: Design Details Granular Systems" book.)"
All in all, the original design by the Prime Video team was great, helping them get to market quickly and bring in revenue. And now, they're taking a step back for the next stage of analysis. "The team followed what I consider to be best practices", "I thought this post by the Prime team was very insightful..."
Reference link:
https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90
https://world.hey.com/dhh/even-amazon-can-t-make-sense-of-serverless-or-microservices-59625580
https://devclass.com/2023/05/05/reduce-costs-by-90-by-moving-from-microservices-to-monolith-amazon-internal-case-study-raises-eyebrows/
https://thestack.technology/amazon-prime-video-microservices-monolith/
https://news.ycombinator.com/item?id=35811741
https://twitter.com/samnewman/status/1654432661337788416
https://www.allthingsdistributed.com/2023/05/monoliths-are-not-dinosaurs.html
https://adrianco.medium.com/so-many-bad-takes-what-is-there-to-learn-from-the-prime-video-microservices-to-monolith-story-4bd0970423d4