How can we synchronize accessing a common resource using synchronization in Java when my java application is deployed on multiple instances behind a load balancer ?
Because, as far as I know, synchronization works only under one JVM. But when we deploy the same Java application on multiple instances to handle the load, then how can we provide a synchronization mechanism?
For example:- There is a HDFS file to which the java application either appends/edits the contents of that HDFS file. When I deploy my java application on multiple instances, then how can I make sure that only one request from Java application accesses that HDFS file ?
Short answer - You can't do it without introducing a lot of complexities in your setup.
While technically you can use something like a distributed lock that's available on Zookeeper. I wouldn't really recommend them. It's kinda hard to reason with them at scale and there's also additional complexity in terms of Zookeeper Operations itself.
Regarding the example you posted, isn't that why systems like HBase was built? Model your data into a Key->[Multiple Columns] format. You can then read / write data on HBase and it would behind the scenes do the heavy lifting of editing / managing multiple files for you.
On the other hand, if you can model your change that you want to do on your file as an event you can then build your system on the principles of Event Driven Architecture.
You can read more about this on
- Introduction video on EDA from Martin Fowler - https://www.youtube.com/watch?v=STKCRSUsyP0
- Part 1 - https://www.confluent.io/blog/build-services-backbone-events/
- Part 2 - https://www.confluent.io/blog/apache-kafka-for-service-architectures/
- CQRS model that Martin Fowler talks about in this talk - https://martinfowler.com/bliki/CQRS.html