All along, the RPC-based communications framework to achieve Akka Spark is proud contrast the main characteristics of distributed computing Hao, also like Hadoop framework and the process of a major bright spot, but in the era of technology and evolution, starting from Spark1.3.1 version, in order to address large blocks of data (e.g., shuffle) transmission problems, Spark introduced Netty communications framework, to version 1.6.0, Akka Netty actually completely replaced, all assume the interior Spark RPC communications and data streaming.
So Akka is what? Akka emerge from the background, it is the Actor's RPC-based communication system, its core concept is the Message, which is based on the Association process, performance beyond doubt; scala based on partial functions, ease of use and no words, but it is after all just RPC communication, data transmission can not be applied large package / stream, which is the reason Spark early introduction of Netty.
So Netty Why can replace Akka? First Akka is no doubt that can be done, Netty can do, but can be done Netty, Akka, but unable to do so because of what? In the software stack, Akka compared to Higher Netty point, it specifically for RPC to do a lot of things, but a little more basic compared to Netty, can support different application layer communication protocol (RPC, FTP, HTTP, etc.), in Akka earlier versions, the NIO is to use the underlying communications Netty; secondly an elegant system engineers will not allow a communication of the two receiving frame, nausea! Finally, although there is no Netty Akka coroutine level of performance advantages, but the internal Netty efficient Reactor threading model, serial no lock-oriented design, efficient serialization, zero-copy, memory pools feature also ensures that there will be no performance Netty problem.
So by Netty Spark is how to replace it Akka? Characteristics of a word, using the partial function, based Netty "fake" a simple version of the Actor model! !
