so amazing! Someone finally understood Hadoop + Spark + HBase + Netty + distributed

1. Hadoop combat

Hadoop is an open source distributed computing platform under the Apache Software Foundation. With Hadoop Distributed File System (HDFS, Hadoop Distributed Filesystem) and MapReduce (the open source implementation of Google MapReduce) as the core, Hadoop provides users with a transparent and low-level distributed infrastructure of the system. The high fault tolerance and high scalability of HDFS allow users to deploy Hadoop on inexpensive hardware to form a distributed system;

The MapReduce distributed programming model allows users to develop parallel applications without understanding the underlying details of the distributed system. Therefore, users can use Hadoop to easily organize computer resources to build their own distributed computing platform, and can make full use of the computing and storage capabilities of the cluster to complete the processing of massive data.

This book is a systematic and practical Hadoop tool book and reference book. The content is comprehensive, and the entire technical system of Hadoop is comprehensively explained, including not only the two core contents of HDFS and MapReduce, but also the contents of Hadoop-related subprojects such as Hive, HBase, Mahout, Pig, ZooKeeper, Avro, Chukwa, etc. . Strong practicality, carefully designed a large number of classic small cases for each knowledge point, easy to understand and strong operability.

table of Contents

• Introduction to Haoop
• Hadoop installation and configuration
• Haoop application case analysis
• MapReduce computing model
• Developing MapReduce applications
• MapRe duce application case
• MapRe duce. Working mechanism
• Hadoop I / 0 operation
• HDFS explained
• Hadoop management
• Hive Detailed
• Detailed HBase
• Mahout detailed
• Pig Detailed
• Zooeeper detailed
• Avro explained
• Chulkv a detailed explanation
• Common plug-ins and development of Hadoop

 

 

Second, Spark big data analysis combat

Spark big data technology is still developing in full swing. The Spark China Summit is held and meetups are held everywhere. The open source software Spark is also rising. Many companies have already implemented and applied Spark on a large scale. The needs of Spark users have been from the initial deployment and installation, running examples, and now more and more need to build rich data analysis applications through Spark. Writing a technical book on Spark practical cases is an idea that lasted a long time. Due to the relatively tight work, I initially only summarized the Spark related cases that I participated in or studied, but over time, I still intend to abstract the common algorithms, system architecture and application scenarios, and simplify them appropriately. A summary and sharing.

table of Contents

• Introduction to Spark
• Spark development and environment configuration program
• Introduction to BDAS
• Lamda architecture log analysis pipeline
• Recommendation system based on cloud platform and user log
• Twitter sentiment analysis
• Hot News Analysis System
• Building a distributed collaborative filtering recommendation system
• Analysis of social networks based on Sparl
• Large-scale news topic analysis based on Spark
• Building a distributed search engine

 

 

Three, Hbase

HBase is a kind of NoSQL storage system, specially designed to read and write large-scale data quickly and randomly. HBase runs on ordinary commercial servers and can be smoothly expanded to support data sets ranging from medium to billions of rows and millions of columns. .

This book is a guide based on experience, which teaches readers how to use HBase to design, build and run big data application systems. The book is divided into 4 parts. The first two parts introduce the development history of distributed systems and large-scale data processing, explain the basic principle of HBase mode design and how to use the advanced features of HBase; the third part through real applications and code examples and support these practical skills The theoretical knowledge further explores some practical technologies of HBase; the fourth part explains how to upgrade the prototype development system to a plump production system.

table of Contents

The first part of HBase basics

1. HBase introduction
2. getting Started
3. Distributed Base, HDFS and MapReduce

Part Two Advanced Concepts

1. HBase table design
2. Extend iBase with coprocessor
3. Other HBase client options

Part III Application System Examples

1. Learn HBase by example: OpenTSDB
2. Query geographic information system on HBase

Part 4 Get HBase running

1. Deploy IBase
2. O & M

 

 

4. Principle, design and actual combat of distributed service architecture

This book takes the current popular distributed service architecture as the main line and explains the principles, design and practice of distributed service architecture.

This book first introduces the background and evolution of the distributed service architecture, and then deeply describes the design ideas and implementable solutions to ensure the consistency, high performance, high availability, etc. of the distributed service; then introduces the large-scale, high-performance Concurrent online service emergency process and technical breakthrough process, and gives an effective and common tool set for finding and locating problems; finally, it introduces in detail the containerized process analysis, agile development and online tools in the distributed service architecture. Developers with high concurrent service architecture provide convenience

table of Contents

• Design principle of distributed microservice architecture
• Completely solve the problem of distributed system consistency
• Service system capacity assessment and performance guarantee
• Construction of big data log system
• Design and Implementation of Service Management System Based on Call Chain
• Online emergency and technical research of Java services
• Service containerization process
• Agile development 2. Automation tools

 

 

Five, Netty actual combat principle

Netty is a Java framework for rapid development of high-performance network applications. It encapsulates the complexity of network programming, so that the latest developments in network programming and Web technology can be touched by a wider range of developers than ever before. Netty is not just a collection of interfaces and classes; it also defines an architectural model and a rich set of design patterns. But until now, there is still a lack of a comprehensive and systematic user guide, which has become an obstacle to getting started with Netty. This situation is also intended to change this book. In addition to explaining the details of the framework's components and APIs, this book will also show how Netty can help developers write more efficient, reusable, and maintainable code.

table of Contents

The first part of Netty's concept and architecture

1. Netty--asynchronous and event-driven
2. Your first Hetty app
3. Netty components and design
4. transmission
5. ByteBuf
6. ChannelHandler和 ChannelPipeline
7. EventLoop and threading model
8. guide
9. unit test

Part 2 codec

1. Codec framework
2. Preset ChannelHandler and codec

Part Three Network Protocol

1. WebSocket
2. Broadcast events using VDP

Part IV Case Study

1. Case study, part one
2. Case study, part two

 

 

These Internet industries are very hot and the popular PDF editors have sorted it out for everyone. Friends who need to receive likes, forwarding and forwarding editors can write this article. Private editor "architecture books".