With distributed technologies evolving ([ɪ'vɒlvɪŋ])all the time, engineers are trying to push those technologies to their limits. Earlier, people were looking for faster, cheaper ways to process data. This need was satisfied when Hadoop was introduced. Everyone started using.Hadoop, started replacing their ETLs with Hadoop-bound ecosystem tools. Now that this need has been satisfied and Hadoop is used in production at so many companies, another need arose to process data in a streaming manner, which gave rise to technologies such as Apache Spark and Flink. Features, such as fast processing engines, the ability to scale in no time, and support for machine learning and graph technologies, are popularizing these technologies among the developer community
（随着分布式技术的持续发展，工程师们在尝试将这些技术推向极限。在这之前，人们寻求的是高性价比的方式去处理数据。这些伴随着Hadoop技术的面世将得到满足。几乎每个人都开始使用Hadoop-bound生态的工具替换他们的ETLs。现在这些需求都已经被满足并且Hadoop在很多公司被用于生产环境。另外，随着流处理需求的出现，催生了像Apache Spark 和Flink技术的出现，比如象快速处理引擎，在线扩展能力及机器学习和图技术，这些技术在开发者社区都非常受欢迎。）
Some of you might have been already using Apache Spark in your day-to-day life and might have been wondering if I have Spark why I need to use Flink? The question is quite expected and the comparison is natural. Let me try to answer that in brief. The very first thing we need to understand here is Flink is based on the streaming first principle which means it is real streaming processing engine and not a fast processing engine that collects streams as mini batches. Flink considers batch processing as a special case of streaming whereas it is vice-versa in the case of Spark. Likewise we will discover more such differences throughout this book
（有些人在日常工作中可能已经使用了Apache Spark，可能也在考虑我已经使用了Spark,为什么还要用Flink？这个问题也是我意料之中的，并且这种比较也很正常。我用简要的方式试着回答一下这个问题。首先，我们需要理解Flink是基于流优先原则的,这意味着这是一个真正的流处理引擎，并且不是那种以搜集流作为mini 批处理的快速处理引擎。Flink 考虑将批处理作为一种特殊的流处理引擎。在这Spark里正好相反。同样的，我们将在本书中发现更多的差异）
this book is about one of the most promising technologies-Apache Flink. In this chapter we are going to talk about the following topics :
(本书是关于这个最有前途的技术(Apache Flink) 在这一节我们将继续讨论以下几个主题)
- History (历史)
- Architecture （架构）
- Distrubuted execution(分布式执行)
- Features (特性)
- Quick start setup（快速开始）
- Cluster setup
- Running a sample application
History
Flink started as a research project named Stratosphere with the goal of building a next generation big data analytics platform at universities in the Berlin area. It was accepted as an Apache Incubator project on April 16, 2014. Initial versions of Stratosphere were based on a research paper by Nephele at http://stratosphere.eu/assets/papers/Nephele_09.pdf.
(Flink 始于一个叫Stratosphere的科研型项目旨在柏林地区的大学里，构建下一代的大数据分析平台。它在2014.4.16被Apache 作为孵化项目接受。Stratosphere的初始版本是基于Nephele的研究论文。
The following diagram shows how the evolution of Stratosphere happened over time:
下图展示了Stratosphere发展历程：
The very first version of Stratosphere was focused on having a runtime, optimizer, and the Java API. Later, as the platform got more mature, it started supporting execution on various local environments as well as on YARN. From version 0.6, Stratosphere was renamed Flink.The latest versions of Flink are focused on supporting various features such as batch processing, stream processing, graph processing, machine learning, and so on
（Stratosphere最初的版本专注在runtime optimizer和Java API,后来，随着平台越加成熟，开始支持在各种本地环境中运行 比如YARN。从0.6开始，Stratosphere 被命名为Flink,Flink的最后版本专注于各种特性的支持，如batch processing,stream processing,graph processing,machine leaning 等等）
Flink 0.7 introduced the most important feature of Flink that is, Flink's streaming API Initially release only had the Java API. Later releases started supporting Scala API as well.Now let's look the current architecture of Flink in the next section.
(0.7版引入的最重要的功能是，Flink 的流API最始的发布版只有JAVA API.后来的发布版支持了scale API. 现在，我们看一下当前版本的架构)
Architecture
Flink 1.X's architecture consists of various components such as deploy, core processing, and APIs. We can easily compare the latest architecture with Stratosphere's architecture and see its evolution. The following diagram shows the components, APIs, and libraries:
（Flink 1.x 的架构包括了很多组件，比如deploy,core processing 和一些API.我们比较容易比较Stratosphere与Flink最新版的架构，并可以看出它们的演变过程。下边的图显示了各种组件和API及包。）
Flink has a layered architecture where each component is a part of a specific layer. Each layer is built on top of the others for clear abstraction. Flink is designed to run on local machines, in a YARN cluster, or on the cloud, Runtime is Flink's core data processing engine that receives the program through APIs in the form of JobGraph. JobGraph is a simple parallel data flow with a set of tasks that produce and consume data streams.
Flink 的架构是分层架构，在这个构架里每个组件都是某层的组成部分。某一层都构建在其他层之前为了更清晰的抽象。Flink 的架构设计可以运行在本地机器上，也可以运行在YARN 集群，或者云，Runtime 是Flink的数据处理引擎核心，该引擎会通过API以JobGraph的形式接受应用程序。JobGraph是一个简单的并行data flow,该data flow是一组生产和消费data stream的任务的集合。
The DataStream and DataSet APIs are the interfaces programmers can use for defining the Job. JobGraphs are generated by these APIs when the programs are compiled. Once compiled, the DataSet API allows the optimizer to generate the optimal execution plan while DataStream API uses a stream build for efficient execution plans
(DataStream和DataSet
APIs 是程序可以用来定义job的接口。当程序被编译时，通过这些API生成JobGraphs。一旦编译完成，DataSet API 允许优化器生成最优的执行计划,而DataStream API 使用stream 构建有效的执行计划。)
The optimized JobGraph is then submitted to the executors according to the deployment model. You can choose a local, remote, or YARN mode of deployment. If you have a Hadoop cluster already running, it is always better to use a YARN mode of deployment.
（优化后的JobGraph随后会根据部署模型被提交给执行器，你可以选择local,remote or YARN 部署模型。如果你已经运行着hadoop cluster,那么使用YARN部署模型会比较好。）
Distributed execution
Flink's distributed execution consists of two important processes, master and worker. When a Flink program is executed, various processes take part in the execution, namely Job Manager, Task Manager, and Job Client
(Flink的分布式运行包括两个重要的进程，master 和worker。当Flink 应用程序被执行时，很多进行会参与执行，它们分别是Job Manager,Task Manager 和Job Client)
The following diagram shows the Flink program execution:
(下面显示了Flink程序的执行过程)
The Flink program needs to be submitted to a job Client. The Job Client then submits the job to the Job Manager. It's the Job Manager's responsibility to orchestrate the resource allocation and job execution. The very first thing it does is allocate the required resources.Once the resource allocation is done, the task is submitted to the respective the Task Manager. On receiving the task, the Task Manager initiates a thread to start the execution.While the execution is in place, the Task Managers keep on reporting the change of states to the Job Manager. There can be various states such as starting the execution, in progress, or finished. Once the job execution is complete, the results are sent back to the client.(Flink 应用程序需要被提交给Job Client.Job Client紧接着会被提交给Job Manager.协调资源分配和job执行是 Job Manager的职责。首先，它需要分配Job所需的资源。一旦资源分配完成，那么任务会被提交给相应的Task Manager. Task Manager 收到任务就会启动一个线程开始执行。execution就位的同时，Task Manager 会持续上报Job Manager状态的变化。会有很多种状态，比如 starting in progress 及finished。一旦job 执行完成,结果将返回给客户端。)
Job Manager
The master processes, also known as Job Managers, coordinate and manage the execution of the program. Their main responsibilities include scheduling tasks, managing checkpoints, failure recovery, and so on.
(master进程，也叫Job Managers用于协调和管理程序的执行。他们的主要职责包括任务调度，管理checkpoints,失败恢复等待。)
There can be many Masters running in parallel and sharing these responsibilities. This helps in achieving high availability. One of the masters needs to be the leader. If the leader node goes down, the master node (standby) will be elected as leader.（可能会有多个Maters并行运行，共享这些职责。这有助于实现高可用。其中的一个master需要成为leader.如果leader节点down掉，那么standby 的master节点会被选为leader.）
The Job Manager consists of the following important components:Job Manager包含以下重要的组件：
- Actor system
- Scheduler
- Check pointing
Flink internally uses the Akka actor system for communication between the Job Managers and the Task Managers.（Flink 内部使用Akka actor系统来实现Job Managers 和Task Managers的通信。）
Actor system
An actor system is a container of actors with various roles. It provides services such as scheduling, configuration, logging, and so on. It also contains a thread pool from where all actors are initiated. All actors reside in a hierarchy. Each newly created actor would be assigned to a parent. Actors talk to each other using a messaging system. Each actor has its own mailbox from where it reads all the messages. If the actors are local, the messages are shared through shared memory but if the actors are remote then messages are passed thought RPC calls.
（actor系统包括很多角色。它提供象scheduling. configuration.logging 这样的服务。它包含一个线程池，所有的actor都在这里启动。所有的actor都驻留在这一层。所有新创建的actor都会被指定一个parent。actors彼此用消息系统 通信.每个actor 有它自己的mailbox,它从这个mailbox里读取消息。如果actors 是在本地，那么消息通过共享内存传递，如果它是在远程，则通过rpc传递。）
Each parent is responsible for the supervision of its children. If any error happens with the children, the parent gets notified. If an actor can solve its own problem then it can restart its children. If it cannot solve the problem then it can escalate the issue to its own parent.
(每个parent都负责监护它们的actor。如果任何一个actor出问题，那么parent将被通知。如果actor自己能解决问题那么它就重启actor.如果解决不了，则上报给parent)
In Flink, an actor is a container having state and behavior. An actor's thread sequentially keeps on processing the messages it will receive in its mailbox. The state and the behavior are determined by the message it has received.
(在Flink中，actor是一个含有状态和行为的容器。actor线程可以顺序地并持续地处理它从mailbox里收到的消息。这个状态和行为由它收到的消息确定。译者注：这里象netty的event Loop,线程自己会有一个队列(mailbox))
Scheduler
Executors in Flink are defined as task slots. Each Task Manager needs to manage one or more task slots. Internally, Flink decides which tasks needs to share the slot and which tasks must be placed into a specific slot. It defines that through the SlotSharingGroup and CoLocationGroup.
(在flink中，Exectors 被定义为task slots。每个task manager 需要管理多个task slots。在内部,Flink 决定哪些任务需要共享slots，哪些任务必须运行在指定的slot里，这些通过SlotShardingGroup和CoLocationGroup来定义的。)