akka-stream data streams may be a combination of some of the components together. These components are collectively referred dataflow graph Graph, which describes the data flow and processing stage. Source, Flow, Sink is the most basic Graph. Graph and can be combined with a base of more complex composite Graph. If a Graph all ports (input and output) are connected, then that it is a closed flow chart RunnableGraph, otherwise it belongs to the open-flow graph PartialGraph. A complete (be operational) data stream is a RunnableGraph. Graph output access port may be used to describe Shape:
/**
* A Shape describes the inlets and outlets of a [[Graph]]. In keeping with the
* philosophy that a Graph is a freely reusable blueprint, everything that
* matters from the outside are the connections that can be made with it,
* otherwise it is just a black box.
*/
abstract class Shape {
/**
* Scala API: get a list of all input ports
*/
def inlets: immutable.Seq[Inlet[_]]
/**
* Scala API: get a list of all output ports
*/
def outlets: immutable.Seq[Outlet[_]]
...
Shape type abstraction function inlets, outlets Graph representing the shape of the input and output ports. Listed below are several existing shape Shape aka-stream provided:
Final Case class SourceShape [+ T] (OUT: Outlet [T @uncheckedVariance]) {...} the extends the Shape
Final Case class FlowShape [-I, + O] (in: Inlet, [the I @uncheckedVariance], OUT: Outlet [O @uncheckedVariance]) {...} the extends the Shape
Final Case class SinkShape [-T] (in: Inlet, [T @uncheckedVariance]) {the extends the Shape. } ..
Sealed abstract class ClosedShape the extends the Shape
/ **
* A Bidirectional Flow of Elements and that consequently has TWO TWO Inputs
* Outputs, Arranged like the this:
*
* {{{
* + ------ +
* ~ In1> | | ~> Out1
* | BIDI |
* Out2 is <~ | | <~ In2 of
* + ------ +
* }}}
*/
final case class BidiShape[-In1, +Out1, -In2, +Out2](
in1: Inlet[In1 @uncheckedVariance],
out1: Outlet[Out1 @uncheckedVariance],
in2: Inlet[In2 @uncheckedVariance],
out2: Outlet[Out2 @uncheckedVariance]) extends Shape {...}
object UniformFanInShape {
def apply[I, O](outlet: Outlet[O], inlets: Inlet[I]*): UniformFanInShape[I, O] =
new UniformFanInShape(inlets.size, FanInShape.Ports(outlet, inlets.toList))
}
object UniformFanOutShape {
def apply[I, O](inlet: Inlet[I], outlets: Outlet[O]*): UniformFanOutShape[I, O] =
new UniformFanOutShape(outlets.size, FanOutShape.Ports(inlet, outlets.toList))
}
Shape是Graph类型的一个参数：
trait Graph[+S <: Shape, +M] {
/**
* Type-level accessor for the shape parameter of this graph.
*/
type Shape = S @uncheckedVariance
/**
* The shape of a graph is all that is externally visible: its inlets and outlets.
*/
def shape: S
...
RunnableGraph类型的Shape是ClosedShape：
/**
* Flow with attached input and output, can be executed.
*/
final case class RunnableGraph[+Mat](override val traversalBuilder: TraversalBuilder) extends Graph[ClosedShape, Mat] {
override def shape = ClosedShape
/**
* Transform only the materialized value of this RunnableGraph, leaving all other properties as they were.
*/
def mapMaterializedValue[Mat2](f: Mat ⇒ Mat2): RunnableGraph[Mat2] =
copy(traversalBuilder.transformMat(f.asInstanceOf[Any ⇒ Any]))
/**
* Run this flow and return the materialized instance from the flow.
*/
def run()(implicit materializer: Materializer): Mat = materializer.materialize(this)
...
We can provide with GraphDSL akka-stream to build Graph. GraphDSL inherited GraphApply create method, GraphDSL.create (...) is a method of constructing Graph:
object GraphDSL extends GraphApply {...}
trait GraphApply {
/**
* Creates a new [[Graph]] by passing a [[GraphDSL.Builder]] to the given create function.
*/
def create[S <: Shape]()(buildBlock: GraphDSL.Builder[NotUsed] ⇒ S): Graph[S, NotUsed] = {
val builder = new GraphDSL.Builder
val s = buildBlock(builder)
createGraph(s, builder)
}
...
def create[S <: Shape, Mat](g1: Graph[Shape, Mat])(buildBlock: GraphDSL.Builder[Mat] ⇒ (g1.Shape) ⇒ S): Graph[S, Mat] = {...}
def create[S <: Shape, Mat, M1, M2](g1: Graph[Shape, M1], g2: Graph[Shape, M2])(combineMat: (M1, M2) ⇒ Mat)(buildBlock: GraphDSL.Builder[Mat] ⇒ (g1.Shape, g2.Shape) ⇒ S): Graph[S, Mat] = {...}
...
def create[S <: Shape, Mat, M1, M2, M3, M4, M5](g1: Graph[Shape, M1], g2: Graph[Shape, M2], g3: Graph[Shape, M3], g4: Graph[Shape, M4], g5: Graph[Shape, M5])(combineMat: (M1, M2, M3, M4, M5) ⇒ Mat)(buildBlock: GraphDSL.Builder[Mat] ⇒ (g1.Shape, g2.Shape, g3.Shape, g4.Shape, g5.Shape) ⇒ S): Graph[S, Mat] = {
...}
buildBlock function type: buildBlock:? GraphDSL.Builder [Mat] ⇒ (g1.Shape, g2.Shape, ..., g5.Shape) ⇒ S, g open flow diagram representative of the combined treatment. Here are a few basic Graph Construction Test Example:
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
object SimpleGraphs extends App{
implicit val sys = ActorSystem("streamSys")
implicit val ec = sys.dispatcher
implicit val mat = ActorMaterializer()
val source = Source(1 to 10)
val flow = Flow[Int].map(_ * 2)
val sink = Sink.foreach(println)
val sourceGraph = GraphDSL.create(){implicit builder =>
import GraphDSL.Implicits._
val src = source.filter(_ % 2 == 0)
val pipe = builder.add(Flow[Int])
src ~> pipe.in
SourceShape(pipe.out)
}
Source.fromGraph(sourceGraph).runWith(sink).andThen{case _ => } // sys.terminate()}
val flowGraph = GraphDSL.create(){implicit builder =>
import GraphDSL.Implicits._
val pipe = builder.add(Flow[Int])
FlowShape(pipe.in,pipe.out)
}
val (_,fut) = Flow.fromGraph(flowGraph).runWith(source,sink)
fut.andThen{case _ => } //sys.terminate()}
val sinkGraph = GraphDSL.create(){implicit builder =>
import GraphDSL.Implicits._
val pipe = builder.add(Flow[Int])
pipe.out.map(_ * 3) ~> Sink.foreach(println)
SinkShape(pipe.in)
}
val fut1 = Sink.fromGraph(sinkGraph).runWith(source)
Thread.sleep(1000)
sys.terminate()
}
Above we demonstrate the preparation Graph Source, Flow, Sink, we use the Flow [Int] as a common basis for assembly. We know: akka-stream of Graph can be easier to combine Partial-Graph, Graph and all are ultimately base flow diagram Core-Graph as Source, Flow, Sink combination. The above example we are using builder.add (...) to be added to a Flow Graph Graph templates in an empty, builder.add return Shape pipe used to expose the input and output ports to be added in the Graph. Then we press the functional requirements of the target Graph port to connect the pipe to complete the design of a data flow diagram. Use these tests prove Graph functions in line with expectations. Now we can also try to customize a similar type Pipe Graph to a more detailed understanding of the process Graph combinations. All the basic components Core-Graph Shape must be defined to describe its input and output ports, the defined GraphStage GraphStateLogic described specific data stream read and write elements.
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
import scala.collection.immutable
case class PipeShape[In,Out](
in: Inlet[In],
out: Outlet[Out]) extends Shape {
override def inlets: immutable.Seq[Inlet[_]] = in :: Nil
override def outlets: immutable.Seq[Outlet[_]] = out :: Nil
override def deepCopy(): Shape =
PipeShape(
in = in.carbonCopy(),
out = out.carbonCopy()
)
}
PipeShape has one input port and one output port. Because it inherits the Shape class must implement the abstract function of the Shape class. Suppose we design a Graph, a function can be used to provide user input elements administration, such as:. Source.via (ApplyPipe (myFunc)) runWith (sink). Of course, we can directly use source.map (r => myFunc) .runWith (sink), but we need is: ApplyPipe may be involved in a number of pre-set sharing, and is one part of the code myFunc. Then the user must provide all of the code if the map (...). ApplyPipe shape is PipeShape, following its GraphState design:
class Pipe[In, Out](f: In => Out) extends GraphStage[PipeShape[In, Out]] {
val in = Inlet[In]("Pipe.in")
val out = Outlet[Out]("Pipe.out")
override def shape = PipeShape(in, out)
override def initialAttributes: Attributes = Attributes.none
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic =
new GraphStageLogic(shape) with InHandler with OutHandler {
private def decider =
inheritedAttributes.get[SupervisionStrategy].map(_.decider).getOrElse(Supervision.stoppingDecider)
override def onPull(): Unit = pull(in)
override def onPush(): Unit = {
try {
push(out, f(grab(in)))
}
catch {
case NonFatal(ex) ⇒ decider(ex) match {
case Supervision.Stop ⇒ failStage(ex)
case _ ⇒ pull(in)
}
}
}
setHandlers(in,out, this)
}
}
In this Pipe GraphStage defined in first define the input and output ports in, out, and then to define GraphStageLogic, InHandler, outHandler by createLogic. InHandler OutHandler and output ports corresponding to the input data processing mode active elements:
/ **
* AN INPUT Collection of the callbacks for Port A of [[GraphStage]]
* /
trait InHandler {
/ **
* When the Called The INPUT A Port has the Available at The Element Element Actual new new CAN BE retrieved at The Via.
* [[GraphStageLogic.grab ()]] Method,.
* /
@throws (classOf [Exception])
DEF the OnPush (): Unit
/ **
* Called the when the INPUT at The Port IS Finished. NO OTHER the After the this the callback BE Called Will the callbacks for the this Port.
* /
@throws (classOf [Exception])
def onUpstreamFinish(): Unit = GraphInterpreter.currentInterpreter.activeStage.completeStage()
/**
* Called when the input port has failed. After this callback no other callbacks will be called for this port.
*/
@throws(classOf[Exception])
def onUpstreamFailure(ex: Throwable): Unit = GraphInterpreter.currentInterpreter.activeStage.failStage(ex)
}
/**
* Collection of callbacks for an output port of a [[GraphStage]]
*/
trait OutHandler {
/**
* Called when the output port has received a pull, and therefore ready to emit an element, i.e. [[GraphStageLogic.push()]]
* is now allowed to be called on this port.
*/
@throws(classOf[Exception])
def onPull(): Unit
/**
* Called when the output port will no longer accept any new elements. After this callback no other callbacks will
* be called for this port.
*/
@throws(classOf[Exception])
def onDownstreamFinish(): Unit = {
GraphInterpreter
.currentInterpreter
.activeStage
.completeStage()
}
}
akka-stream Graph input and output processing performs Reactive-Stream protocol. So we had better use akka-stream providing ready-pull, push to rewrite the abstract function onPull, onPush. SetHandlers then to set the input and output GraphStage handler Handler:
/ **
* Linear Stage for the Assign the callbacks for both [[Inlet,]] and [[Outlet]]
* /
Final DEF setHandlers protected (in: Inlet, [_ ], OUT: Outlet [_], Handler: InHandler with OutHandler): Unit = {
setHandler (in, Handler)
setHandler (OUT, Handler)
}
/ **
* Assigns The Events for the callbacks for AN [[Inlet,]]
* /
DEF setHandler protected Final (in: Inlet, [_], Handler: InHandler): Unit = {
handlers (in.id) Handler =
IF (! = null _interpreter) _interpreter.setHandler (Conn (in), Handler)
}
/**
* Assigns callbacks for the events for an [[Outlet]]
*/
final protected def setHandler(out: Outlet[_], handler: OutHandler): Unit = {
handlers(out.id + inCount) = handler
if (_interpreter != null) _interpreter.setHandler(conn(out), handler)
}
With the Shape and GraphStage we can build a Graph:
DEF applyPipe [the In, Out] (F: the In => Out) = GraphDSL.create () = {Implicit Builder>
Val = builder.add pipe (new new Pipe (F ))
FlowShape (pipe.in, pipe.out)
}
It may be directly used to combine a compound Graph:
RunnableGraph.fromGraph (
GraphDSL.create () = {Implicit Builder>
Import GraphDSL.Implicits._
Val the Source Source = (. 1 to 10)
Val = Sink.foreach sink (the println)
Val F : Int => Int. 3 * _ =
Val = pipeShape builder.add (new new Pipe [Int, Int] (F))
Source ~> pipeShape.in
pipeShape.out ~> sink
ClosedShape
}
) .run ()
整个例子源代码如下：
import akka.actor._
import akka.stream._
import akka.stream.scaladsl._
import akka.stream.ActorAttributes._
import akka.stream.stage._
import scala.collection.immutable
import scala.util.control.NonFatal
object PipeOps {
case class PipeShape[In, Out](
in: Inlet[In],
out: Outlet[Out]) extends Shape {
override def inlets: immutable.Seq[Inlet[_]] = in :: Nil
override def outlets: immutable.Seq[Outlet[_]] = out :: Nil
override def deepCopy(): Shape =
PipeShape(
in = in.carbonCopy(),
out = out.carbonCopy()
)
}
class Pipe[In, Out](f: In => Out) extends GraphStage[PipeShape[In, Out]] {
val in = Inlet[In]("Pipe.in")
val out = Outlet[Out]("Pipe.out")
override def shape = PipeShape(in, out)
override def initialAttributes: Attributes = Attributes.none
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic =
new GraphStageLogic(shape) with InHandler with OutHandler {
private def decider =
inheritedAttributes.get[SupervisionStrategy].map(_.decider).getOrElse(Supervision.stoppingDecider)
override def onPull(): Unit = pull(in)
override def onPush(): Unit = {
try {
push(out, f(grab(in)))
}
catch {
case NonFatal(ex) ⇒ decider(ex) match {
case Supervision.Stop ⇒ failStage(ex)
case _ ⇒ pull(in)
}
}
}
setHandlers(in,out, this)
}
}
def applyPipe[In,Out](f: In => Out) = GraphDSL.create() {implicit builder =>
val pipe = builder.add(new Pipe(f))
FlowShape(pipe.in,pipe.out)
}
}
object ShapeDemo1 extends App {
import PipeOps._
implicit val sys = ActorSystem("streamSys")
implicit val ec = sys.dispatcher
implicit val mat = ActorMaterializer()
RunnableGraph.fromGraph(
GraphDSL.create(){implicit builder =>
import GraphDSL.Implicits._
val source = Source(1 to 10)
val sink = Sink.foreach(println)
val f: Int => Int = _ * 3
val pipeShape = builder.add(new Pipe[Int,Int](f))
source ~> pipeShape.in
pipeShape.out~> sink
ClosedShape
}
).run()
val fut = Source(1 to 10).via(applyPipe[Int,Int](_ * 2)).runForeach(println)
scala.io.StdIn.readLine()
sys.terminate()
}
---------------------
Author: TIGER_XC
Source: CSDN
Original: https: //blog.csdn.net/TIGER_XC/article/details/77259839
Disclaimer: This article as a blogger original article, reproduced, please attach Bowen link!
