1. storm how to protect the message fully processed?
Whenever spout generate a tuple of tuples, as a string anchor incoming call ack ack method and method, and the ack tupleid process 64-bit binary code XORed, ack method will each bolt processed tuple of tuples and generates a new call ack method XORed tuple, the tuple, when the tuple Tree processed, ack method of binary codes necessarily 0, if the binary code is not 0, the ack method notifies spout, spout distributed incoming anchor, the anchor will spout corresponding tuple resent (for all downstream Bolt), if the binary code is 0, the tuple is notified spout fully processed tuple . (only to finally know the results, we can only all retransmission, halfway stateless record)
2. storm in drpc what to do with? What is architecture?
It is used to implement the interface for real-time access of the storm, and drpc client establishes the connection, the receiving client demand by the drpc, drpc spout connection node and forwarded to the customer's needs spout node (included with request-id), spout connection data the source, and pulling data tuple created tuples, the received request-id tuple into tuple collectively sealed, Bolt node receives tuple tuple business process, the processing result is sent to the downstream resultbolt, resultbolt receiving element tuple group summarize, by request-id and sends the result to the corresponding drpc result, the reception by the drpc result, by request-id corresponding to the response back to the client
client---->drpc---->spolt---->bolt
| |
<---------------resultbolt
3. DETAILED DESCRIPTION kafka architecture
Client ---> producer ---> --- broker> Consumer ---> Storm
kafka node generates a message producer by producer (acquisition) and transmit data to the broker server node, broker the received data is stored in the corresponding topic theme (to be created in advance), the connection broker node by the consumer news consumer to obtain data, conduct their own business process or sent to other cluster for processing.
4. DETAILED DESCRIPTION kafka Message Queuing installation steps
java environment deployment
zookeeper deployment environment
download kafka compressed
decompression
modify server.properties file
broker.id (middleware message number)
zookeeper.connect (zookeeper cluster address)
start
5. The detailed overall framework of telecommunications projects
acquired by kafka producer of news producers call quality data, uploaded to kafka the broker messaging middleware node, create a spout node corresponds kafkaconsumer node by storm, get kafka cluster node broker of the spout the data node, to the downstream node transmissions bolt, bolt node by specific business process data, the result is transmitted to the cluster hbase save, which is connected by a web hbase cluster read data with the data item results are shown.