Faust is a streaming library, the idea of transplanting from Kafka Streams to Python.
It Robinhood used to build high-performance distributed systems and real-time data pipelines, processing billions of events per day.
Faust provide processing and event stream processing, sharing similarities with Kafka Streams, Apache Spark / Storm / Samza / Flink and other tools,
It does not use DSL, it's just Python! This means you can use all your favorite Python library when streaming: NumPy, PyTorch, Pandas, NLTK, Django, Flask, SQLAlchemy, ++
Faust requires Python 3.6 or later to use the new async / await syntax and variable type comments.
characteristic:
- Simple and easy to use
- Highly available
- fast
- flexibility
The following are examples of processing incoming order flow:
# Python Streams ٩(◕‿◕)۶
# Forever scalable event processing & in-memory durable K/V store;
# w/ asyncio & static typing.
import faust
app = faust.App('myapp', broker='kafka://localhost') # Models describe how messages are serialized: # {"account_id": "3fae-...", amount": 3} class Order(faust.Record): account_id: str amount: int @app.agent(value_type=Order) async def order(orders): async for order in orders: # process infinite stream of orders. print(f'Order for {order.account_id}: {order.amount}')