Celebrating 1000 stars on GitHub ⭐️

We're thrilled to achieve 1000 stars on GitHub, a symbol of trust and community in the tech world! Every line of code represents a shared journey of development, growth, and creativity. This milestone is a tribute to all who have contributed, supported, and believed in us. Thank you for all the stars (_and a special shout-out goes to our star champions 👏👏👏: Matt Palmer, Jove Zhong, Daniel Meyer, and Srini Kadamati), feedback, and encouragement. Our journey continues, fueled by a passion to keep improving and building our dream with the support of our community. For all new and old friends, let us share what we are doing again.

Haven't starred us? It's just a click away!

What is Bytewax

Bytewax is a Python framework that simplifies event and stream processing. Bytewax couples the stream and event processing capabilities of Flink, Spark, and Kafka Streams with the friendly and familiar interface of Python. Bytewax's core principle is Time To Value:

• Fast to ship. Bytewax runs locally on-prem, on the edge, or in the cloud. It runs on your existing infrastructure. no separate orchestration cluster is needed. With the command line tool, waxctl, you can deploy the same code remotely without any changes and manage your dataflows efficiently.

• Power of the Python Ecosystem. Seamless integration with Python ecosystems and tools. Easy to use APIs and developer-friendly documentation. You can re-use the Python libraries you already know and love!

• Lower footprint, full performance. Bytewax leverages a lightweight Rust processing engine, reducing both your infrastructure footprint and maintenance workload.

How Bytewax works

Here is the high-level architecture diagram of Bytewax:

Read more about our philosophy when building Bytewax and what we hope to achieve in our blog Whywax?

What Bytewax is used for

The use cases for Bytewax, as well as data streaming in general, are endless, from real-time analytics to IoT, leading companies from different industries leverage real-time data: financial services, e-commerce, media and entertainment, and more.

Bytewax connects to many input sources and output sinks while running in all kinds of environments. Explore top use cases.

Build RAG-powered (Retrieval Augmented Generation) apps

1. In just 2 hours, you can supercharge your SlackBots to leverage data you store in a vector DB. Check out our workshop. The recording is available on YouTube!
2. Read financial news in real-time, transform them into embeddings, store in Qdrant, and enrich your ChatGPT clone answers.
3. We're passionate about OSS 💛! Look at this example: a RAG ingestion pipeline for social media comments. This pipeline uses the strengths of OSS tools like Unstructured.IO for data extraction, HuggingFace Transformers for state-of-the-art natural language processing, and Milvus, a highly scalable vector database, everything powered by Bytewax.

Detect anomalies and monitor patterns

1. How we detect anomalies in our AWS infrastructure (And Have Peaceful Nights)
2. Build interactive dashboards to detect data anomalies in real time with Rerun
3. Monitor prolific Hacker News commenters and trending stories with Proton and Grafana

Getting started

Here are the instructions for installing and configuring Bytewax.

Bytewax currently supports the following versions of Python: 3.8, 3.9, 3.10, and 3.11.

We recommend creating a virtual environment for your project when installing Bytewax. For more information on setting up a virtual environment, see the Python documentation.

Once you have your environment set up, you can install Bytewax. We recommend that you pin the version of Bytewax that you are using in your project by specifying the version of Bytewax that you wish to install:

pip install bytewax==0.18.2

Write your first Hello, world!.

Learn more

• Visit our Github ⭐️: https://github.com/bytewax/bytewax
• Join us in Slack 💬: https://bytewaxcommunity.slack.com/ssb/redirect
• Discover our latest release 🌐: https://bytewax.io/blog/release-v0-18-0
• Check out our platform 🖥️: https://bytewax.io/platform
• Subscribe to the newsletter 📰: https://bytewax.substack.com/
• Explore our events 📅: https://bytewax.io/events
• Use our Bitly link 🔗: the one-stop for all things Bytewax - https://bit.ly/m/bytewax

Here's to more milestones together 🎉 fueled by our shared dedication to making our mark. Let's keep moving forward!