Redpanda One: a beginning

Upgrade inventory management with real-time technologies in just 7 steps

Explore how real-time data analytics helps businesses make smart, quick decisions

Pros, cons, emerging trends, and getting started

A friendly beginner's guide to real-time data streaming and how to get started

The simplest way to build a Customer 360 view for supercharged customer satisfaction and personalization

The life of a product manager at one of the next billion-dollar startups

Over 100 pre-built connectors now in limited availability for BYOC and beta for Serverless

Build powerful, flexible AI pipelines with Redpanda Connect

Mix and match different topics within the same cluster to optimize each workload for latency, cost, and performance

A single, multi-modal engine

Redpanda’s upcoming Iceberg integration makes accessing your streaming data radically more efficient

Move your workloads from any Kafka system to Redpanda with a single command

The Redpanda approach to protecting your data

The simplest way to handle large-scale data processing and visualization with Redpanda, Propel's Serverless ClickHouse, and Next.js

How we introduced sampling memory profiler into Seastar to debug and fix issues in Redpanda faster

Your cloud. Your data. Period.

How we built a simple yet efficient crypto data hub with Redpanda, Rust, and Samsa

Learn how to consume data from Redpanda then use Dremio for sales data analysis

New features for Redpanda Serverless, improved developer experience, enterprise compliance, and more

Sync your BigQuery data in real time to get your data instantly ready for action

A summary of the process and an easier path for those who want to skip the struggle

A blueprint to help you equip users with the right insights at the right time

Leave compromises in the dust with flexible per-topic controls and predictable durability guarantees

Pros, cons, and considerations when choosing the right ML framework

Learn how it works, popular use cases, and how to implement it in an async Python app

Connect, secure, and stream—all in one simple platform

The possibilities are endless when you have 220+ connectors at your fingertips — but here are 10 to get you started

Learn how to swiftly ingest and process insights on customer feedback in real time

Bringing the power of Redpanda Cloud even closer to developers

Learn why Zafin switched from Kafka to accelerate core modernization at world-leading banks

Announcing Redpanda Connect and the acquisition of benthos.dev

Learn how to build a real-time fraudulent click detection system using Redpanda, Spark, and Python

The skyrocketing SaaS industry demands adaptable architectures that can handle real-time data. Here’s how to build them with Redpanda.

Learn how to build a sales CDC pipeline with Debezium, Flink, and the simplest Kafka alternative

The simplest implementation with no external dependencies, nothing extra to manage, and no code to write

Flexible write caching, role-based access control, and the GA of Data Transforms powered by WebAssembly

Get insights from customer reviews using GenAI and streaming data technologies

How we eliminated the data ping-pong for simple, efficient data transformation

The simplest setup to process and analyze large-scale ad data in real time

Learn how they compare, their benefits, and potential use cases

A frontend dev’s secret sauce for cooking up streamlined real-time apps

GCP Private Service Connect support and new cloud provider region availability lead the way for customers in regulated industries

Get predictions on the fly using real-time data with simplified data transformations—powered by WASM

Learn how to optimize data ingestion, network, and storage when streaming data on Google Cloud Platform

How to balance performance, resources, and operational costs more efficiently on MSK — or simply switch to Redpanda BYOC

From one-click cluster creation to automatic scaling — welcome to the fastest and easiest way to start streaming data

Cloud data transfer can be expensive. Here’s how to work out your monthly cloud bill—and how Redpanda can help lower it

Here’s how we supercharged Redpanda Cloud while saving India’s largest social media company 70% on cloud spend

It just got even simpler for Python developers to build powerful streaming data apps

Your ticket to robust data pipelines and real-time search apps for seamless streaming, ETL operations, and data indexing

From team-building to tacos — here’s a peek at what it means to be a Redpanda

Audit logging and AWS PrivateLink simplify compliance for large organizations, plus new high-throughput tiers support >1 GBps ingress workloads

A reference architecture for manufacturers to push innovation, adaptability, and continuous improvement.

How we built a robust system for real-time clickstream recommendations using generative AI, Redpanda, DataSQRL, and AWS

Say hello to redaction made simple with our pre-built, no-code library

A crash course on rpk to get you on track for simple, no-fuss cluster management

Learn how to process sensor data in real time, detect repeating patterns, and trigger alarms to prevent equipment damage and defects

The latest release brings in-broker data transforms (beta), plus enhancements to enterprise security, compliance, reliability, and efficiencies that lower TCO.

Your step-by-step guide on switching to an easy, cost-effective Kafka-compatible alternative

Learn how to build a data pipeline for real-time fraud detection in a financial use case

A guide on choosing the right technologies for streaming data APIs and how to implement them

A friendly introduction to real-time analytics and how to get started

Learn how to capture, move, and analyze IoT telemetry data with Redpanda and Apache Spark

Get to know our latest champion—and how you can be our next winner

Learn when to use an event-driven, layered, or microservice architecture pattern

Learn how to build a scalable and resilient system that helps run a futuristic coffee shop

How LiveRamp is modernizing its privacy-centric data infrastructure for the post-cookie era with Redpanda Cloud

Learn about Amazon DynamoDB then migrate data from PostgreSQL using Debezium and Kafka Connect

The latest update to Redpanda Cloud brings you improved performance, simpler schema management, a new Iceberg connector, and lower costs.

Learn how to update a frontend service and connect it to fresh data coming from a new topic.

Get an overview of ETL pipelines, where to use them, and how to build an ETL pipeline for streaming data with Apache Beam and Redpanda.

Build a real-time ad analysis system using Redpanda, Flink, and Pinot

How a leading IT service company navigated the transition from Kafka to Redpanda for a low-latency, real-time alerting system

Build a real-time CDC pipeline that automatically moves your data from a Postgres database to GCS

Run a ready-made chat app with built-in sentiment analysis and even a profanity filter

Learn how to deliver powerful analytics over data in motion with Feldera and Redpanda

A helpful checklist to set you up for a smooth transition to the simpler streaming data platform

Learn what they are, when to use them, and popular tools to implement them

Learn why a leading real-time data platform for blockchains chose Redpanda to power exciting Web3 projects

When it comes to your data, the best governance is self-governance

A step-by-step tutorial to capture changed data for high-throughput, low-latency applications

Learn the key architectural components and explore a real-time bidding use case featuring Redpanda, Aerospike, and Flink

Learn the differences between these methods for real-time data processing

Build a data processing pipeline for audio files in a few simple steps

Introducing more granular data access control and compliance-readiness in Redpanda Cloud

Learn the data streaming strategies to lay a solid foundation for AI

Explore a goldmine of practical templates and out-of-the-box ideas that you can use for your own Redpanda projects

Your guide through the wild west of techniques, tools, and technologies for building real-time data pipelines

Get to know our latest champion—and how you can nab the title next time

Get an overview of the stream processing patterns for real-time data—and when to use them

Integrate Redpanda and MSF for enhanced performance, lower TCO, and fully managed cloud services

A step-by-step tutorial on building a real-time ML analysis system for fraud detection.

A deep dive into the challenges blocking the FSI from successfully adopting AI—and how to solve them

Don't choose between data sovereignty and a fully managed experience. BYOC lets you have your cake and eat it too.

Join our quest for a radically simpler architecture for data processing—powered by Wasm.

Learn why Jump Trading chose Redpanda to power their next-gen messaging platform and tackle its extreme data throughput challenges

Learn the proven architectural guidelines powering today’s successful FinTech operational systems

How Fortis Games is building a developer-friendly games data platform focused around self-service using Redpanda, Flink, ClickHouse, and Iceberg

Learn how to use Redpanda, Flink, and MongoDB to seamlessly handle customer data for financial transactions

Monitoring that even (Franz) Kafka would approve of

Take a closer look into our latest cost-saving feature—now available in Redpanda 23.2

What to consider when deploying a cluster in multiple availability zones

Redpanda Platform and Redpanda Cloud get boosts to power, scale, cost-efficiency, operational simplicity, and Kafka ecosystem support.

Get familiar with the basics of real-time data streaming, how they play together, and how they differ.

Get familiar with the basics of real-time data streaming, how they play together, and how they differ.

Build your own time series data stream to detect anomalies in a payment system using TimescaleDB, Kafka Connect, and Redpanda.

Learn how to build and run a streaming data pipeline application in Java using Redpanda and a Neo4j graph database

An example security intelligence architecture to swiftly process and action real-time data

How to build safe, scalable, high-performing apps that unlock the full potential of real-time data

A reference framework for efficiently processing and analyzing vast amounts of industrial IoT data in real time.

Learn how Redpanda helps you solve the biggest challenges of building event-driven microservice architectures.

Want to start streaming data with Redpanda? Here's a dev-friendly intro to kickstart your journey.

Still streaming data like it’s 2011? The world has changed—and maybe your streaming data platform should too.

Get the 101 on real-time streaming data, common use cases, and popular tools to start your journey on the right foot.

Streaming data in real time just got a whole lot simpler

Learn how to build a modern clickstream analytics system using Redpanda and SingleStore

Meet the dream team for all your streaming database and real-time analytics use cases

A promising experiment showing how Redpanda could power large-scale data streaming without breaking a sweat

Follower fetching in Redpanda 23.2 and Redpanda Cloud means no more choosing between cost, consistency, and performance

Harnessing data streams to deliver better application security in the cloud

Power up your data streaming performance in BigQuery with our fast, scalable, and fully-managed Redpanda Cloud

Enterprise integration patterns (EIP) are especially relevant in the cloud-native era. Learn how to implement a Dead Letter Channel EIP using Apache Camel and Redpanda!

Switching from Kafka to Repanda is easier than you think. In this post, Pathway proves it with an example video-on-demand platform.

Match your players in a blink using Redpanda—no matter how big your game gets

Here’s how you can simplify test-driven development and continuous integration with Redpanda

How to build a scalable, reliable, and secure chat application to upgrade your gaming experience

Follow an example p2p payment application to see a simplified CQRS pattern in action

Regardless of the replication mechanism you must fsync() your data to prevent global data loss in non-Byzantine protocols.

How does Apache Kafka measure up after introducing KRaft?

Learn how to set up a delivery system for webhooks to build scalable, real-time applications

Command Query Responsibility Segregation (CQRS)—simplified

Get the data privacy and sovereignty of self-hosting with the ease and scalability of fully-managed services—all in your own cloud

Struggling with low throughput, high latencies, and data loss? It might be time for an upgrade

Build a scalable architecture for personalized, in-game ads using Redpanda, Quarkus, and Streamlit

The most popular container orchestration platform meets the simplest streaming data platform! Learn the basics of Redpanda in K8s to make the most of this dreamy duo.

Walk through the thinking behind Redpanda and how it truly puts you in the driver’s seat

Capture real-time insights using Redpanda’s high speed, low-cost streaming data platform.

Check out how we're making it simpler to run Redpanda on Kubernetes, and my vision for the future of this powerful integration.

Learn how to use TriggerMesh to stream data from a Redpanda topic into an Amazon S3 bucket—the Kubernetes-native way.

IoT security involves lots of data streamed in near real time for accurate monitoring. Here’s how Redpanda helped Exein make streaming data faster, simpler, and safer.

Many companies run K8s in private environments for security and control. Redpanda’s single binary architecture makes it easy to support them. Here's what you should know.

The “secret” ingredient to building faster, safer, low-cost streaming data applications at scale

Plain English goes in. Code comes out. We tested if OpenAI can create message filters in Redpanda Console. Here's how it went.

Build a clickstream processor with a faster, simpler Kafka-compatible schema registry

Learn how Redpanda makes real-time leaderboards scalable, reliable, and affordable.

Learn the fundamentals of schema registry for Kafka and how Redpanda schema registry makes it all much simpler

Learn how Kafka Streams and Spark Streaming measure up for real-time data processing.

Learn the pros and cons of Apache Kafka Tools, UI for Apache Kafka, Conduktor, Lenses.io and Redpanda Console to choose the right web UI for you.

See how Redpanda is powering the booming gaming industry with real-time data

Get the 101 on advertised Kafka address, and how to use it in Docker and K8s.

Learn how to build a simple streaming ETL pipeline for clickstream processing using Apache Flink and Redpanda.

Learn how to deploy a resilient Redpanda cluster on EC2 instances by leveraging AWS Placement Groups with Redpanda’s rack awareness feature.

Introducing managed connectors for Snowflake/BigQuery/Postgres/MySQL, Azure Blob Storage support, Kerberos integration, and more.

Learn how to set up an end-to-end platform for reliable, scalable, high performance log unification.

Learn how to build a real-time chat application using Redpanda, React, and Flask.

Streaming data means using the latest data to make timely business decisions. Here’s how we how we built a streaming data pipeline in SQL using Upsolver SQLake and Redpanda.

Here’s how SmartLunch streamlined its meal-ordering service using Redpanda as a plug-and-play replacement for Apache Kafka®.

Meet Christina Lin and learn what it’s like to be a developer advocate at a remote-first startup.

Learn how to write a Python client that produces and consumes data from Redpanda in just five steps.

Learn how to write your first Apache Camel route using the latest JBang release to integrate an API to a Kafka endpoint.

Learn the difference between five popular orchestration and automation tools for cloud-native applications: Ansible, Terraform, Chef, Salt, and Puppet.

Build a real-time search application with Redpanda and ZincSearch, and process your business data the second it enters your system!

Want to perform real-time analysis of your data stream? Here’s how to integrate Redpanda with Rockset so you can make business decisions based on your latest data.

This article explains the differences between databases, data lakes, and data warehouses. It also expands on suitable use cases and popular examples of each.

This second post in our multi-part series covers pros/cons, infrastructure, Redpanda configuration, and resilience testing in High Availability environments.

What is change data capture (CDC), and what are the common methods for implementing CDC?

A systematic approach for selecting data technologies and vendors when starting a new data engineering project.

When to choose Redpanda over Apache Kafka.

How to integrate Redpanda with Apache Druid for real-time A/B testing.

Run your own tests to evaluate the performance for a self-hosted Redpanda cluster.

Learn how to build a CDC pipeline with Redpanda that streams operational data from PostgreSQL to DuckDB for OLAP analytics.

Learn how to ingest website clickstream data into Redpanda and process them to uncover real-time insights with Decodable.

Learn how to build a real-time data pipeline using Kubernetes, Redpanda, Postgres, and Go.

Learn about nine different stream processing engines that you can configure to ingest events from Redpanda for stream processing.

Learn how to streamline real-time inventory operations with MongoDB change data streams and Redpanda.

Integrate BigQuery and Redpanda to get real-time analytics insights for better decision-making.

How to use dead letter topics and asynchronous retries for reliable message processing with Redpanda.

New capabilities to reduce storage costs, increase data safety and improve security

Redpanda Cloud brings the fastest Kafka API in the industry to developers, complete with a privacy-preserving architecture, integrated Redpanda Console, managed connectors, VPC peering, and SOC 2 compliance.

Learn how to connect MySQL and Debezium for real-time change data capture use cases.

Learn how to build a stream processing application by integrating Faust and Redpanda.

Learn why we took a single-binary approach to our architecture and how working with a single-fault mode influences usability.

Redpanda is 6x more cost effective than Kafka for the same workload running on legacy platforms, while helping you reduce your hardware footprint.

Redpanda delivers at least 10x faster tail latencies than Apache Kafka and uses up to 3x fewer nodes to do so!

In the next installment of our CDC series, we show you how to integrate SQL Server with Debezium for CDC use cases.

Integrate Redpanda with Apache Spark and BigQuery to build an engine of record.

Improve Apache Kafka application testing by using Redpanda and Testcontainers.

Learn how we engineered our Raft group protocol to provide consistency during reconfiguration.

Meet Redpanda technical writer and documentation perfector, Anne Leeper.

Learn how to ingest your Redpanda data into OpenMetadata to centralize information and prevent data silos.

Getting up-close and personal with producer configs and how to optimize them for various use cases.

Manage your Apache Kafka Access Control Lists with a user-friendly Web UI.

Learn how to implement CDC in Postgres, leveraging its logical replication feature using Debezium with Redpanda.

Learn how to use Trino with Redpanda to run federated queries against a changing data set.

Learn how to use MongoDB with Debezium and Redpanda for real-time change data capture.

Find out how we engineered our Continuous Data Balancing solution to make running Redpanda clusters safer and easier.

Learn how to use Spark Streaming and Redpanda to create a real-time, structured stream for analytics.

Meet Evgeny Lazin of the Redpanda Core team, the first engineer to work on our Shadow Indexing architecture.

Learn the differences between RabbitMQ and Redpanda, and when it makes sense to use Redpanda as a RabbitMQ alternative.

Learn how to monitor Redpanda using Datadog's built-in Redpanda integration.

Learn what makes Redpanda Console the best UI for managing Apache Kafka environments.

Learn how to do real-time anomaly detection using Redpanda and Bytewax with Timely Dataflow.

Capabilities that further simplify day-2 operations in production environments – including Remote Read Replicas, Continuous Data Balancing, and a brand new Redpanda Console.

Learn about the newest feature of Redpanda Tiered Storage: remote, read-only replica topics.

In the first post of this multi-part series, we cover common failure scenarios, deployment patterns, and cluster characteristics to consider for high-availability deployments.

10x more speed through intentional design. C++, DMA, and autotuning, oh my!

Learn how to use these popular integration testing tools to feed test data into your streaming data stack and ensure it’s set up correctly.

Using Waterstream to turn Redpanda into a fully fledged MQTT broker, plus a live demo.

Want to containerize your streaming data applications? Here’s when to use Docker vs Kubernetes.

Correctly size your Redpanda clusters by understanding how to manage the key variables that will maximize cluster performance.

Meet Dimitris Bachtis, a Redpanda SRE and one of the original developers of Redpanda Cloud.

Learn how to integrate Delta Lake with Redpanda using Spark Streaming in a data-compliant eCommerce use case scenario.

Understand the risks and rewards of data streaming with Kubernetes.

Learn how to integrate ClickHouse with Redpanda and build scalable, performant, real-time databases.

Understand the differences between using Redpanda with Spark, Flink, or ksqlDB for processing data streams.

Learn how to use ksqlDB with Redpanda to create a real-time materialized cache.

Learn how out of memory events created challenges for our team, and how we solved them.

Learn how to integrate Elasticsearch with Redpanda using Kafka Connect and build a real-time text-searching application.

Get up and running with a real-time word count application by integrating Apache Flink® with Redpanda.

Learn how Redpanda engineers use Buildkite and GitHub to automatically trigger multiple instances of CI steps running in parallel.

Learn how to integrate Databricks with Redpanda for real-time analysis of clickstream data.

How we extended our Raft protocol and simplified its implementation.

Learn how to use CockroachDB with Redpanda for change data capture.

Learn how Redpanda delivers robustness and usability improvements, enabled by powerful underlying technologies such as C++20 and the Seastar framework.

Learn how you can combine the power of Materialize and Redpanda to create a Fitbit-style activity-tracking application.

Learn about Redpanda’s free courses and how to get started with real-time app development today.

Learn how engineers at DataCater use Redpanda to simplify real-time application development.

Learn how to use Redpanda and Kafka Connect to feed data into BigQuery in real time.

Learn how the rpk command line interface makes it easier to install Redpanda and manage your workflows.

Learn about Redpanda’s new centralized cluster configuration and how it’s stored and replicated using Raft.

Meet Noah Watkins and learn what it’s like to work on platforms engineered for multi-core systems.

Learn how to join multiple KStreams in Redpanda.

New capabilities to simplify cluster management and maintenance, rack awareness, advanced tiered storage, and more!

New capabilities to simplify cluster management and maintenance, rack awareness, advanced tiered storage, and more!

Learn how to easily test your Redpanda-based development project with the Redpanda GitHub Action.

A detailed analysis of Redpanda’s official Jepsen report, including a discussion on write-write conflicts in the Apache Kafka® protocol.

Learn how to integrate Apache Pinot with Redpanda using a practical use case scenario.

Learn how to use Kafka Connect to seamlessly ingest data from Redpanda to Snowflake in real time.

Learn how to implement Redpanda TLS, SASL, and mTLS on Kubernetes.

This is the fastest way to get up and running with Redpanda on your Mac and be API-compatible with Apache Kafka.

Learn how to stream your data to S3 and GCS using Redpanda and Kafka Connect.

Comparing the performance of Graviton-based systems vs x86 hardware for real-time event-driven applications.

Kowl unlocks interactive Apache Kafka® management and time-travel debugging for real-time data.

Meet Daria Grudzien, one of Redpanda’s Site Reliability Engineers.

How Seventh Sense evolved its event capture and storage system to a streaming data system with Redpanda.

How Timeplus and Redpanda make real-time applications simpler, faster, and transformational to digital business.

Architectural overview of Shadow Indexing architecture, including write and read path optimizations.

Learn how to deploy Redpanda on Kubernetes with kind, minikube, or public cloud.

We’re a group of seasoned engineers, doers, hackers, builders, defining the future of streaming.

Realizing a faster, more developer-friendly, and safer Kappa architecture.

Meet Tucker DeWitt, the Redpanda SRE Manager.

Build a complete streaming system with NiFi and Redpanda.

How to build a streaming application with Docker, Apache Spark, and Redpanda.

The Redpanda platform is getting three new features.

Better performance and lower latencies = happy customer

How to develop applications for Redpanda and MongoDB using Docker Compose.

Using Docker Compose to prototype an end-to-end streaming application with Flink SQL, Redpanda, MySQL, Elastic and Kibana.

Schema registry provides tools for describing your events.

Leveraging Redpanda's Compatibility with Debezium to stream changes from MySQL.

Alex and Noah talk with Alexey Fyodorov about Co-designing Raft + thread-per-core execution model for the Kafka-API.

Securing Redpanda data access with ACLs.

Special guest Nick Chiu shares his view on integrating PHP with Redpanda.

Pandaproxy allows access to Redpanda via a REST API.

Using Boost.Beast to implement a custom HTTP client.

Alex joins Eric from Contributor Podcast to talk about Redpanda and the future of streaming.

We talk with the Head of Infastructure about running streaming systems at scale and the challenges involved.

Alex joins Rob and Jason from CppCast to talk about event streaming in C++.

Ashley Jeffs shares the view of Redpanda and Benthos.

Benchmarking Redpanda and Apache Kafka.

How to turn data streams into data products.

Funding to fuel development of Redpanda and its cloud services.

Hello world! with Tensorflow and Redpanda.

A new storage engine built from the ground up for Kafka-API compatibility with Kernel bypass technologies for 10x lower tail latencies.

A not-so-technical start-up guide for Redpanda with Node.js.

Big news! Today we are announcing that Redpanda is now free and source available!

How Redpanda makes use of code generation to improve system reliability.

Use Homebrew to install Redpanda.

Big news! Today we are announcing that Redpanda is now free and source available!

See our fireside chat with LinkedIn's Kafka and Samza teams as we dive deep into the technical architecture of Redpanda.

Developing with Redpanda just got easier.

Effects of fault injections on an availability of Redpanda and Apache Kafka.

Use rpk to bootstrap the config, start nodes and manage topics.

The Data Engineering Podcast interview - Speed Up And Simplify Your Streaming Data Workloads With Redpanda, Episode 152

Superscalar CPUs with wide GB/s memory, NVMe access times in the order of 10-100’s of microseconds, necessitates a new buffer management for low latency storage systems.

Going beyond Jepsen and our journey to a scalable, linearizable Raft implementation.

Step-by-step guide to configuring TLS on Redpanda brokers and rpk.

Autotuning, or increasing the vertical scalability via performance engineering, is focused on the time, energy and ultimately, the cost to run a system.

Hardware is fast, but most do not architect software to take advantage of these hardware advances. At worse, running software on significantly better hardware crashes your app because of latent bugs.