feat: add redpanda
This commit is contained in:
Sun-ZhenXing
@@ -89,6 +89,7 @@ Compose Anything helps users quickly deploy various services by providing a set
|
||||
| [Qdrant](./src/qdrant) | 1.15.4 |
|
||||
| [RabbitMQ](./src/rabbitmq) | 4.1.4 |
|
||||
| [Ray](./src/ray) | 2.42.1 |
|
||||
| [Redpanda](./src/redpanda) | v24.3.1 |
|
||||
| [Redis Cluster](./src/redis-cluster) | 8.2.1 |
|
||||
| [Redis](./src/redis) | 8.2.1 |
|
||||
| [Restate Cluster](./src/restate-cluster) | 1.5.3 |
|
||||
|
||||
@@ -89,6 +89,7 @@ Compose Anything 通过提供一组高质量的 Docker Compose 配置文件，
|
||||
| [Qdrant](./src/qdrant) | 1.15.4 |
|
||||
| [RabbitMQ](./src/rabbitmq) | 4.1.4 |
|
||||
| [Ray](./src/ray) | 2.42.1 |
|
||||
| [Redpanda](./src/redpanda) | v24.3.1 |
|
||||
| [Redis Cluster](./src/redis-cluster) | 8.2.1 |
|
||||
| [Redis](./src/redis) | 8.2.1 |
|
||||
| [Restate Cluster](./src/restate-cluster) | 1.5.3 |
|
||||
|
||||
@@ -9,6 +9,9 @@ Apache Pulsar is a cloud-native, distributed messaging and streaming platform. I
|
||||
### Default (Standalone Mode)
|
||||
|
||||
- `pulsar`: Single-node Pulsar instance for development and testing.
|
||||
- Runs with `--no-functions-worker` flag for simplicity and reduced resource usage
|
||||
- Uses RocksDB as metadata store by default (since Pulsar 2.11+)
|
||||
- Includes embedded ZooKeeper and BookKeeper in the same JVM process
|
||||
|
||||
### Cluster Mode (profile: `cluster`)
|
||||
|
||||
@@ -48,7 +51,25 @@ Please modify the `.env` file as needed for your use case.
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. Access Pulsar:
|
||||
2. Wait for Pulsar to be ready (check logs):
|
||||
|
||||
```bash
|
||||
docker compose logs -f pulsar
|
||||
```
|
||||
|
||||
You should see a message like:
|
||||
|
||||
```log
|
||||
INFO org.apache.pulsar.broker.PulsarService - messaging service is ready
|
||||
```
|
||||
|
||||
3. Verify the cluster is healthy:
|
||||
|
||||
```bash
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. Access Pulsar:
|
||||
- Broker: `pulsar://localhost:6650`
|
||||
- Admin API: `http://localhost:8080`
|
||||
|
||||
@@ -187,8 +208,39 @@ client.close()
|
||||
|
||||
- Standalone mode uses RocksDB as metadata store by default (recommended for single-node).
|
||||
- Set `PULSAR_STANDALONE_USE_ZOOKEEPER=1` to use ZooKeeper as metadata store.
|
||||
- Cluster mode is configured for single-node BookKeeper (ensemble size = 1).
|
||||
- For production, adjust quorum settings and add more bookies.
|
||||
- Functions worker is disabled by default to reduce resource usage and startup time.
|
||||
- For production, use cluster mode with dedicated ZooKeeper and BookKeeper instances.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Standalone Mode Issues
|
||||
|
||||
If you encounter connection errors like "NoRouteToHostException" or "Bookie handle is not available":
|
||||
|
||||
1. **Clear existing data** (if upgrading or switching metadata store):
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. **Check container logs**:
|
||||
|
||||
```bash
|
||||
docker compose logs pulsar
|
||||
```
|
||||
|
||||
3. **Verify healthcheck**:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. **Ensure sufficient resources**: Standalone mode requires at least:
|
||||
- 2GB RAM
|
||||
- 2 CPU cores
|
||||
- 5GB disk space
|
||||
|
||||
## Ports
|
||||
|
||||
|
||||
@@ -9,6 +9,9 @@ Apache Pulsar 是一个云原生的分布式消息和流处理平台。它结合
|
||||
### 默认（单机模式）
|
||||
|
||||
- `pulsar`：单节点 Pulsar 实例，适用于开发和测试。
|
||||
- 使用 `--no-functions-worker` 标志运行，简化部署并减少资源使用
|
||||
- 默认使用 RocksDB 作为元数据存储（从 Pulsar 2.11+ 开始）
|
||||
- 在同一个 JVM 进程中包含内嵌的 ZooKeeper 和 BookKeeper
|
||||
|
||||
### 集群模式（profile: `cluster`）
|
||||
|
||||
@@ -48,7 +51,25 @@ Apache Pulsar 是一个云原生的分布式消息和流处理平台。它结合
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. 访问 Pulsar：
|
||||
2. 等待 Pulsar 就绪（查看日志）：
|
||||
|
||||
```bash
|
||||
docker compose logs -f pulsar
|
||||
```
|
||||
|
||||
您应该看到类似以下的消息：
|
||||
|
||||
```log
|
||||
INFO org.apache.pulsar.broker.PulsarService - messaging service is ready
|
||||
```
|
||||
|
||||
3. 验证集群健康状态：
|
||||
|
||||
```bash
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. 访问 Pulsar：
|
||||
- Broker：`pulsar://localhost:6650`
|
||||
- Admin API：`http://localhost:8080`
|
||||
|
||||
@@ -187,8 +208,39 @@ client.close()
|
||||
|
||||
- 单机模式默认使用 RocksDB 作为元数据存储（推荐用于单节点）。
|
||||
- 设置 `PULSAR_STANDALONE_USE_ZOOKEEPER=1` 可使用 ZooKeeper 作为元数据存储。
|
||||
- 集群模式配置为单节点 BookKeeper（ensemble size = 1）。
|
||||
- 生产环境请调整 quorum 设置并添加更多 bookie。
|
||||
- 默认禁用 Functions Worker 以减少资源使用和启动时间。
|
||||
- 生产环境请使用集群模式，配置专用的 ZooKeeper 和 BookKeeper 实例。
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 单机模式问题
|
||||
|
||||
如果遇到连接错误，如"NoRouteToHostException"或"Bookie handle is not available"：
|
||||
|
||||
1. **清除现有数据**（如果升级或切换元数据存储）：
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. **检查容器日志**：
|
||||
|
||||
```bash
|
||||
docker compose logs pulsar
|
||||
```
|
||||
|
||||
3. **验证健康检查**：
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. **确保资源充足**：单机模式至少需要：
|
||||
- 2GB 内存
|
||||
- 2 个 CPU 核心
|
||||
- 5GB 磁盘空间
|
||||
|
||||
## 端口
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ services:
|
||||
pulsar:
|
||||
<<: *defaults
|
||||
image: ${GLOBAL_REGISTRY:-}apachepulsar/pulsar:${PULSAR_VERSION:-4.0.7}
|
||||
hostname: pulsar-standalone
|
||||
ports:
|
||||
- "${PULSAR_BROKER_PORT_OVERRIDE:-6650}:6650"
|
||||
- "${PULSAR_HTTP_PORT_OVERRIDE:-8080}:8080"
|
||||
@@ -24,7 +25,7 @@ services:
|
||||
- TZ=${TZ:-UTC}
|
||||
- PULSAR_MEM=${PULSAR_MEM:--Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m}
|
||||
- PULSAR_STANDALONE_USE_ZOOKEEPER=${PULSAR_STANDALONE_USE_ZOOKEEPER:-0}
|
||||
command: bin/pulsar standalone
|
||||
command: sh -c "bin/apply-config-from-env.py conf/standalone.conf && bin/pulsar standalone --no-functions-worker"
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
@@ -34,11 +35,11 @@ services:
|
||||
cpus: ${PULSAR_CPU_RESERVATION:-0.50}
|
||||
memory: ${PULSAR_MEMORY_RESERVATION:-512M}
|
||||
healthcheck:
|
||||
test: ["CMD", "bin/pulsar-admin", "brokers", "healthcheck"]
|
||||
test: ["CMD-SHELL", "bin/pulsar-admin brokers healthcheck || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 60s
|
||||
start_period: 90s
|
||||
|
||||
# ==================== Cluster Mode (profile: cluster) ====================
|
||||
|
||||
|
||||
47
src/redpanda/.env.example
Normal file
47
src/redpanda/.env.example
Normal file
@@ -0,0 +1,47 @@
|
||||
# Redpanda version
|
||||
REDPANDA_VERSION=v24.3.1
|
||||
|
||||
# Redpanda Console version
|
||||
REDPANDA_CONSOLE_VERSION=v2.8.2
|
||||
|
||||
# Global registry prefix (optional, for using mirror registries)
|
||||
# GLOBAL_REGISTRY=your-registry.com/
|
||||
|
||||
# Timezone (default: UTC)
|
||||
TZ=UTC
|
||||
|
||||
# Redpanda Kafka API port (external)
|
||||
REDPANDA_KAFKA_PORT_OVERRIDE=19092
|
||||
|
||||
# Redpanda Schema Registry port (external)
|
||||
REDPANDA_SCHEMA_PORT_OVERRIDE=18081
|
||||
|
||||
# Redpanda HTTP Proxy port (external)
|
||||
REDPANDA_PROXY_PORT_OVERRIDE=18082
|
||||
|
||||
# Redpanda Admin API port (external)
|
||||
REDPANDA_ADMIN_PORT_OVERRIDE=19644
|
||||
|
||||
# Redpanda Console UI port
|
||||
REDPANDA_CONSOLE_PORT_OVERRIDE=8080
|
||||
|
||||
# Redpanda CPU cores for processing (default: 1)
|
||||
REDPANDA_SMP=1
|
||||
|
||||
# Redpanda memory allocation (default: 1G)
|
||||
REDPANDA_MEMORY=1G
|
||||
|
||||
# Redpanda log level (trace, debug, info, warn, error)
|
||||
REDPANDA_LOG_LEVEL=info
|
||||
|
||||
# Redpanda resource limits
|
||||
REDPANDA_CPU_LIMIT=2.0
|
||||
REDPANDA_MEMORY_LIMIT=2G
|
||||
REDPANDA_CPU_RESERVATION=1.0
|
||||
REDPANDA_MEMORY_RESERVATION=1G
|
||||
|
||||
# Redpanda Console resource limits
|
||||
REDPANDA_CONSOLE_CPU_LIMIT=0.5
|
||||
REDPANDA_CONSOLE_MEMORY_LIMIT=512M
|
||||
REDPANDA_CONSOLE_CPU_RESERVATION=0.25
|
||||
REDPANDA_CONSOLE_MEMORY_RESERVATION=256M
|
||||
183
src/redpanda/README.md
Normal file
183
src/redpanda/README.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Redpanda
|
||||
|
||||
[English](README.md) | [中文](README.zh.md)
|
||||
|
||||
Redpanda is a Kafka-compatible streaming data platform built for performance and developer experience. It is designed to be simple to deploy and manage, with no dependencies on JVM or Zookeeper.
|
||||
|
||||
## Features
|
||||
|
||||
- **Kafka API Compatible**: Works with existing Kafka clients and tools
|
||||
- **High Performance**: Built in C++ for maximum throughput and low latency
|
||||
- **No Zookeeper**: Simplified architecture with built-in consensus
|
||||
- **Schema Registry**: Built-in schema registry support
|
||||
- **Developer Friendly**: Easy to deploy and manage with minimal configuration
|
||||
- **Redpanda Console**: Modern web UI for monitoring and management
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Copy the environment file and customize if needed:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
2. Start the services:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
3. Access Redpanda Console at <http://localhost:8080>
|
||||
|
||||
4. Verify the cluster is healthy:
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk cluster health
|
||||
```
|
||||
|
||||
## Service Endpoints
|
||||
|
||||
- **Kafka API** (external): `localhost:19092`
|
||||
- **Schema Registry**: `localhost:18081`
|
||||
- **HTTP Proxy**: `localhost:18082`
|
||||
- **Admin API**: `localhost:19644`
|
||||
- **Redpanda Console**: `http://localhost:8080`
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### Create a topic
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic create my-topic
|
||||
```
|
||||
|
||||
### List topics
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic list
|
||||
```
|
||||
|
||||
### Produce messages
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic produce my-topic
|
||||
```
|
||||
|
||||
Type messages and press Ctrl+C to exit.
|
||||
|
||||
### Consume messages
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic consume my-topic
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
| -------------------------------- | --------- | ------------------------------------------- |
|
||||
| `REDPANDA_VERSION` | `v24.3.1` | Redpanda version |
|
||||
| `REDPANDA_CONSOLE_VERSION` | `v2.8.2` | Redpanda Console version |
|
||||
| `TZ` | `UTC` | Timezone |
|
||||
| `REDPANDA_KAFKA_PORT_OVERRIDE` | `19092` | Kafka API external port |
|
||||
| `REDPANDA_SCHEMA_PORT_OVERRIDE` | `18081` | Schema Registry port |
|
||||
| `REDPANDA_PROXY_PORT_OVERRIDE` | `18082` | HTTP Proxy port |
|
||||
| `REDPANDA_ADMIN_PORT_OVERRIDE` | `19644` | Admin API port |
|
||||
| `REDPANDA_CONSOLE_PORT_OVERRIDE` | `8080` | Console UI port |
|
||||
| `REDPANDA_SMP` | `1` | Number of CPU cores for processing |
|
||||
| `REDPANDA_MEMORY` | `1G` | Memory allocation |
|
||||
| `REDPANDA_LOG_LEVEL` | `info` | Log level (trace, debug, info, warn, error) |
|
||||
|
||||
## Resource Configuration
|
||||
|
||||
Default resource limits:
|
||||
|
||||
- **Redpanda**:
|
||||
- CPU: 1.0-2.0 cores
|
||||
- Memory: 1G-2G
|
||||
|
||||
- **Console**:
|
||||
- CPU: 0.25-0.5 cores
|
||||
- Memory: 256M-512M
|
||||
|
||||
Adjust these values in `.env` file based on your workload requirements.
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
1. **CPU Cores**: Increase `REDPANDA_SMP` to match available CPU cores
|
||||
2. **Memory**: Allocate more memory with `REDPANDA_MEMORY` for better performance
|
||||
3. **Storage**: Use high-performance storage (SSD/NVMe) for production
|
||||
|
||||
### Cluster Mode
|
||||
|
||||
This configuration runs Redpanda in single-node development mode. For production clusters:
|
||||
|
||||
1. Remove `--mode dev-container` flag
|
||||
2. Configure multiple Redpanda instances
|
||||
3. Set appropriate replication factors
|
||||
4. Configure proper security (SASL, TLS)
|
||||
|
||||
### Security
|
||||
|
||||
For production use, consider:
|
||||
|
||||
1. Enable SASL authentication
|
||||
2. Configure TLS for encryption
|
||||
3. Set up ACLs for authorization
|
||||
4. Restrict network access
|
||||
|
||||
## Monitoring
|
||||
|
||||
Redpanda Console provides a comprehensive monitoring interface including:
|
||||
|
||||
- Cluster overview and health
|
||||
- Topic management and inspection
|
||||
- Consumer group monitoring
|
||||
- Schema registry management
|
||||
- Message browser
|
||||
|
||||
Additional metrics are available through Redpanda's Admin API at port 19644.
|
||||
|
||||
## Data Persistence
|
||||
|
||||
Data is stored in a Docker named volume `redpanda_data`. To backup or migrate:
|
||||
|
||||
```bash
|
||||
# Backup
|
||||
docker run --rm -v redpanda_redpanda_data:/data -v $(pwd):/backup alpine tar czf /backup/redpanda-backup.tar.gz -C /data .
|
||||
|
||||
# Restore
|
||||
docker run --rm -v redpanda_redpanda_data:/data -v $(pwd):/backup alpine tar xzf /backup/redpanda-backup.tar.gz -C /data
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container fails to start
|
||||
|
||||
1. Check available memory: Redpanda requires at least 1GB
|
||||
2. Verify port availability
|
||||
3. Check logs: `docker compose logs redpanda`
|
||||
|
||||
### Cannot connect to Kafka
|
||||
|
||||
1. Ensure ports are properly mapped
|
||||
2. Verify firewall settings
|
||||
3. Check advertised listeners configuration
|
||||
|
||||
### Performance issues
|
||||
|
||||
1. Increase `REDPANDA_SMP` to use more CPU cores
|
||||
2. Allocate more memory with `REDPANDA_MEMORY`
|
||||
3. Monitor resource usage in Console
|
||||
|
||||
## References
|
||||
|
||||
- [Redpanda Documentation](https://docs.redpanda.com/)
|
||||
- [Redpanda GitHub](https://github.com/redpanda-data/redpanda)
|
||||
- [Redpanda Console](https://docs.redpanda.com/current/manage/console/)
|
||||
- [rpk CLI Reference](https://docs.redpanda.com/current/reference/rpk/)
|
||||
|
||||
## License
|
||||
|
||||
Redpanda is licensed under the [Redpanda Business Source License](https://github.com/redpanda-data/redpanda/blob/dev/licenses/bsl.md).
|
||||
183
src/redpanda/README.zh.md
Normal file
183
src/redpanda/README.zh.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Redpanda
|
||||
|
||||
[English](README.md) | [中文](README.zh.md)
|
||||
|
||||
Redpanda 是一个与 Kafka 兼容的流数据平台，专为性能和开发者体验而构建。它设计简单易部署和管理，不依赖 JVM 或 Zookeeper。
|
||||
|
||||
## 特性
|
||||
|
||||
- **Kafka API 兼容**：可与现有的 Kafka 客户端和工具配合使用
|
||||
- **高性能**：使用 C++ 构建，实现最大吞吐量和低延迟
|
||||
- **无需 Zookeeper**：内置共识机制的简化架构
|
||||
- **Schema Registry**：内置 Schema Registry 支持
|
||||
- **开发者友好**：易于部署和管理，配置最少
|
||||
- **Redpanda Console**：用于监控和管理的现代化 Web UI
|
||||
|
||||
## 快速开始
|
||||
|
||||
1. 复制环境文件并根据需要自定义：
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
2. 启动服务：
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
3. 访问 Redpanda Console：<http://localhost:8080>
|
||||
|
||||
4. 验证集群健康状态：
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk cluster health
|
||||
```
|
||||
|
||||
## 服务端点
|
||||
|
||||
- **Kafka API**（外部）：`localhost:19092`
|
||||
- **Schema Registry**：`localhost:18081`
|
||||
- **HTTP Proxy**：`localhost:18082`
|
||||
- **Admin API**：`localhost:19644`
|
||||
- **Redpanda Console**：`http://localhost:8080`
|
||||
|
||||
## 基本使用
|
||||
|
||||
### 创建主题
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic create my-topic
|
||||
```
|
||||
|
||||
### 列出主题
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic list
|
||||
```
|
||||
|
||||
### 生产消息
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic produce my-topic
|
||||
```
|
||||
|
||||
输入消息后按 Ctrl+C 退出。
|
||||
|
||||
### 消费消息
|
||||
|
||||
```bash
|
||||
docker compose exec redpanda rpk topic consume my-topic
|
||||
```
|
||||
|
||||
## 环境变量
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| -------------------------------- | --------- | ------------------------------------------- |
|
||||
| `REDPANDA_VERSION` | `v24.3.1` | Redpanda 版本 |
|
||||
| `REDPANDA_CONSOLE_VERSION` | `v2.8.2` | Redpanda Console 版本 |
|
||||
| `TZ` | `UTC` | 时区 |
|
||||
| `REDPANDA_KAFKA_PORT_OVERRIDE` | `19092` | Kafka API 外部端口 |
|
||||
| `REDPANDA_SCHEMA_PORT_OVERRIDE` | `18081` | Schema Registry 端口 |
|
||||
| `REDPANDA_PROXY_PORT_OVERRIDE` | `18082` | HTTP Proxy 端口 |
|
||||
| `REDPANDA_ADMIN_PORT_OVERRIDE` | `19644` | Admin API 端口 |
|
||||
| `REDPANDA_CONSOLE_PORT_OVERRIDE` | `8080` | Console UI 端口 |
|
||||
| `REDPANDA_SMP` | `1` | 用于处理的 CPU 核心数 |
|
||||
| `REDPANDA_MEMORY` | `1G` | 内存分配 |
|
||||
| `REDPANDA_LOG_LEVEL` | `info` | 日志级别（trace、debug、info、warn、error） |
|
||||
|
||||
## 资源配置
|
||||
|
||||
默认资源限制：
|
||||
|
||||
- **Redpanda**：
|
||||
- CPU：1.0-2.0 核
|
||||
- 内存：1G-2G
|
||||
|
||||
- **Console**：
|
||||
- CPU：0.25-0.5 核
|
||||
- 内存：256M-512M
|
||||
|
||||
根据您的工作负载需求在 `.env` 文件中调整这些值。
|
||||
|
||||
## 生产环境注意事项
|
||||
|
||||
### 性能调优
|
||||
|
||||
1. **CPU 核心**：增加 `REDPANDA_SMP` 以匹配可用的 CPU 核心
|
||||
2. **内存**：使用 `REDPANDA_MEMORY` 分配更多内存以获得更好的性能
|
||||
3. **存储**：在生产环境中使用高性能存储（SSD/NVMe）
|
||||
|
||||
### 集群模式
|
||||
|
||||
此配置在单节点开发模式下运行 Redpanda。对于生产集群：
|
||||
|
||||
1. 移除 `--mode dev-container` 标志
|
||||
2. 配置多个 Redpanda 实例
|
||||
3. 设置适当的副本因子
|
||||
4. 配置适当的安全性（SASL、TLS）
|
||||
|
||||
### 安全性
|
||||
|
||||
在生产环境中使用时，请考虑：
|
||||
|
||||
1. 启用 SASL 认证
|
||||
2. 配置 TLS 加密
|
||||
3. 设置 ACL 进行授权
|
||||
4. 限制网络访问
|
||||
|
||||
## 监控
|
||||
|
||||
Redpanda Console 提供全面的监控界面，包括：
|
||||
|
||||
- 集群概览和健康状态
|
||||
- 主题管理和检查
|
||||
- 消费者组监控
|
||||
- Schema Registry 管理
|
||||
- 消息浏览器
|
||||
|
||||
通过端口 19644 的 Redpanda Admin API 可以获取其他指标。
|
||||
|
||||
## 数据持久化
|
||||
|
||||
数据存储在 Docker 命名卷 `redpanda_data` 中。备份或迁移：
|
||||
|
||||
```bash
|
||||
# 备份
|
||||
docker run --rm -v redpanda_redpanda_data:/data -v $(pwd):/backup alpine tar czf /backup/redpanda-backup.tar.gz -C /data .
|
||||
|
||||
# 恢复
|
||||
docker run --rm -v redpanda_redpanda_data:/data -v $(pwd):/backup alpine tar xzf /backup/redpanda-backup.tar.gz -C /data
|
||||
```
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 容器启动失败
|
||||
|
||||
1. 检查可用内存：Redpanda 至少需要 1GB
|
||||
2. 验证端口可用性
|
||||
3. 查看日志：`docker compose logs redpanda`
|
||||
|
||||
### 无法连接到 Kafka
|
||||
|
||||
1. 确保端口正确映射
|
||||
2. 验证防火墙设置
|
||||
3. 检查 advertised listeners 配置
|
||||
|
||||
### 性能问题
|
||||
|
||||
1. 增加 `REDPANDA_SMP` 以使用更多 CPU 核心
|
||||
2. 使用 `REDPANDA_MEMORY` 分配更多内存
|
||||
3. 在 Console 中监控资源使用情况
|
||||
|
||||
## 参考资料
|
||||
|
||||
- [Redpanda 文档](https://docs.redpanda.com/)
|
||||
- [Redpanda GitHub](https://github.com/redpanda-data/redpanda)
|
||||
- [Redpanda Console](https://docs.redpanda.com/current/manage/console/)
|
||||
- [rpk CLI 参考](https://docs.redpanda.com/current/reference/rpk/)
|
||||
|
||||
## 许可证
|
||||
|
||||
Redpanda 使用 [Redpanda Business Source License](https://github.com/redpanda-data/redpanda/blob/dev/licenses/bsl.md) 许可。
|
||||
86
src/redpanda/docker-compose.yaml
Normal file
86
src/redpanda/docker-compose.yaml
Normal file
@@ -0,0 +1,86 @@
|
||||
# Redpanda - A Kafka-compatible streaming data platform built for performance
|
||||
# This configuration provides a single-node Redpanda setup with console for monitoring
|
||||
|
||||
x-defaults: &defaults
|
||||
restart: unless-stopped
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: 100m
|
||||
max-file: "3"
|
||||
|
||||
services:
|
||||
redpanda:
|
||||
<<: *defaults
|
||||
image: ${GLOBAL_REGISTRY:-}docker.redpanda.com/redpandadata/redpanda:${REDPANDA_VERSION:-v24.3.1}
|
||||
command:
|
||||
- redpanda
|
||||
- start
|
||||
- --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:${REDPANDA_KAFKA_PORT_OVERRIDE:-19092}
|
||||
- --advertise-kafka-addr internal://redpanda:9092,external://localhost:${REDPANDA_KAFKA_PORT_OVERRIDE:-19092}
|
||||
- --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:${REDPANDA_PROXY_PORT_OVERRIDE:-18082}
|
||||
- --advertise-pandaproxy-addr internal://redpanda:8082,external://localhost:${REDPANDA_PROXY_PORT_OVERRIDE:-18082}
|
||||
- --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:${REDPANDA_SCHEMA_PORT_OVERRIDE:-18081}
|
||||
- --rpc-addr redpanda:33145
|
||||
- --advertise-rpc-addr redpanda:33145
|
||||
- --mode dev-container
|
||||
- --smp ${REDPANDA_SMP:-1}
|
||||
- --default-log-level=${REDPANDA_LOG_LEVEL:-info}
|
||||
- --reserve-memory 0M
|
||||
- --memory ${REDPANDA_MEMORY:-1G}
|
||||
ports:
|
||||
- "${REDPANDA_SCHEMA_PORT_OVERRIDE:-18081}:8081"
|
||||
- "${REDPANDA_PROXY_PORT_OVERRIDE:-18082}:8082"
|
||||
- "${REDPANDA_KAFKA_PORT_OVERRIDE:-19092}:19092"
|
||||
- "${REDPANDA_ADMIN_PORT_OVERRIDE:-19644}:9644"
|
||||
volumes:
|
||||
- redpanda_data:/var/lib/redpanda/data
|
||||
environment:
|
||||
- TZ=${TZ:-UTC}
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "rpk cluster health | grep -E 'Healthy:.+true' || exit 1"]
|
||||
interval: 15s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: ${REDPANDA_CPU_LIMIT:-2.0}
|
||||
memory: ${REDPANDA_MEMORY_LIMIT:-2G}
|
||||
reservations:
|
||||
cpus: ${REDPANDA_CPU_RESERVATION:-1.0}
|
||||
memory: ${REDPANDA_MEMORY_RESERVATION:-1G}
|
||||
|
||||
console:
|
||||
<<: *defaults
|
||||
image: ${GLOBAL_REGISTRY:-}docker.redpanda.com/redpandadata/console:${REDPANDA_CONSOLE_VERSION:-v2.8.2}
|
||||
ports:
|
||||
- "${REDPANDA_CONSOLE_PORT_OVERRIDE:-8080}:8080"
|
||||
environment:
|
||||
- KAFKA_BROKERS=redpanda:9092
|
||||
- KAFKA_SCHEMAREGISTRY_ENABLED=true
|
||||
- KAFKA_SCHEMAREGISTRY_URLS=http://redpanda:8081
|
||||
- REDPANDA_ADMINAPI_ENABLED=true
|
||||
- REDPANDA_ADMINAPI_URLS=http://redpanda:9644
|
||||
- TZ=${TZ:-UTC}
|
||||
depends_on:
|
||||
redpanda:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://localhost:8080/admin/health"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: ${REDPANDA_CONSOLE_CPU_LIMIT:-0.5}
|
||||
memory: ${REDPANDA_CONSOLE_MEMORY_LIMIT:-512M}
|
||||
reservations:
|
||||
cpus: ${REDPANDA_CONSOLE_CPU_RESERVATION:-0.25}
|
||||
memory: ${REDPANDA_CONSOLE_MEMORY_RESERVATION:-256M}
|
||||
|
||||
volumes:
|
||||
redpanda_data:
|
||||
Reference in New Issue
Block a user