feat: add redpanda
This commit is contained in:
@@ -9,6 +9,9 @@ Apache Pulsar is a cloud-native, distributed messaging and streaming platform. I
|
||||
### Default (Standalone Mode)
|
||||
|
||||
- `pulsar`: Single-node Pulsar instance for development and testing.
|
||||
- Runs with `--no-functions-worker` flag for simplicity and reduced resource usage
|
||||
- Uses RocksDB as metadata store by default (since Pulsar 2.11+)
|
||||
- Includes embedded ZooKeeper and BookKeeper in the same JVM process
|
||||
|
||||
### Cluster Mode (profile: `cluster`)
|
||||
|
||||
@@ -48,7 +51,25 @@ Please modify the `.env` file as needed for your use case.
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. Access Pulsar:
|
||||
2. Wait for Pulsar to be ready (check logs):
|
||||
|
||||
```bash
|
||||
docker compose logs -f pulsar
|
||||
```
|
||||
|
||||
You should see a message like:
|
||||
|
||||
```log
|
||||
INFO org.apache.pulsar.broker.PulsarService - messaging service is ready
|
||||
```
|
||||
|
||||
3. Verify the cluster is healthy:
|
||||
|
||||
```bash
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. Access Pulsar:
|
||||
- Broker: `pulsar://localhost:6650`
|
||||
- Admin API: `http://localhost:8080`
|
||||
|
||||
@@ -187,8 +208,39 @@ client.close()
|
||||
|
||||
- Standalone mode uses RocksDB as metadata store by default (recommended for single-node).
|
||||
- Set `PULSAR_STANDALONE_USE_ZOOKEEPER=1` to use ZooKeeper as metadata store.
|
||||
- Cluster mode is configured for single-node BookKeeper (ensemble size = 1).
|
||||
- For production, adjust quorum settings and add more bookies.
|
||||
- Functions worker is disabled by default to reduce resource usage and startup time.
|
||||
- For production, use cluster mode with dedicated ZooKeeper and BookKeeper instances.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Standalone Mode Issues
|
||||
|
||||
If you encounter connection errors like "NoRouteToHostException" or "Bookie handle is not available":
|
||||
|
||||
1. **Clear existing data** (if upgrading or switching metadata store):
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. **Check container logs**:
|
||||
|
||||
```bash
|
||||
docker compose logs pulsar
|
||||
```
|
||||
|
||||
3. **Verify healthcheck**:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. **Ensure sufficient resources**: Standalone mode requires at least:
|
||||
- 2GB RAM
|
||||
- 2 CPU cores
|
||||
- 5GB disk space
|
||||
|
||||
## Ports
|
||||
|
||||
|
||||
@@ -9,6 +9,9 @@ Apache Pulsar 是一个云原生的分布式消息和流处理平台。它结合
|
||||
### 默认(单机模式)
|
||||
|
||||
- `pulsar`:单节点 Pulsar 实例,适用于开发和测试。
|
||||
- 使用 `--no-functions-worker` 标志运行,简化部署并减少资源使用
|
||||
- 默认使用 RocksDB 作为元数据存储(从 Pulsar 2.11+ 开始)
|
||||
- 在同一个 JVM 进程中包含内嵌的 ZooKeeper 和 BookKeeper
|
||||
|
||||
### 集群模式(profile: `cluster`)
|
||||
|
||||
@@ -48,7 +51,25 @@ Apache Pulsar 是一个云原生的分布式消息和流处理平台。它结合
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. 访问 Pulsar:
|
||||
2. 等待 Pulsar 就绪(查看日志):
|
||||
|
||||
```bash
|
||||
docker compose logs -f pulsar
|
||||
```
|
||||
|
||||
您应该看到类似以下的消息:
|
||||
|
||||
```log
|
||||
INFO org.apache.pulsar.broker.PulsarService - messaging service is ready
|
||||
```
|
||||
|
||||
3. 验证集群健康状态:
|
||||
|
||||
```bash
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. 访问 Pulsar:
|
||||
- Broker:`pulsar://localhost:6650`
|
||||
- Admin API:`http://localhost:8080`
|
||||
|
||||
@@ -187,8 +208,39 @@ client.close()
|
||||
|
||||
- 单机模式默认使用 RocksDB 作为元数据存储(推荐用于单节点)。
|
||||
- 设置 `PULSAR_STANDALONE_USE_ZOOKEEPER=1` 可使用 ZooKeeper 作为元数据存储。
|
||||
- 集群模式配置为单节点 BookKeeper(ensemble size = 1)。
|
||||
- 生产环境请调整 quorum 设置并添加更多 bookie。
|
||||
- 默认禁用 Functions Worker 以减少资源使用和启动时间。
|
||||
- 生产环境请使用集群模式,配置专用的 ZooKeeper 和 BookKeeper 实例。
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 单机模式问题
|
||||
|
||||
如果遇到连接错误,如"NoRouteToHostException"或"Bookie handle is not available":
|
||||
|
||||
1. **清除现有数据**(如果升级或切换元数据存储):
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
2. **检查容器日志**:
|
||||
|
||||
```bash
|
||||
docker compose logs pulsar
|
||||
```
|
||||
|
||||
3. **验证健康检查**:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker exec pulsar bin/pulsar-admin brokers healthcheck
|
||||
```
|
||||
|
||||
4. **确保资源充足**:单机模式至少需要:
|
||||
- 2GB 内存
|
||||
- 2 个 CPU 核心
|
||||
- 5GB 磁盘空间
|
||||
|
||||
## 端口
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ services:
|
||||
pulsar:
|
||||
<<: *defaults
|
||||
image: ${GLOBAL_REGISTRY:-}apachepulsar/pulsar:${PULSAR_VERSION:-4.0.7}
|
||||
hostname: pulsar-standalone
|
||||
ports:
|
||||
- "${PULSAR_BROKER_PORT_OVERRIDE:-6650}:6650"
|
||||
- "${PULSAR_HTTP_PORT_OVERRIDE:-8080}:8080"
|
||||
@@ -24,7 +25,7 @@ services:
|
||||
- TZ=${TZ:-UTC}
|
||||
- PULSAR_MEM=${PULSAR_MEM:--Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m}
|
||||
- PULSAR_STANDALONE_USE_ZOOKEEPER=${PULSAR_STANDALONE_USE_ZOOKEEPER:-0}
|
||||
command: bin/pulsar standalone
|
||||
command: sh -c "bin/apply-config-from-env.py conf/standalone.conf && bin/pulsar standalone --no-functions-worker"
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
@@ -34,11 +35,11 @@ services:
|
||||
cpus: ${PULSAR_CPU_RESERVATION:-0.50}
|
||||
memory: ${PULSAR_MEMORY_RESERVATION:-512M}
|
||||
healthcheck:
|
||||
test: ["CMD", "bin/pulsar-admin", "brokers", "healthcheck"]
|
||||
test: ["CMD-SHELL", "bin/pulsar-admin brokers healthcheck || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 60s
|
||||
start_period: 90s
|
||||
|
||||
# ==================== Cluster Mode (profile: cluster) ====================
|
||||
|
||||
|
||||
Reference in New Issue
Block a user