feat: add FalkorDB, LMDeploy, and Pogocache with configuration files and documentation
This commit is contained in:
27
src/lmdeploy/.env.example
Normal file
27
src/lmdeploy/.env.example
Normal file
@@ -0,0 +1,27 @@
|
||||
# LMDeploy Version
|
||||
# Find more tags at: https://hub.docker.com/r/openmmlab/lmdeploy/tags
|
||||
LMDEPLOY_VERSION=v0.11.1-cu12.8
|
||||
|
||||
# Host port override
|
||||
LMDEPLOY_PORT_OVERRIDE=23333
|
||||
|
||||
# Model path or HuggingFace model ID
|
||||
# Examples:
|
||||
# - internlm/internlm2-chat-1_8b
|
||||
# - Qwen/Qwen2.5-7B-Instruct
|
||||
LMDEPLOY_MODEL=internlm/internlm2-chat-1_8b
|
||||
|
||||
# HuggingFace token for private models
|
||||
HF_TOKEN=
|
||||
|
||||
# Resource limits
|
||||
LMDEPLOY_CPU_LIMIT=4.0
|
||||
LMDEPLOY_MEMORY_LIMIT=8G
|
||||
LMDEPLOY_CPU_RESERVATION=2.0
|
||||
LMDEPLOY_MEMORY_RESERVATION=4G
|
||||
|
||||
# Shared memory size (required for some models)
|
||||
LMDEPLOY_SHM_SIZE=4g
|
||||
|
||||
# Timezone
|
||||
TZ=UTC
|
||||
29
src/lmdeploy/README.md
Normal file
29
src/lmdeploy/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# LMDeploy Docker Compose
|
||||
|
||||
[LMDeploy](https://github.com/InternLM/lmdeploy) is a toolkit for compressing, deploying, and serving LLMs.
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. (Optional) Configure the model and port in `.env`.
|
||||
2. Start the service:
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
3. Access the OpenAI compatible API at `http://localhost:23333/v1`.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Environment Variable | Default | Description |
|
||||
| ------------------------ | ------------------------------ | ------------------------------------ |
|
||||
| `LMDEPLOY_VERSION` | `v0.11.1-cu12.8` | LMDeploy image version |
|
||||
| `LMDEPLOY_PORT_OVERRIDE` | `23333` | Host port for the API server |
|
||||
| `LMDEPLOY_MODEL` | `internlm/internlm2-chat-1_8b` | HuggingFace model ID or local path |
|
||||
| `HF_TOKEN` | | HuggingFace token for private models |
|
||||
|
||||
## Monitoring Health
|
||||
|
||||
The service includes a health check that verifies if the OpenAI `/v1/models` endpoint is responsive.
|
||||
|
||||
## GPU Support
|
||||
|
||||
By default, this configuration reserves 1 NVIDIA GPU. Ensure you have the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) installed on your host.
|
||||
29
src/lmdeploy/README.zh.md
Normal file
29
src/lmdeploy/README.zh.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# LMDeploy Docker Compose
|
||||
|
||||
[LMDeploy](https://github.com/InternLM/lmdeploy) 是一个用于压缩、部署和服务大语言模型(LLM)的工具包。
|
||||
|
||||
## 快速开始
|
||||
|
||||
1. (可选)在 `.env` 中配置模型和端口。
|
||||
2. 启动服务:
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
3. 通过 `http://localhost:23333/v1` 访问与 OpenAI 兼容的 API。
|
||||
|
||||
## 配置项
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
| ------------------------ | ------------------------------ | ------------------------------------ |
|
||||
| `LMDEPLOY_VERSION` | `v0.11.1-cu12.8` | LMDeploy 镜像版本 |
|
||||
| `LMDEPLOY_PORT_OVERRIDE` | `23333` | API 服务器的主机端口 |
|
||||
| `LMDEPLOY_MODEL` | `internlm/internlm2-chat-1_8b` | HuggingFace 模型 ID 或本地路径 |
|
||||
| `HF_TOKEN` | | 用于访问私有模型的 HuggingFace Token |
|
||||
|
||||
## 健康检查
|
||||
|
||||
该配置包含健康检查,用于验证 OpenAI `/v1/models` 接口是否响应。
|
||||
|
||||
## GPU 支持
|
||||
|
||||
默认情况下,此配置会预留 1 个 NVIDIA GPU。请确保您的主机已安装 [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)。
|
||||
50
src/lmdeploy/docker-compose.yaml
Normal file
50
src/lmdeploy/docker-compose.yaml
Normal file
@@ -0,0 +1,50 @@
|
||||
x-defaults: &defaults
|
||||
restart: unless-stopped
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: 100m
|
||||
max-file: "3"
|
||||
|
||||
services:
|
||||
lmdeploy:
|
||||
<<: *defaults
|
||||
image: ${GLOBAL_REGISTRY:-}openmmlab/lmdeploy:${LMDEPLOY_VERSION:-v0.11.1-cu12.8}
|
||||
ports:
|
||||
- "${LMDEPLOY_PORT_OVERRIDE:-23333}:23333"
|
||||
volumes:
|
||||
- lmdeploy_data:/root/.cache
|
||||
environment:
|
||||
- TZ=${TZ:-UTC}
|
||||
- HF_TOKEN=${HF_TOKEN:-}
|
||||
command:
|
||||
- lmdeploy
|
||||
- serve
|
||||
- api_server
|
||||
- ${LMDEPLOY_MODEL:-internlm/internlm2-chat-1_8b}
|
||||
- --server-name
|
||||
- "0.0.0.0"
|
||||
- --server-port
|
||||
- "23333"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:23333/v1/models"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: ${LMDEPLOY_CPU_LIMIT:-4.0}
|
||||
memory: ${LMDEPLOY_MEMORY_LIMIT:-8G}
|
||||
reservations:
|
||||
cpus: ${LMDEPLOY_CPU_RESERVATION:-2.0}
|
||||
memory: ${LMDEPLOY_MEMORY_RESERVATION:-4G}
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
shm_size: ${LMDEPLOY_SHM_SIZE:-4g}
|
||||
|
||||
volumes:
|
||||
lmdeploy_data:
|
||||
Reference in New Issue
Block a user