feat: add mcp-servers/**

src/litellm/.env.example

@@ -0,0 +1,55 @@
+# LiteLLM version
+LITELLM_VERSION=main-stable
+
+# LiteLLM port (default: 4000)
+LITELLM_PORT_OVERRIDE=4000
+
+# PostgreSQL configuration
+POSTGRES_VERSION=16
+POSTGRES_PASSWORD=xxxxxx
+POSTGRES_PORT_OVERRIDE=5432
+
+# Prometheus configuration (optional, enable with --profile metrics)
+PROMETHEUS_VERSION=v3.3.1
+PROMETHEUS_PORT_OVERRIDE=9090
+
+# LiteLLM authentication keys
+LITELLM_MASTER_KEY=sk-xxxxxx
+LITELLM_SALT_KEY=sk-xxxxxx
+
+# Timezone
+TZ=UTC
+
+# ===== API Keys =====
+# OpenAI
+OPENAI_API_KEY=
+OPENAI_BASE_URL=
+
+# Cohere
+COHERE_API_KEY=
+
+# OpenRouter
+OR_SITE_URL=
+OR_APP_NAME=LiteLLM Example app
+OR_API_KEY=
+
+# Azure
+AZURE_API_BASE=
+AZURE_API_VERSION=
+AZURE_API_KEY=
+
+# Replicate
+REPLICATE_API_KEY=
+REPLICATE_API_TOKEN=
+
+# Anthropic
+ANTHROPIC_API_KEY=
+
+# Infisical
+INFISICAL_TOKEN=
+
+# Novita AI
+NOVITA_API_KEY=
+
+# INFINITY
+INFINITY_API_KEY=

src/litellm/README.md

@@ -0,0 +1,111 @@
+# LiteLLM
+
+[English](./README.md) | [中文](./README.zh.md)
+
+This service deploys LiteLLM, a unified interface to 100+ LLM APIs (OpenAI, Azure, Anthropic, Cohere, Replicate, etc.) with load balancing, fallbacks, and cost tracking.
+
+## Services
+
+- `litellm`: The LiteLLM proxy service
+- `db`: PostgreSQL database for storing model configurations and usage data
+- `prometheus`: Prometheus metrics collector (optional, enabled with `--profile metrics`)
+
+## Environment Variables
+
+| Variable Name            | Description                                                    | Default Value |
+| ------------------------ | -------------------------------------------------------------- | ------------- |
+| LITELLM_VERSION          | LiteLLM image version                                          | `main-stable` |
+| LITELLM_PORT_OVERRIDE    | Host port mapping for LiteLLM (maps to port 4000 in container) | 4000          |
+| POSTGRES_VERSION         | PostgreSQL image version                                       | `16`          |
+| POSTGRES_PASSWORD        | PostgreSQL database password                                   | `xxxxxx`      |
+| POSTGRES_PORT_OVERRIDE   | Host port mapping for PostgreSQL                               | 5432          |
+| PROMETHEUS_VERSION       | Prometheus image version (used with metrics profile)           | `v3.3.1`      |
+| PROMETHEUS_PORT_OVERRIDE | Host port mapping for Prometheus                               | 9090          |
+| LITELLM_MASTER_KEY       | Master key for LiteLLM authentication                          | `sk-xxxxxx`   |
+| LITELLM_SALT_KEY         | Salt key for secure key generation                             | `sk-xxxxxx`   |
+| TZ                       | Timezone setting                                               | `UTC`         |
+
+Additional API keys can be configured in the `.env` file for various LLM providers (OpenAI, Azure, Anthropic, etc.).
+
+Please modify the `.env` file as needed for your use case.
+
+## Volumes
+
+- `postgres_data`: PostgreSQL data persistence
+- `prometheus_data`: Prometheus data storage (optional)
+- `./config.yaml`: LiteLLM configuration file (optional, uncomment in docker-compose.yaml to use)
+- `./prometheus.yml`: Prometheus configuration file (optional)
+
+## Ports
+
+- `4000`: LiteLLM proxy API and Web UI
+- `5432`: PostgreSQL database
+- `9090`: Prometheus metrics (optional, enabled with `--profile metrics`)
+
+## First-Time Setup
+
+1. Start the services (with optional metrics):
+
+   ```bash
+   docker compose up -d
+   # Or with Prometheus metrics:
+   docker compose --profile metrics up -d
+   ```
+
+2. Access LiteLLM UI at `http://localhost:4000`
+
+3. Default credentials:
+   - Username: `admin`
+   - Password: Value of `LITELLM_MASTER_KEY` environment variable
+
+4. Configure your LLM API keys in the `.env` file or through the web UI
+
+## Configuration
+
+### Using a Config File
+
+To use a `config.yaml` file for configuration:
+
+1. Create a `config.yaml` file in the same directory as `docker-compose.yaml`
+2. Uncomment the volumes and command sections in `docker-compose.yaml`
+3. Configure your models, API keys, and routing rules in `config.yaml`
+
+### API Keys
+
+Add API keys for your LLM providers in the `.env` file:
+
+- `OPENAI_API_KEY`: OpenAI API key
+- `ANTHROPIC_API_KEY`: Anthropic API key
+- `AZURE_API_KEY`: Azure OpenAI API key
+- And more (see `.env.example`)
+
+## Usage
+
+### Making API Calls
+
+Use the LiteLLM proxy endpoint with your master key:
+
+```bash
+curl -X POST http://localhost:4000/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [{"role": "user", "content": "Hello!"}]
+  }'
+```
+
+### Monitoring with Prometheus
+
+If you enabled the metrics profile, access Prometheus at `http://localhost:9090` to view metrics about:
+
+- Request counts and latencies
+- Token usage
+- Cost tracking
+- Error rates
+
+## Additional Information
+
+- Official Documentation: <https://docs.litellm.ai/>
+- GitHub Repository: <https://github.com/BerriAI/litellm>
+- Supported LLM Providers: <https://docs.litellm.ai/docs/providers>

src/litellm/README.zh.md

@@ -0,0 +1,3 @@
+# LiteLLM
+
+默认情况下，用户名是 `admin`，密码是 `$MASTER_KEY` 变量的值。

src/litellm/docker-compose.yaml

@@ -0,0 +1,110 @@
+x-default: &default
+  restart: unless-stopped
+  logging:
+    driver: json-file
+    options:
+      max-size: 100m
+      max-file: "3"
+
+services:
+  litellm:
+    <<: *default
+    build:
+      context: .
+      args:
+        target: runtime
+    image: ghcr.io/berriai/litellm:${LITELLM_VERSION:-main-stable}
+    # Uncomment these lines to start proxy with a config.yaml file
+    # volumes:
+    #   - ./config.yaml:/app/config.yaml:ro
+    # command:
+    #   - "--config=/app/config.yaml"
+    ports:
+      - "${LITELLM_PORT_OVERRIDE:-4000}:4000"
+    environment:
+      - DATABASE_URL=postgresql://llmproxy:${POSTGRES_PASSWORD}@db:5432/litellm
+      - STORE_MODEL_IN_DB=True
+      - TZ=${TZ:-UTC}
+    env_file:
+      - .env
+    depends_on:
+      db:
+        condition: service_healthy
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:4000/health/liveliness"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    deploy:
+      resources:
+        limits:
+          cpus: '2.00'
+          memory: 2G
+        reservations:
+          cpus: '0.50'
+          memory: 512M
+
+  db:
+    <<: *default
+    image: postgres:${POSTGRES_VERSION:-16}
+    environment:
+      - POSTGRES_DB=litellm
+      - POSTGRES_USER=llmproxy
+      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
+      - TZ=${TZ:-UTC}
+    ports:
+      - "${POSTGRES_PORT_OVERRIDE:-5432}:5432"
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 10s
+    deploy:
+      resources:
+        limits:
+          cpus: '1.00'
+          memory: 1G
+        reservations:
+          cpus: '0.25'
+          memory: 256M
+
+  prometheus:
+    <<: *default
+    image: prom/prometheus:${PROMETHEUS_VERSION:-v3.3.1}
+    profiles:
+      - metrics
+    volumes:
+      - prometheus_data:/prometheus
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
+    ports:
+      - "${PROMETHEUS_PORT_OVERRIDE:-9090}:9090"
+    command:
+      - "--config.file=/etc/prometheus/prometheus.yml"
+      - "--storage.tsdb.path=/prometheus"
+      - "--storage.tsdb.retention.time=15d"
+    environment:
+      - TZ=${TZ:-UTC}
+    healthcheck:
+      test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 10s
+    deploy:
+      resources:
+        limits:
+          cpus: '1.00'
+          memory: 1G
+        reservations:
+          cpus: '0.25'
+          memory: 256M
+
+volumes:
+  prometheus_data:
+  postgres_data:

src/litellm/prometheus.yml

@@ -0,0 +1,7 @@
+global:
+  scrape_interval: 15s
+
+scrape_configs:
+  - job_name: 'litellm'
+    static_configs:
+      - targets: ['litellm:4000']  # Assuming Litellm exposes metrics at port 4000