Update MinerU, Gitea, InfluxDB, Phoenix, and Selenium configurations

- Bump MinerU version from 2.7.1 to 2.7.2 in .env.example, Dockerfile, README files, and docker-compose.yaml.
- Update Gitea version from 1.25.2-rootless to 1.25.4-rootless in .env.example and docker-compose.yaml.
- Add InfluxDB configuration files including .env.example, README.md, README.zh.md, and docker-compose.yaml with version 2.8.0.
- Bump Phoenix version from 12.28.1-nonroot to 12.31.2-nonroot in .env.example and docker-compose.yaml, and update README files.
- Introduce Selenium standalone configuration with version 144.0-20260120, including .env.example, README.md, README.zh.md, and docker-compose.yaml.
This commit is contained in:
Sun-ZhenXing
2026-01-25 22:59:55 +08:00
parent 32742dc535
commit aeddac52bf
24 changed files with 1101 additions and 20 deletions

48
src/selenium/.env.example Normal file
View File

@@ -0,0 +1,48 @@
# Selenium Standalone Configuration
# Image Registry (optional)
# GLOBAL_REGISTRY=
# Selenium Version (stable version tag recommended)
# Visit https://hub.docker.com/r/selenium/standalone-chrome/tags for available versions
# Format: <browser-version>-<date> or <browser-version>-chromedriver-<driver-version>-grid-<grid-version>-<date>
SELENIUM_VERSION=144.0-20260120
# Shared Memory Size (required for browser stability)
# Chrome and Firefox need sufficient shared memory to prevent crashes
SELENIUM_SHM_SIZE=2g
# Port Configuration
# Selenium Grid HTTP port
SELENIUM_GRID_PORT_OVERRIDE=4444
# VNC port for viewing browser sessions (browser debugger)
SELENIUM_VNC_PORT_OVERRIDE=7900
# Timezone
TZ=UTC
# Screen Resolution Settings
SE_SCREEN_WIDTH=1920
SE_SCREEN_HEIGHT=1080
SE_SCREEN_DEPTH=24
SE_SCREEN_DPI=96
# VNC Configuration
# Password for VNC access (default: secret)
SE_VNC_PASSWORD=secret
# Session Configuration
# Maximum concurrent sessions per container
SE_NODE_MAX_SESSIONS=1
# Session timeout in seconds (default: 300)
SE_NODE_SESSION_TIMEOUT=300
# Xvfb Configuration
# Start virtual display server (required for headless mode in Chrome/Chromium v127+)
SE_START_XVFB=true
# Resource Limits
SELENIUM_CPU_LIMIT=2.0
SELENIUM_MEMORY_LIMIT=2G
SELENIUM_CPU_RESERVATION=1.0
SELENIUM_MEMORY_RESERVATION=1G

281
src/selenium/README.md Normal file
View File

@@ -0,0 +1,281 @@
# Selenium Standalone with Chrome
[![Docker Image](https://img.shields.io/docker/v/selenium/standalone-chrome?sort=semver)](https://hub.docker.com/r/selenium/standalone-chrome)
[![Docker Pulls](https://img.shields.io/docker/pulls/selenium/standalone-chrome)](https://hub.docker.com/r/selenium/standalone-chrome)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/LICENSE.md)
Selenium Grid in Standalone mode with Chrome browser for browser automation at scale.
## Quick Start
```bash
# Start the service
docker compose up -d
# Verify the service is running
docker compose ps
# View logs
docker compose logs -f
# Stop the service
docker compose down
```
## Service Information
### Ports
| Port | Service | Description |
| ---- | ------------- | -------------------------------------------- |
| 4444 | Selenium Grid | HTTP endpoint for WebDriver |
| 7900 | noVNC | Browser viewing interface (password: secret) |
### Default Credentials
- VNC Password: `secret` (configurable via `SE_VNC_PASSWORD`)
### Volumes
- `selenium_downloads`: Browser downloads directory (`/home/seluser/Downloads`)
## Configuration
### Environment Variables
All configuration can be customized via the `.env` file:
```bash
# Copy the example configuration
cp .env.example .env
# Edit the configuration
nano .env
```
Key configurations:
| Variable | Default | Description |
| ----------------------------- | ---------------- | --------------------------------------------------- |
| `SELENIUM_VERSION` | `144.0-20260120` | Docker image tag (Chrome version + date) |
| `SELENIUM_SHM_SIZE` | `2g` | Shared memory size (required for browser stability) |
| `SELENIUM_GRID_PORT_OVERRIDE` | `4444` | Grid HTTP endpoint port |
| `SELENIUM_VNC_PORT_OVERRIDE` | `7900` | noVNC viewer port |
| `SE_SCREEN_WIDTH` | `1920` | Browser screen width |
| `SE_SCREEN_HEIGHT` | `1080` | Browser screen height |
| `SE_NODE_MAX_SESSIONS` | `1` | Max concurrent sessions per container |
| `SE_NODE_SESSION_TIMEOUT` | `300` | Session timeout in seconds |
For a complete list of environment variables, see the [Selenium Docker documentation](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/ENV_VARIABLES.md).
## Usage
### Basic WebDriver Test (Python)
```python
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# Configure Chrome options
options = Options()
# Connect to Selenium Grid
driver = webdriver.Remote(
command_executor='http://localhost:4444',
options=options
)
# Run your test
driver.get('https://www.selenium.dev/')
print(driver.title)
# Clean up
driver.quit()
```
### Basic WebDriver Test (Node.js)
```javascript
const { Builder } = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');
(async function example() {
let driver = await new Builder()
.forBrowser('chrome')
.usingServer('http://localhost:4444')
.build();
try {
await driver.get('https://www.selenium.dev/');
console.log(await driver.getTitle());
} finally {
await driver.quit();
}
})();
```
### Viewing Browser Sessions
You can watch tests execute in real-time using noVNC:
1. Open your browser to `http://localhost:7900/?autoconnect=1&resize=scale&password=secret`
2. The default VNC password is `secret`
3. You'll see the browser session in real-time
Alternatively, use a VNC client to connect to `localhost:5900` (if exposed).
## Advanced Configuration
### Changing Browser Version
To use a specific Chrome version, update the `SELENIUM_VERSION` in your `.env` file:
```bash
# Use Chrome 143.0
SELENIUM_VERSION=143.0-20260120
# Or use a specific Selenium Grid version
SELENIUM_VERSION=144.0-chromedriver-144.0-grid-4.40.0-20260120
```
Visit [Docker Hub](https://hub.docker.com/r/selenium/standalone-chrome/tags) for available versions.
### Increasing Concurrent Sessions
To run multiple concurrent sessions in one container (not recommended for production):
```bash
SE_NODE_MAX_SESSIONS=5
```
**Note:** For better stability, scale containers instead:
```bash
docker compose up -d --scale selenium-chrome=3
```
### Retrieving Downloaded Files
To access files downloaded during tests, mount the downloads directory:
```yaml
volumes:
- ./downloads:/home/seluser/Downloads
```
**Linux users:** Set proper permissions before mounting:
```bash
mkdir -p ./downloads
sudo chown 1200:1201 ./downloads
```
### Running in Headless Mode
For newer Chrome versions (127+), headless mode requires Xvfb:
```bash
SE_START_XVFB=true
```
Then configure headless in your test:
```python
options = Options()
options.add_argument('--headless=new')
```
### Custom Screen Resolution
Adjust screen resolution for your test needs:
```bash
SE_SCREEN_WIDTH=1366
SE_SCREEN_HEIGHT=768
SE_SCREEN_DEPTH=24
SE_SCREEN_DPI=74
```
## Health Check
The container includes a built-in health check that polls the Grid status endpoint every 30 seconds:
```bash
# Check container health
docker compose ps
# Or inspect the health status
docker inspect --format='{{json .State.Health.Status}}' <container-id>
```
## Troubleshooting
### Browser Crashes
If you see errors like "Chrome failed to start" or "invalid argument: can't kill an exited process":
1. **Ensure sufficient shared memory:** The default `2g` should work for most cases
```bash
SELENIUM_SHM_SIZE=2g
```
2. **Check headless mode configuration:** Make sure `SE_START_XVFB=true` if using headless mode with Chrome 127+
### Permission Issues (Linux)
When mounting volumes on Linux, ensure correct permissions:
```bash
# For downloads directory
mkdir -p ./downloads
sudo chown 1200:1201 ./downloads
# Check user/group IDs in container
docker compose exec selenium-chrome id
```
### Resource Constraints
If tests are slow or containers are being OOM killed:
```bash
# Increase resource limits
SELENIUM_CPU_LIMIT=4.0
SELENIUM_MEMORY_LIMIT=4G
```
### VNC Connection Issues
If you can't connect to VNC:
1. Check that port 7900 is not in use
2. Verify the VNC password is correct (default: `secret`)
3. Try disabling VNC authentication: `SE_VNC_NO_PASSWORD=true`
## Multi-Browser Support
For running multiple browser types (Chrome, Firefox, Edge), consider using:
- **Hub & Nodes architecture:** See `docker-compose-grid.yaml` example
- **Dynamic Grid:** Automatically spawns containers on demand
- **Selenium Grid 4:** Full distributed mode with Router, Distributor, etc.
## Additional Resources
- [Selenium Documentation](https://www.selenium.dev/documentation/)
- [Docker Selenium GitHub](https://github.com/SeleniumHQ/docker-selenium)
- [Selenium Grid Configuration](https://www.selenium.dev/documentation/grid/)
- [Environment Variables Reference](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/ENV_VARIABLES.md)
## Security Notes
- **VNC Password:** Change the default `secret` password in production
- **Network Exposure:** Do not expose Selenium Grid directly to the internet
- **Resource Limits:** Always set CPU and memory limits to prevent resource exhaustion
- **User Permissions:** Selenium runs as non-root user `seluser` (UID 1200, GID 1201)
## License
This configuration is provided under the Apache License 2.0, following the Selenium project's licensing.
The Selenium Docker images are maintained by the SeleniumHQ team and community contributors.

281
src/selenium/README.zh.md Normal file
View File

@@ -0,0 +1,281 @@
# Selenium Standalone Chrome
[![Docker Image](https://img.shields.io/docker/v/selenium/standalone-chrome?sort=semver)](https://hub.docker.com/r/selenium/standalone-chrome)
[![Docker Pulls](https://img.shields.io/docker/pulls/selenium/standalone-chrome)](https://hub.docker.com/r/selenium/standalone-chrome)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/LICENSE.md)
Selenium Grid 独立模式,配备 Chrome 浏览器,用于大规模浏览器自动化。
## 快速开始
```bash
# 启动服务
docker compose up -d
# 验证服务运行状态
docker compose ps
# 查看日志
docker compose logs -f
# 停止服务
docker compose down
```
## 服务信息
### 端口
| 端口 | 服务 | 说明 |
| ---- | ------------- | ------------------------------ |
| 4444 | Selenium Grid | WebDriver HTTP 端点 |
| 7900 | noVNC | 浏览器查看界面密码secret |
### 默认凭据
- VNC 密码:`secret`(可通过 `SE_VNC_PASSWORD` 配置)
### 数据卷
- `selenium_downloads`:浏览器下载目录(`/home/seluser/Downloads`
## 配置说明
### 环境变量
所有配置都可以通过 `.env` 文件自定义:
```bash
# 复制示例配置文件
cp .env.example .env
# 编辑配置
nano .env
```
主要配置:
| 变量 | 默认值 | 说明 |
| ----------------------------- | ---------------- | ------------------------------------- |
| `SELENIUM_VERSION` | `144.0-20260120` | Docker 镜像标签Chrome 版本 + 日期) |
| `SELENIUM_SHM_SIZE` | `2g` | 共享内存大小(浏览器稳定性所需) |
| `SELENIUM_GRID_PORT_OVERRIDE` | `4444` | Grid HTTP 端点端口 |
| `SELENIUM_VNC_PORT_OVERRIDE` | `7900` | noVNC 查看器端口 |
| `SE_SCREEN_WIDTH` | `1920` | 浏览器屏幕宽度 |
| `SE_SCREEN_HEIGHT` | `1080` | 浏览器屏幕高度 |
| `SE_NODE_MAX_SESSIONS` | `1` | 每个容器最大并发会话数 |
| `SE_NODE_SESSION_TIMEOUT` | `300` | 会话超时时间(秒) |
完整的环境变量列表请参考 [Selenium Docker 文档](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/ENV_VARIABLES.md)。
## 使用方法
### 基础 WebDriver 测试Python
```python
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# 配置 Chrome 选项
options = Options()
# 连接到 Selenium Grid
driver = webdriver.Remote(
command_executor='http://localhost:4444',
options=options
)
# 运行测试
driver.get('https://www.selenium.dev/')
print(driver.title)
# 清理资源
driver.quit()
```
### 基础 WebDriver 测试Node.js
```javascript
const { Builder } = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');
(async function example() {
let driver = await new Builder()
.forBrowser('chrome')
.usingServer('http://localhost:4444')
.build();
try {
await driver.get('https://www.selenium.dev/');
console.log(await driver.getTitle());
} finally {
await driver.quit();
}
})();
```
### 查看浏览器会话
您可以使用 noVNC 实时查看测试执行过程:
1. 在浏览器中打开 `http://localhost:7900/?autoconnect=1&resize=scale&password=secret`
2. 默认 VNC 密码是 `secret`
3. 您将实时看到浏览器会话
或者,使用 VNC 客户端连接到 `localhost:5900`(如果已暴露)。
## 高级配置
### 更改浏览器版本
要使用特定的 Chrome 版本,请在 `.env` 文件中更新 `SELENIUM_VERSION`
```bash
# 使用 Chrome 143.0
SELENIUM_VERSION=143.0-20260120
# 或使用特定的 Selenium Grid 版本
SELENIUM_VERSION=144.0-chromedriver-144.0-grid-4.40.0-20260120
```
访问 [Docker Hub](https://hub.docker.com/r/selenium/standalone-chrome/tags) 查看可用版本。
### 增加并发会话数
在单个容器中运行多个并发会话(生产环境不推荐):
```bash
SE_NODE_MAX_SESSIONS=5
```
**注意:** 为了更好的稳定性,建议通过扩展容器来实现:
```bash
docker compose up -d --scale selenium-chrome=3
```
### 获取下载的文件
要访问测试期间下载的文件,挂载下载目录:
```yaml
volumes:
- ./downloads:/home/seluser/Downloads
```
**Linux 用户:** 挂载前设置正确的权限:
```bash
mkdir -p ./downloads
sudo chown 1200:1201 ./downloads
```
### 无头模式运行
对于新版 Chrome127+),无头模式需要 Xvfb
```bash
SE_START_XVFB=true
```
然后在测试中配置无头模式:
```python
options = Options()
options.add_argument('--headless=new')
```
### 自定义屏幕分辨率
根据测试需求调整屏幕分辨率:
```bash
SE_SCREEN_WIDTH=1366
SE_SCREEN_HEIGHT=768
SE_SCREEN_DEPTH=24
SE_SCREEN_DPI=74
```
## 健康检查
容器包含内置的健康检查,每 30 秒轮询 Grid 状态端点:
```bash
# 检查容器健康状态
docker compose ps
# 或检查健康状态详情
docker inspect --format='{{json .State.Health.Status}}' <container-id>
```
## 故障排除
### 浏览器崩溃
如果看到 "Chrome failed to start" 或 "invalid argument: can't kill an exited process" 等错误:
1. **确保足够的共享内存:** 默认的 `2g` 应该适用于大多数情况
```bash
SELENIUM_SHM_SIZE=2g
```
2. **检查无头模式配置:** 如果在 Chrome 127+ 中使用无头模式,请确保 `SE_START_XVFB=true`
### 权限问题Linux
在 Linux 上挂载卷时,确保正确的权限:
```bash
# 对于下载目录
mkdir -p ./downloads
sudo chown 1200:1201 ./downloads
# 检查容器中的用户/组 ID
docker compose exec selenium-chrome id
```
### 资源限制
如果测试缓慢或容器被 OOM 终止:
```bash
# 增加资源限制
SELENIUM_CPU_LIMIT=4.0
SELENIUM_MEMORY_LIMIT=4G
```
### VNC 连接问题
如果无法连接到 VNC
1. 检查端口 7900 是否被占用
2. 验证 VNC 密码是否正确(默认:`secret`
3. 尝试禁用 VNC 认证:`SE_VNC_NO_PASSWORD=true`
## 多浏览器支持
要运行多种浏览器类型Chrome、Firefox、Edge请考虑使用
- **Hub & Nodes 架构:** 参见 `docker-compose-grid.yaml` 示例
- **动态 Grid** 按需自动生成容器
- **Selenium Grid 4** 完整的分布式模式,包含 Router、Distributor 等
## 其他资源
- [Selenium 文档](https://www.selenium.dev/documentation/)
- [Docker Selenium GitHub](https://github.com/SeleniumHQ/docker-selenium)
- [Selenium Grid 配置](https://www.selenium.dev/documentation/grid/)
- [环境变量参考](https://github.com/SeleniumHQ/docker-selenium/blob/trunk/ENV_VARIABLES.md)
## 安全注意事项
- **VNC 密码:** 生产环境中更改默认的 `secret` 密码
- **网络暴露:** 不要将 Selenium Grid 直接暴露到互联网
- **资源限制:** 始终设置 CPU 和内存限制以防止资源耗尽
- **用户权限:** Selenium 以非 root 用户 `seluser` 运行UID 1200GID 1201
## 许可证
本配置遵循 Apache License 2.0 提供,与 Selenium 项目的许可保持一致。
Selenium Docker 镜像由 SeleniumHQ 团队和社区贡献者维护。

View File

@@ -0,0 +1,50 @@
# Selenium Standalone with Chrome
# This configuration runs Selenium Grid in Standalone mode with Chrome browser
# Suitable for single-browser automation needs
x-defaults: &defaults
restart: unless-stopped
logging:
driver: json-file
options:
max-size: 100m
max-file: "3"
services:
selenium-chrome:
<<: *defaults
image: ${GLOBAL_REGISTRY:-}selenium/standalone-chrome:${SELENIUM_VERSION:-144.0-20260120}
shm_size: ${SELENIUM_SHM_SIZE:-2g}
ports:
- "${SELENIUM_GRID_PORT_OVERRIDE:-4444}:4444"
- "${SELENIUM_VNC_PORT_OVERRIDE:-7900}:7900"
volumes:
- selenium_downloads:/home/seluser/Downloads
environment:
- TZ=${TZ:-UTC}
- SE_SCREEN_WIDTH=${SE_SCREEN_WIDTH:-1920}
- SE_SCREEN_HEIGHT=${SE_SCREEN_HEIGHT:-1080}
- SE_SCREEN_DEPTH=${SE_SCREEN_DEPTH:-24}
- SE_SCREEN_DPI=${SE_SCREEN_DPI:-96}
- SE_VNC_PASSWORD=${SE_VNC_PASSWORD:-secret}
- SE_NODE_MAX_SESSIONS=${SE_NODE_MAX_SESSIONS:-1}
- SE_NODE_SESSION_TIMEOUT=${SE_NODE_SESSION_TIMEOUT:-300}
- SE_START_XVFB=${SE_START_XVFB:-true}
healthcheck:
test:
["CMD", "/opt/bin/check-grid.sh", "--host", "0.0.0.0", "--port", "4444"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
deploy:
resources:
limits:
cpus: ${SELENIUM_CPU_LIMIT:-2.0}
memory: ${SELENIUM_MEMORY_LIMIT:-2G}
reservations:
cpus: ${SELENIUM_CPU_RESERVATION:-1.0}
memory: ${SELENIUM_MEMORY_RESERVATION:-1G}
volumes:
selenium_downloads: