OpenTelemetry Collector is a vendor-agnostic service for receiving, processing, and exporting telemetry data (traces, metrics, and logs). It supports multiple protocols and can send data to various backends.

Features

Protocol Support: OTLP, Jaeger, Zipkin, Prometheus, and more
Flexible Processing: Filter, transform, and enrich telemetry data
Multiple Exporters: Send data to various observability backends
High Performance: Efficient data processing with low overhead
Extensible: Rich ecosystem of receivers, processors, and exporters
Vendor Neutral: Works with any OpenTelemetry-compatible backend

Quick Start

Copy the environment file and adjust if needed:
```
cp .env.example .env
```

Create a configuration file config.yaml:

# See example below or use the provided template

Start the collector:
```
docker compose up -d
```

Default Ports

Protocol	Port	Description
OTLP gRPC	4317	OpenTelemetry Protocol over gRPC
OTLP HTTP	4318	OpenTelemetry Protocol over HTTP
Jaeger gRPC	14250	Jaeger gRPC receiver
Jaeger Thrift HTTP	14268	Jaeger Thrift HTTP receiver
Zipkin	9411	Zipkin HTTP receiver
Prometheus	8888	Internal metrics endpoint
Health Check	13133	Health check endpoint

Configuration

Environment Variables

Key environment variables (see .env.example for complete list):

OTEL_COLLECTOR_VERSION: Collector version (default: 0.115.1)
OTEL_COLLECTOR_OTLP_GRPC_PORT_OVERRIDE: OTLP gRPC port (default: 4317)
OTEL_COLLECTOR_OTLP_HTTP_PORT_OVERRIDE: OTLP HTTP port (default: 4318)
OTEL_COLLECTOR_GOMEMLIMIT: Go memory limit for the collector

Configuration File

Create a config.yaml file to define the collector pipeline. Here's a minimal example:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 10s
    send_batch_size: 1024

exporters:
  logging:
    loglevel: info
  
  # Example: Export to Jaeger
  # otlp/jaeger:
  #   endpoint: jaeger:4317
  #   tls:
  #     insecure: true

  # Example: Export to Prometheus
  # prometheusremotewrite:
  #   endpoint: http://prometheus:9090/api/v1/write

extensions:
  health_check:
    endpoint: 0.0.0.0:13133
  pprof:
    endpoint: 0.0.0.0:1777

service:
  extensions: [health_check, pprof]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging]

For production use, configure appropriate exporters for your observability backend (e.g., Jaeger, Prometheus, Grafana Tempo, DataDog, etc.).

Common Receivers

OTLP: Native OpenTelemetry protocol (gRPC and HTTP)
Jaeger: Jaeger native formats
Zipkin: Zipkin JSON format
Prometheus: Prometheus scraping
Kafka: Receive from Kafka topics

Common Processors

Batch: Batch telemetry data before export
Memory Limiter: Prevent out-of-memory situations
Resource Detection: Automatically detect resource attributes
Attributes: Modify attributes on telemetry data
Sampling: Sample traces based on various strategies

Common Exporters

OTLP: Send to OTLP-compatible backends
Jaeger: Export to Jaeger
Zipkin: Export to Zipkin
Prometheus: Expose metrics for Prometheus scraping
Prometheus Remote Write: Push metrics to Prometheus
Logging: Log telemetry to console (for debugging)

Sending Data to the Collector

Using OpenTelemetry SDKs

Configure your application to send data to the collector:

Environment Variables:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc

Python Example:

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="localhost:4317", insecure=True))
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

Node.js Example:

const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');

const provider = new NodeTracerProvider();
const exporter = new OTLPTraceExporter({
  url: 'http://localhost:4317',
});
provider.addSpanProcessor(new BatchSpanProcessor(exporter));
provider.register();

Resource Requirements

Minimum recommended resources:

CPU: 0.5 cores
Memory: 1GB RAM

For high-throughput environments, increase resources accordingly.

Data Persistence

The collector itself is stateless. Data persistence depends on the configured exporters and backend systems.

Security Considerations

Configure TLS for production deployments
Use authentication when available (e.g., API keys, mTLS)
Restrict network access to necessary ports
Consider using the memory_limiter processor to prevent OOM
Review and minimize exposed ports
Use secrets management for sensitive configuration

Healthchecks

The collector exposes a health check endpoint on port 13133:

http://localhost:13133/ - Overall health status

Monitoring the Collector

The collector exposes its own metrics on the Prometheus metrics endpoint (default port 8888):

http://localhost:8888/metrics

Key metrics to monitor:

otelcol_receiver_accepted_spans: Number of spans accepted
otelcol_receiver_refused_spans: Number of spans refused
otelcol_exporter_sent_spans: Number of spans sent
otelcol_processor_batch_batch_send_size: Batch sizes

Troubleshooting

Data not being received: Check receiver configuration and port bindings
High memory usage: Configure memory_limiter processor
Slow processing: Adjust batch processor settings
Export failures: Check exporter configuration and backend connectivity

Performance Tuning

Use the batch processor to improve efficiency
Configure memory_limiter to prevent OOM
Adjust queue sizes based on throughput requirements
Use sampling processors for high-volume trace data
Monitor collector metrics for bottlenecks

License

OpenTelemetry Collector is licensed under the Apache 2.0 License. See the official repository for more details.

README.md

OpenTelemetry Collector