Skip to main content

OpenTelemetry Collector

Overview​

The OpenTelemetry Collector is a cross-platform high-performance agent to receive, process, and forward telemetry data, written in golang. The OpenTelemetry Collector receives data from various sources, optionally performs data transformations, and then ships data to destinations (like SparkLogs).

Over 80 data sources (called receivers) are supported. Popular receivers for logs include Windows event logs, files, journald, syslog, Kubernetes, Kafka, and netflow. OpenTelemetry Collector can ship logs to SparkLogs using the elasticsearch exporter.

Although AutoExtract will automatically extract structured field data from your raw logs and is recommended in the typical case, you can also manually parse log data within the collector into structured fields using receiver operators. You can also enrich your data using processors, including commonly used ones like resource detection (add information about the host system like hostname) and attributes (add custom attributes to each shipped event).

How to Use​

Follow these steps for each logical agent that will receive data from OpenTelemetry Collector:

1. Consider deployment design and topology​

In its simplest form, OpenTelemetry Collector is deployed as an agent on each machine and will send data directly to SparkLogs. SparkLogs is highly scalable and can receive data from distributed OpenTelemetry Collector agents without bottlenecks.

OpenTelemetry Collector can also be deployed to aggregate log data locally, and then ship this aggregated log data to SparkLogs. If you have complex requirements, review the OpenTelemetry Collector deployment guide, and collector overview.

2. Create agent and get config template​

In the app, click the Configure sidebar button:
Configure Sidebar Button
and then click the Agents tab.

As appropriate, create a new agent, or highlight an existing agent and click View API Key. In the dialog that shows the agent configuration template, click the OpenTelemetry Collector tab and copy the configuration template.

3. Customize configuration​

Copy the configuration template and customize it based on your needs. At a minimum, add receivers in the YAML config and modify the exporter with your SparkLogs authentication credentials.

Example OpenTelemetry Collector config.yaml

This example template is for Windows event logs. Customize the receivers as appropriate (e.g., for files, journald, syslog, etc.).

Make sure to get your configuration template from the app, as your ingestion endpoint can vary based on your provisioned region. This is an example of what it will look like:

extensions:
  health_check:
    endpoint: 127.0.0.1:13133

receivers:
  windowseventlog/application:
    channel: application
  windowseventlog/system:
    channel: system
  windowseventlog/security:
    channel: security
    
processors:
  # Add host system information to events (hostname, etc.)
  resourcedetection:
    detectors: [system]
    system:
      hostname_sources: ["os"]
  # Tune for SparkLogs
  batch:
    send_batch_size: 2000
    send_batch_max_size: 5000
    timeout: 5s

exporters:
  debug:
    verbosity: basic
  elasticsearch/sparklogs:
    endpoints: ["https://es8.ingest-<REGION>.engine.sparklogs.app/"]
    logs_index: otel-logs
    user: "<AGENT-ID>"
    password: "<AGENT-ACCESS-TOKEN>"
    timeout: 90s

service:
  pipelines:
    logs:
      receivers: [windowseventlog/application, windowseventlog/system, windowseventlog/security]
      processors: [resourcedetection, batch]
      exporters: [elasticsearch/sparklogs]
  telemetry:
    metrics:
      level: detailed
      readers:
        - pull:
            exporter:
              prometheus:
                host: 127.0.0.1
                port: 8888
  extensions: [health_check]

4. Deploy OpenTelemetry Collector agents​

On each system that will ship data to SparkLogs for this agent, install the OpenTelemetry Collector. We recommend using the contrib distribution, which includes common receivers and exporters. For example, there are MSI installer files for Windows and packages for various Linux distributions.

Once installed, customize the config.yaml file with your configuration and make sure the collector starts as a service on system boot (the Windows MSI installer and Linux packages already do this for you). On Linux the config file is typically at /etc/otelcol-contrib/config.yaml and on Windows it will be at C:\Program Files\OpenTelemetry Collector Contrib\config.yaml.