Skip to main content

Operating Systems

Linux​

We recommend using your preferred log collector, such as Vector (journald source), OpenTelemetry Collector (journald receiver), Fluent Bit (systemd or kmesg inputs), filebeat (journald input), or Grafana Alloy (journal source).

Each of these log collectors has robust support for forwarding kernel logs and other files you want to ship.

Windows​

We recommend using the SparkLogs Agent for Windows to ship Windows event logs and logfiles to SparkLogs. The SparkLogs Agent is a first-party, signed, self-updating endpoint agent that automatically discovers and collects operating system and application logs plus system health signals. There are no collector configs to write and no per-endpoint maintenance: install it and data flows.

Open Source Tools for Windows Logs​

You can also use open source tools like OpenTelemetry Collector to ship Windows event logs and logfiles to SparkLogs. We recommend the MSI installer of the contrib distribution of the collector, which automatically installs the collector as a Windows service. You can then customize C:\Program Files\OpenTelemetry Collector\config.yaml as desired and restart the service.

Example OpenTelemetry Collector configuration template for Windows

OpenTelemetry Collector is fast and efficient for reading Windows event logs and local log files. Set <INGEST-KEY-ID> and <INGEST-KEY-ACCESS-TOKEN> from Configure → Ingest Keys.

Here is an example to ship the application, system, and security Windows event logs.

extensions:
  health_check:
    endpoint: 127.0.0.1:13133

receivers:
  windowseventlog/application:
    channel: application
  windowseventlog/system:
    channel: system
  windowseventlog/security:
    channel: security
    
processors:
  # Add host system information to events (hostname, etc.)
  resourcedetection:
    detectors: [system]
    system:
      hostname_sources: ["os"]
  # Tune batch sizes for SparkLogs ingest
  batch:
    send_batch_size: 2000
    send_batch_max_size: 5000
    timeout: 5s

exporters:
  debug:
    verbosity: basic
  elasticsearch/sparklogs:
    endpoints: ["https://es8.ingest-us.engine.sparklogs.app/"]
    logs_index: otel-logs
    user: "<INGEST-KEY-ID>"
    password: "<INGEST-KEY-ACCESS-TOKEN>"
    timeout: 90s

service:
  pipelines:
    logs:
      receivers: [windowseventlog/application, windowseventlog/system, windowseventlog/security]
      processors: [resourcedetection, batch]
      exporters: [elasticsearch/sparklogs]
  telemetry:
    metrics:
      level: detailed
      readers:
        - pull:
            exporter:
              prometheus:
                host: 127.0.0.1
                port: 8888
  extensions: [health_check]