Operating Systems

Linux

We recommend using your preferred log forwarding agent, such as Vector (journald source), Fluent Bit (systemd or kmesg inputs), filebeat (journald input), or Grafana Alloy (journal source).

Each of these log forwarding agents has robust support for forwarding kernel logs and other files you want to ship.

Windows

We recommend using Fluent Bit to ship Windows event logs and logfiles to SparkLogs. We recommend the EXE installer and registering it as a Windows service that starts automatically with the system.

Example Fluent Bit configuration template for Windows

Fluent Bit is fast and efficient for reading Windows event logs and local logfiles. Make sure to customize your <REGION>, <AGENT-ID>, and <AGENT_ACCESS-TOKEN>. Here is an example to ship the application and system logs.

# Customize as appropriate
[SERVICE]
    flush       5
    log_level   Info
    grace       30

[INPUT]
    name                    winevtlog
    tag                     windows_events
    # Change to False if you only want to read new events at first run
    read_existing_events    True
    render_event_as_xml     False
    mem_buf_limit           32MB
    # Add more channels as needed. e.g., Setup,Windows PowerShell
    channels                Application,System,Security
    read_limit_per_cycle    1MB
    DB                      winlog.sqlite

# add other INPUT sections as needed, e.g., for shipping additional logfiles
#[INPUT]
#    name                    tail
#    tag                     mylogfile
#    path                    C:\myapp\mylogfile*.log
#    key                     message
#    path_key                file
#    read_from_head          True
#    mem_buf_limit           32MB
#    skip_empty_lines        True
#    DB                      mylogfile.sqlite

[OUTPUT]
    name                http
    match               *
    host                ingest-<REGION>.engine.sparklogs.app
    port                443
    uri                 /ingest/v1
    net.connect_timeout 30
    tls                 On
    tls.verify          On
    tls.verify_hostname On
    format              json
    json_date_format    iso8601
    json_date_key       observedtimestamp
    compress            gzip
    workers             4
    http_user           <AGENT-ID>
    http_passwd         <AGENT-ACCESS-TOKEN>
    # Customize headers as desired, e.g., set to "true" to disable AutoExtract
    #header              X-No-AutoExtract false