# Syslog Data Ingestion

## Overview

Syslog is a widespread standard for transmitting log messages from network devices and servers,
and is both a protocol and a message format. A variety of syslog standards have emerged
over its 40+ years of use. Syslog messages are sent over UDP, TCP, or TLS over TCP.

While a variety of standards exist for syslog message formatting, many vendors and devices
generate log messages in a semi-compliant or custom format. This historically makes it
tedious and time consuming to configure parsing rules for each vendor and device that is
sending syslog data.

SparkLogs solves this by automatically parsing a wide variety of standard syslog formats as well
as through [AutoExtract technology](https://sparklogs.com/docs/ingest/autoextract.md) that extracts structured
field data out of log text with zero configuration.

## Supported Syslog Formats

SparkLogs is tested to be compatible with the many variants of [RFC3164](#rfc3164), [RFC5424](#rfc5424) and proprietary
log formats from [Cisco](#cisco), [Juniper](#juniper), [SonicWall](#sonicwall), [WatchGuard](#watchguard), and [Fortinet](#fortinet). We expect that most syslog
formats will parse automatically without issue, and we welcome feedback on any incompatibilities.

## Capturing Syslog Data

The syslog protocol is not directly supported by the SparkLogs cloud, so you should use an open source
syslog log collector to receive data via UDP, TCP, or TLS and then forward it to SparkLogs via HTTPS.

We recommend using [Vector](https://sparklogs.com/docs/ingest/tools/vector.md) or [Fluentbit](https://sparklogs.com/docs/ingest/tools/fluentbit.md)
to receive and forward syslog data.

### Collection Topology

Decide where you want to place the agents that will receive syslog data, whether in each
LAN that has devices that transmit syslog data, or in a central location that will receive
syslog data from all devices across the Internet.

If you are sending syslog data over UDP or plain TCP we recommend you place log agents
within the same LAN network if possible so that syslog data is not sent over the Internet.
If you do transmit syslog data over UDP on the Internet, choose a non-standard port and/or
consider IP allow-listing restrictions.

If you are aggregating syslogs centrally consider hosting your log collector (e.g., Vector)
in a public cloud for reliability. For low scale environments you can use an inexpensive
Linux VMs such as AWS t3.nano (\~$4/month), GCP f1-micro (\~$4/month), or Azure B1s (\~$10/month).
Even fractional vCPU VMs will be able to handle thousands of syslog messages per second.

For higher-scale environments you could deploy a single VM with more resources, or consider
using network load balancers to distribute syslog data across multiple log collector VMs.

### Vector Syslog Configuration

We recommend using the Vector [socket](https://vector.dev/docs/reference/configuration/sources/socket/) source
to receive syslog data over UDP, TCP, or TLS. Even though Vector does have a syslog source, it will drop packets
that it cannot parse, and it does not support non-standard syslog formats. It is simpler to use the socket source
to receive unparsed syslog data, forward that data to SparkLogs, and let AutoExtract automatically parse
the syslog data as appropriate.

If you are transmitting syslog over TCP or TLS there are different [framing options](https://vector.dev/docs/reference/configuration/sources/socket/#framing.method), which determines how to tell
where each individual syslog message ends. The most robust method is [octet counting](https://datatracker.ietf.org/doc/html/rfc6587#section-3.4.1)
which prepends the length of each syslog message before the message itself. Another common method is the so-called
[non-transparent framing](https://datatracker.ietf.org/doc/html/rfc6587#section-3.4.2) (as defined in RFC6587) which ends each message with
a newline character. This naturally limits each syslog message to a single line. We recommend selecting the octet
counting framing method if supported by your syslog source.

You may need to setup multiple vector sources on different ports and protocols if you have syslog sources that
support different transmission protocols (UDP, TCP, or TLS) and framing methods (octet counting or non-transparent).

Here is an example Vector configuration that receives syslog data over UDP on port 9139:

```yaml
sources:
  socket_udp_input:
    type: "socket"
    address: "0.0.0.0:9139"
    mode: udp
```

Use this vector source as part of your overall Vector config file, as documented [here](https://sparklogs.com/docs/ingest/tools/vector.md).

## Syslog Format Reference

All the common syslog standard formats are supported, as well as many proprietary formats.
See how SparkLogs will parse using AutoExtract on your own syslog messages here:

```text
<38>Apr  9 18:37:24 subdomain.domain.tld sshd[1234]: User login successful user="root"
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### Core Syslog Format

The most basic form of a syslog message begins `<PRIORITY>` where `PRIORITY` is an integer (0..191).
`PRIORITY` encodes a facility level (0..23) and severity level (0..7):

```text
FACILITY = PRIORITY / 8
SEVERITY = PRIORITY % 8
```

For example, a PRIORITY value of `<134>` indicates a facility of `16` (local use 0) and severity of `6` (informational).

Syslog severity levels are mapped to the appropriate SparkLogs severity and are defined as follows:

| Syslog Severity | Mapped SparkLogs Severity | Description |
| :- | :- | :- |
| 0 | Emergency (24) | System is unusable |
| 1 | Alert (22) | Action must be taken immediately |
| 2 | Critical (20) | Critical conditions |
| 3 | Error (17) | Error conditions |
| 4 | Warning (13) | Warning conditions |
| 5 | Notice (11) | Normal but significant condition |
| 6 | Informational (9) | Informational messages |
| 7 | Debug (5) | Debug-level messages |

SparkLogs uses the standard syslog facility levels, which are defined as follows:

| Facility Level | Description |
| :- | :- |
| 0 | Kernel messages |
| 1 | User-level messages |
| 2 | Mail system |
| 3 | System daemons |
| 4 | Security/authorization messages |
| 5 | Syslog messages |
| 6 | Line printer subsystem |
| 7 | Network news subsystem |
| 8 | UUCP subsystem |
| 9 | Clock daemon (cron) |
| 10 | Security/authorization messages (private) |
| 11 | FTP daemon |
| 12 | NTP subsystem |
| 13 | Audit logs |
| 14 | Logged alerts |
| 15 | Clock daemon |
| 16-23 | Local use 0-7 |

### RFC3164

[RFC3164](https://datatracker.ietf.org/doc/html/rfc3164) documented the many variations of syslog messages in use at the time and is only loosely followed.
The format takes the general shape of one of the following:

```text
<PRIORITY>TIMESTAMP HOSTNAME TAG: MESSAGE
<PRIORITY>TIMESTAMP HOSTNAME TAG[PID]: MESSAGE
```

`TIMESTAMP` is canonically `MMM DD HH:MM:SS` (e.g., `Apr 9 18:37:00`) but there are countless variations,
some of which include the year. Many vendors do not include a `:` after the `TAG`. The RFC dictates there
should be no whitespace before the TIMESTAMP but many devices do not follow this.

For this format, SparkLogs captures the `PRIORITY` into the standard fields `severity` and `facility`,
`TIMESTAMP` as the standard `timestamp` field, `HOSTNAME` as the standard `source` field, and `TAG` as the
standard `app` field. If `PID` is present it is captured in the `proc_id` field.

### RFC5424

[RFC5424](https://datatracker.ietf.org/doc/html/rfc5424) is a strict and structured syslog format
that also allows for arbitrary key-value pairs of data to be captured.
If your device supports this format it is recommended to use it. RFC5424 follows this format:

```text
<PRIORITY>VERSION TIMESTAMP HOSTNAME APPNAME PROC_ID MSG_ID STRUCTURED-DATA MESSAGE
```

Key points:

- `VERSION` should be `1`. Timestamp should be RFC3339 format (e.g., `2023-04-09T18:37:00Z`).
- `HOSTNAME` is the hostname or IP of the device that generated the message or `-` (none): stored into the `source` field.
- `APPNAME` is the name of the application or service that generated the message or `-` (none): stored into the `service` field.
- `PROC_ID` is the process ID of the application or `-` (none).
- `MSG_ID` is a unique identifier for the message or `-` (none).
- `STRUCTURED-DATA` is either `-` (none) or a series of named lists of key-value pairs each enclosed in brackets:

```text
[SD-ID foo="bar" f2="hello world" ...]
```

`SD-ID` is a unique identifier for the structured data and is either a standard name
(`timeQuality`, `origin`, or `meta`) or is a custom ID followed by `@ENTERPRISE_NUMBER` where
`ENTERPRISE_NUMBER` is an integer that uniquely identifies the vendor.

In each bracketed section, after the `SD-ID` you can have zero or more key-value pairs. If there are zero pairs,
then it is considered as if the value named by `SD-ID` for that section is `true`.
The key-value pairs are separated by spaces and the values must be quoted.

Field values are captured similar to RFC3164, and `MSG_ID` is additionally captured into both the `msg_id` field
and forms the start to the [AutoClassify category](https://sparklogs.com/docs/ingest/autoclassify.md).
Additionally, structured data is captured into custom fields, with each SD-ID forming a top-level field that is
either set to `true` (for zero key-value pairs) or an object with each of the key-value pairs.

Example RFC5424 with structured data:

```text
<9>1 2026-04-09T18:37:24.000Z myhost appname 1234 ID13 [exampleSDID@32473 field="3" source="Application"][sdidNoFields@1234][origin ip="192.168.0.1" ip="192.168.1.1"] example message hello=world
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### Cisco

Syslog messages generated by Cisco devices are surprisingly varied in format. Almost every device type sends
syslog data in a non-standard format, and it may or may not send the `<PRIORITY>` syslog header.
Messages generally follow the shape:

```text
<PRIORITY>SEQNO: HOSTNAME: TIMESTAMP: %TEXTFACILITY-SEVERITY-MNEMONIC: MESSAGE
```

Key points:

- `SEQNO` is a message sequence number that may not be present.
- `HOSTNAME` is the hostname or IP of the originating device and may not be present.
- `TIMESTAMP` is the message time, and may be preceded by a `.` or `*` if the device's clock is unsynchronized.
- `TEXTFACILITY` is a text description of the device module that generated the message.
- `SEVERITY` is the syslog severity level (0..7) and will be mapped to a SparkLogs severity level.
- `MNEMONIC` is a message ID that should be unique for the given `TEXTFACILITY`.

Even across the many varied formats most Cisco message fields will be recognized and captured automatically,
especially `TIMESTAMP`, `TEXTFACILITY` (into the `net_facility` field), `SEVERITY` (into `net_severity` and
mapped to `severity` field if `PRIORITY` field is missing), and `MNEMONIC` (into `msg_id` field).

Check the Cisco syslog configuration manual for your device. Enable extended timestamps (to include the full year)
when possible, and also prefer to send messages with the `<PRIORITY>` header and include the hostname. Select
RFC5424 or RFC3164 format if available.

Example Cisco syslog message:

```text
<134>: *Apr  9 2026 18:37:24.123: %ASA-6-305012: Teardown dynamic UDP translation from INSIDE:192.168.1.10/1234 to OUTSIDE:10.0.0.100/1234 duration 0:00:05
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### Juniper

[Juniper](https://www.juniper.net/) network appliances send syslog data in RFC3164 format.

Example Juniper syslog message:

```text
<134>Apr  9 18:37:24 Junos RT_FLOW: RT_FLOW_SESSION_CREATE: session created...
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### SonicWall

[SonicWall](https://www.sonicwall.com/) network appliances can be [configured](https://www.sonicwall.com/support/technical-documentation/docs/sonicos-7-1-device_log/Content/Logs_Syslog/logs-syslog-settings.htm) to send syslog messages via UDP.
We recommend selecting the default syslog format. Messages have the basic `<PRIORITY>` header
and then follow a proprietary format with key-value pairs with mixed types. These will
automatically be captured by AutoExtract.

You can optionally configure SonicWall to include a custom `id` field value. If this is configured,
it will use it as a hostname and store it to the `source` standard field. If that is not available,
it will then look for the `sn` field (serial number) and then fallback to the `fw` field (IP address
of the firewall).

Example SonicWall syslog message:

```text
<134> id=somecustomer sn=ABCDEF123456 time="2026-04-09 18:37:24" fw=99.111.222.123 msg="Connection Closed hello=world" src=11.12.33.44:60321 srcMac=11:22:33:aa:bb:cc dst=8.8.8.8:53 dstMac=01:02:03:f1:f2:f3 proto=udp/dns app=
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### WatchGuard

[WatchGuard](https://watchguard.com/) network appliances can be [configured](https://www.watchguard.com/help/docs/help-center/en-us/Content/en-US/Fireware/logging/send_logs_to_syslog_c.html) to send syslog data.
Choose the "syslog" format so that it will send data following RFC3164.

Example WatchGuard syslog message:

```text
<134>Apr  9 18:37:24 myhost Member2 Allow 192.168.228.202 10.0.1.1 webcache/tcp 42973 8080 3-Trusted 1-WCI Allowed 60 63 (Outgoing-proxy-00) proc_id="firewall" rc="100" src_ip_nat="69.164.168.163" tcp_info="offset 10 S 2982213793 win 2105" msg_id="3000-0148"
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

### Fortinet

[Fortinet](https://www.fortinet.com/) network appliances send a core syslog header `<PRIORITY>` followed by a proprietary format with key/value pairs.
Also, you should [configure the device](https://docs.fortinet.com/document/fortiweb/7.6.1/cli-reference/172335/log-syslogd) to send the `hostname` field as that is not on by default. For example:

```text
config log syslogd
  set status enable
  set severity information
  set policy "my-syslog-policy"
  config custom-field
    edit 1
      set name hostname
      set value $hostname
end
```

Example Fortinet syslog message:

```text
<134>date=2019-05-13 time=11:20:54 logid="0100032001" type="event" subtype="system" level="information" hostname="mydevice" vd="vdom1" eventtime=1557771654587081441 logdesc="Admin login successful" sn="1557771654" user="admin" ui="ssh(172.16.200.254)" method="ssh" srcip=172.16.200.254 dstip=172.16.200.2 action="login" status="success" reason="none" profile="super_admin" msg="Administrator admin logged in successfully from ssh(172.16.200.254)"
```

[Try this sample on the page](https://sparklogs.com/docs/ingest/data-sources/syslog)

---

Source: <https://sparklogs.com/docs/ingest/data-sources/syslog>
All SparkLogs documentation in Markdown: <https://sparklogs.com/llms.txt>
