What Is Log Parsing?

Last verified on Nov 8, 2026.

Log parsing is extracting fields from a raw log line so the values can be used on their own. A line such as an access log entry becomes a record with a client, a method, a path, a status, and a size. NIST SP 800-92 defines it as "extracting data from a log so that the parsed values can be used as input for another logging process." Filtering, alerting, and charting all depend on it.

Three Ways to Parse

JSON. If your application writes structured logs, parsing is a single call. Fluent Bit's JSON parser turns {"key1": 12345, "key2": "abc", "time": "2006-07-28T13:22:04Z"} into a record and converts the time key into a timestamp. The OpenTelemetry filelog receiver offers a json_parser operator for the same job. From a shell:

echo '{"level":"error","msg":"payment failed","order_id":"ord_8123"}' \
  | jq -r '[.level, .order_id] | @tsv'

Regular expressions. For text you cannot change, name each part with a capture group. This Python example parses the Apache access log line used in Fluent Bit's documentation:

import re

line = '192.168.2.20 - - [29/Jul/2015:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 3395'
rx = re.compile(
    r'^(?P<host>\S+) \S+ (?P<user>\S+) \[(?P<time>[^\]]+)\] '
    r'"(?P<method>\S+) (?P<path>\S+) (?P<proto>[^"]+)" (?P<code>\d{3}) (?P<size>\d+|-)$'
)
print(rx.match(line).groupdict())

It prints {'host': '192.168.2.20', 'user': '-', 'time': '29/Jul/2015:10:27:10 -0300', 'method': 'GET', 'path': '/cgi-bin/try/', 'proto': 'HTTP/1.0', 'code': '200', 'size': '3395'}. Fluent Bit's regex parser works the same way with Onigmo named groups, and its docs warn that complicated expressions can trigger regular expression denial of service because the engine backtracks.

Grok. Logstash's grok filter names reusable pieces in the form %{SYNTAX:SEMANTIC}, where the syntax is what to match and the semantic is the field name. For the line 55.3.244.1 GET /index.html 15824 0.043, the documentation gives:

%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}

Every field is a string unless you add a conversion such as %{NUMBER:bytes:int}. Grok is built on regular expressions, so any regex also works inside it. The same docs point to dissect as the delimiter-based alternative, which does not use regular expressions and works well when the line layout never varies.

Parsing Functions in Pipelines

Vector's Remap Language ships functions including parse_grok, parse_regex, parse_json, parse_syslog, and parse_nginx_log. Each is fallible, so you must handle a failed parse explicitly. That detail matters more than it looks: a parser that quietly drops lines it cannot match is how fields disappear from a dashboard.

What Goes Wrong

• Unmatched lines. A format change or an unexpected value makes the match fail. Count and review unparsed lines instead of discarding them. Field drift detection covers fields that appear, vanish, or change type.
• Wrong types. Parsed values are strings until converted. A status of "200" will not compare as a number.
• Inconsistent timestamps. NIST calls the fix normalization, which converts each field to one representation, with dates and times in a single format as the usual example.
• Slow expressions. Anchor your regex with ^ and $, and avoid nested wildcards.
• Multi-line events. Stack traces and some audit events span lines, and parsers that work one line at a time will split them.

For syslog lines, parse the priority and header first, then the message. Parsing usually happens in your log forwarding agent, close to the source, before logs reach log aggregation. Better still, avoid parsing by logging JSON from the start. For web servers, see nginx logs.

Parsing in Sazabi

Some Sazabi sources parse for you. The Linux auditd source, for example, turns audit fields such as auid, uid, exe, acct, addr, and res into attributes. Sazabi, an AI-native observability and security platform, then filters on those attributes during an investigation. Each audit line stays its own record, so multi-line events share an audit_serial but are not joined. See the auditd source docs and the Fluent Bit integration.