Link Search Menu Expand Document Documentation Menu
Documentation

parse_json

The parse_json processor parses JSON-formatted strings within an event, including nested fields. It can optionally use a JSON pointer to extract a specific part of the source JSON and add the extracted data to the event.

Configuration

You can configure the parse_json processor with the following options.

Option Required Type Description
source No String The field in the event that will be parsed. Default is message.
destination No String The destination field for the parsed JSON. Default is the root of the event. Cannot be "", /, or any white-space-only string.
pointer No String A JSON pointer (as defined by RFC 6901) to a specific field in the source JSON. If omitted, the entire source is parsed. If the pointer is invalid, the full source is parsed instead. When writing to the root destination, existing keys will be preserved unless overwritten.
parse_when No String A condition expression that determines when to parse the field. Accepts a string following the expression syntax.
overwrite_if_destination_exists No Boolean Whether to overwrite the destination field if it already exists. Default is true.
delete_source No Boolean Whether to delete the source field after parsing. Default is false.
tags_on_failure No String A list of tags to apply if parsing fails or an unexpected exception occurs.

Usage

To use the parse_json processor, add it to your pipeline.yaml configuration file:

parse-json-pipeline:
  source:
    ...
  ...
  processor:
    - parse_json:

All examples use the following JSON message for the event output:

{"outer_key": {"inner_key": "inner_value"}}

Basic example

The following example parses a JSON message field and flattens the data into the event. The original message from the example event remains, and the parsed content is added at the root level, as shown in the following output:

{
  "message": "{\"outer_key\": {\"inner_key\": \"inner_value\"}}",
  "outer_key": {
    "inner_key": "inner_value"
  }
}

Delete a source

If you want to remove the original field from the originating JSON message, use the delete_source option, as shown in the following example pipeline:

parse-json-pipeline:
  source:
    ...
  ...
  processor:
    - parse_json:
        delete_source: true

In the following event, the message field is parsed and removed, leaving only the structured output:

{
  "outer_key": {
    "inner_key": "inner_value"
  }
}

Example using a JSON pointer

You can use the pointer option to extract a specific nested field from the JSON data, as shown in the following example pipeline:

parse-json-pipeline:
  source:
    ...
  ...
  processor:
    - parse_json:
        pointer: "/outer_key/inner_key"

Only the value at the pointer path /outer_key/inner_key is extracted and added to the event. If you set destination, the extracted value will be added to that field instead:

{
  "message": "{\"outer_key\": {\"inner_key\": \"inner_value\"}}",
  "inner_key": "inner_value"
}
350 characters left

Have a question? .

Want to contribute? or .