Link Search Menu Expand Document Documentation Menu
Documentation

Parse Ion processor

The parse_ion processor parses Amazon Ion data.

Configuration

You can configure the parse_ion processor with the following options.

Option Required Type Description
source No String The field in the event that is parsed. Default value is message.
destination No String The destination field of the parsed JSON. Defaults to the root of the event. Cannot be "", /, or any white-space-only string because these are not valid event fields.
pointer No String A JSON pointer to the field to be parsed. There is no pointer by default, meaning that the entire source is parsed. The pointer can access JSON array indexes as well. If the JSON pointer is invalid, then the entire source data is parsed into the outgoing event. If the key that is pointed to already exists in the event and the destination is the root, then the pointer uses the entire path of the key.
parse_when No String Specifies under which conditions the processor should perform parsing. Default is no condition. Accepts an OpenSearch Data Prepper expression string following the expression syntax.
overwrite_if_destination_exists No Boolean Overwrites the destination if set to true. Set to false to prevent changing a destination value that exists. Defaults is true.
delete_source No Boolean If set to true, then the source field is deleted. Defaults is false.
tags_on_failure No String A list of strings specifying the tags to be set in the event that the processor fails or an unknown exception occurs during parsing.
handle_failed_events No String Determines how to handle events containing Ion processing errors. Valid values are skip (log the error and send the event downstream to the next processor) and skip_silently (send the event downstream to the next processor without logging the error). Default is skip.

Usage

The following examples show how to use the parse_ion processor in your pipeline.

Example: Minimum configuration

The following example shows the minimum configuration for the parse_ion processor:

parse-json-pipeline:
  source:
    stdin:
  processor:
    - parse_json:
        source: "my_ion"
  sink:
    - stdout:

When the input event contains the following data:

{"my_ion": "{ion_value1: \"hello\", ion_value2: \"world\"}"}

The processor parses the event into the following output:

{"ion_value1": "hello", "ion_value2" : "world"}

Metrics

The following table describes common Abstract processor metrics.

Metric name Type Description
recordsIn Counter Metric representing the ingress of records to a pipeline component.
recordsOut Counter Metric representing the egress of records from a pipeline component.
timeElapsed Timer Metric representing the time elapsed during execution of a pipeline component.

The parse_ion processor includes the following custom metrics.

Counter

  • parseErrors: The number of parse errors resulting from invalid Ion format in events. This indicates that the Ion format could not be parsed.
  • processingFailures: The number of processing failures that have occurred in the parse_ion processor. This indicates unexpected errors not related to invalid Ion format.
350 characters left

Have a question? .

Want to contribute? or .