Substitute string processor
The substitute_string processor matches a key’s value against a regular expression and replaces all matches with a replacement string.
Configuration
The following table describes the options you can use to configure the substitute_string processor.
| Option | Required | Type | Description |
|---|---|---|---|
| entries | Yes | List | List of entries. Valid values are source, from, and to. |
| source | N/A | N/A | The key to modify. |
| from | N/A | N/A | The Regex String to be replaced. Special regex characters such as [ and ] must be escaped using \\ when using double quotes and \ when using single quotes. See Java Patterns for more information. |
| to | N/A | N/A | The String to be substituted for each match of from. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- substitute_string:
entries:
- source: "message"
from: ":"
to: "-"
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path of the file source in your pipeline.yaml file with your file path. For more detailed information, see Configuring OpenSearch Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"message": "ab:cd:ab:cd"}
After you run Data Prepper, the source is converted to the following format:
{"message": "ab-cd-ab-cd"}
from defines which string is replaced, and to defines the string that replaces the from string. In the preceding example, string ab:cd:ab:cd becomes ab-cd-ab-cd. If the from regex string does not return a match, the key is returned without any changes.