ottlfunc regex doesn't support double quotes

[Marty1703]

Component(s)

pkg/ottl

Describe the issue you're reporting

Using the replace_pattern ottl func, I am nog able to use a double quote within the regex param. Which i require to transform within my JSON body

I would like to use something like this:
- replace_pattern(attributes["Body"], (?U)name\":\"(.*)\", name\":\"**********\",)

But it gives me the error:
Error: invalid configuration: processors::transform/avg: unable to parse OTTL statement "replace_pattern(attributes[\"Body\"], (?U)name\\\":\\\"(.*)\\\", name\\\":\\\"**********\\\",)": statement has invalid syntax: 1:38: invalid input text "?U)name\\\":\\\"(.*)..." 2024/06/13 05:45:43 collector server run finished with error: invalid configuration: processors::transform/avg: unable to parse OTTL statement "replace_pattern(attributes[\"Body\"], (?U)name\\\":\\\"(.*)\\\", name\\\":\\\"**********\\\",)": statement has invalid syntax: 1:38: invalid input text "?U)name\\\":\\\"(.*)..."

[github-actions]

Pinging code owners:

• pkg/ottl: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

The code owners for pkg/ottl will have better insight than me here, but at a glance, the regex and replacement arguments in the replace_pattern function call need to be enclosed in double quotes as they're strings.

The following works for me:

- replace_pattern(attributes["Body"], "(?U)name\":\"(.*)\"", "name\":\"**********\"")

[TylerHelmuth]

(?U)name\":\"(.*)\", name\":\"**********\", should become "(?U)name\":\"(.*)\", name\":\"**********\","

[Marty1703]

Doing so, the Collector fails with error:

2024-06-17T08:02:14.204Z warn expandconverter@v0.101.0/expand.go:73 Variable substitution using $VAR will be deprecated in favor of ${VAR} and ${env:VAR}, please update $$ {"variable": "$"}
Error: invalid configuration: processors::transform/avg: unable to parse OTTL statement "replace_pattern(attributes[\"Body\"], \"(?U)name\\\":\\\"(.*)\\\", \"name\\\":\\\"**********\\\")": statement has invalid syntax: 1:64: invalid input text "\\\":\\\"**********\\..."
2024/06/17 08:02:14 collector server run finished with error: invalid configuration: processors::transform/avg: unable to parse OTTL statement "replace_pattern(attributes[\"Body\"], \"(?U)name\\\":\\\"(.*)\\\", \"name\\\":\\\"**********\\\")": statement has invalid syntax: 1:64: invalid input text "\\\":\\\"**********\\..."

[TylerHelmuth]

You'll need to escape * as well: replace_pattern(attributes["Body"], "(?U)name\":\"(.*)\", name\":\"\*\*\*\*\*\*\*\*\*\*\",").

I suggest testing that your regex is well formatted and then adding it to your config.

[Marty1703]

Hi @TylerHelmuth ,

This one is correct, so it has nothing to do with the * thats has to be escaped
- replace_pattern(attributes["Body"], "(?U)name(...)(.*),{1}", "name$$1**********\",")

We are using this image and it does the parsing, as far as i can see: otel/opentelemetry-collector-contrib:0.101.0

[TylerHelmuth]

I got mixed up on the number of parameters. The second parameter is a regex string. The third parameter is the replacement string, not regex, so you're right that nothing needs escaping.

OTTL requires string literals to be in double quoates (""). This statement compiles: replace_pattern(attributes["Body"], "(?U)name\":\"(.*)\"", "name\":\"**********\",")

[Marty1703]

It works, thanks @TylerHelmuth! Not sure how I missed this during trial and error.