Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preserve comments isolated in documents (and an error decrypting in edge cases) #936

Open
patricknelson opened this issue Sep 10, 2021 · 2 comments

Comments

@patricknelson
Copy link

patricknelson commented Sep 10, 2021

When testing comments at the top of YAML files in #757, I found that comments in YAML streams which are isolated into their own documents are coalesced into the adjacent document (but only if they are at the very top of the YAML stream). Also, I found a possible bug where sops will encrypt a document that it cannot decrypt later on whenever these isolated comments are located anywhere else in the YAML stream.

Coalescing separate documents into a single document:

#@data/values
---
foo: bar

Becomes:

#@data/values
foo: bar

Errors: Each of the following variants of the above YAML stream will encrypt without an error. However, they cannot be decrypted, resulting in the exact same error message:

Could not marshal tree: Error marshaling to yaml: yaml: expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS, but got document end

---
#@data/values
---
foo: bar
foo: bar
---
#@data/values
---
baz: qux
foo: bar
---
#@data/values

Note: While these examples utilize some code from ytt which handles comments semantically, it's not expected for sops to support ytt syntax per se. Rather, it's just a real-world example of the usefulness of having comments in this positions and isolated in this way, particularly since sops can be used to encrypt values that would be used in a templating engine like ytt (for annotations in a document containing sensitive data that are then interpreted by a parser).

@patricknelson patricknelson changed the title Preserve comments isolated isolated in documents (and an error decrypting in edge cases) Preserve comments isolated in documents (and an error decrypting in edge cases) Sep 10, 2021
@felixfontein
Copy link
Contributor

This are actually two problems:

  1. sops currently cannot distinguish between empty documents and empty top-level maps. So the empty document will probably decode as {} (with the comment ahead) if the invalid merging bug is fixed. (See Allow empty maps for yaml (#907) #908, Cannot output empty yaml file since sops-v3.7.0 #907.)
  2. The invalid merging bug comes from a bug in yaml.v3, for which I have a fix bug that fix is still waiting for a review ([v3] Improve empty document handling go-yaml/yaml#690).

@patricknelson
Copy link
Author

Since this is a little esoteric for me, can you interpret (or simplify) this for me a bit? I'm inferring that you're saying that a single comment in a document is "empty" so are you suggesting that there's no path to being able to ever encrypt/decrypt this symmetrically and cleanly?

#@data/values
---
foo: bar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants