Combining multiple metadata blocks

[gfelbing]

Hello,

I am using a shared metadata yaml for all my documents, which contains definitions, which I need in all of my documents, for example:

---
documentclass: article
header-includes:
  - \usepackage{tikz}

---

Now, I have some document, in which I want to overwrite/extend some values:

---
documentclass: IEEEtran
header-includes:
  - \IEEEspecialpapernotice{Some notice}

---

I would have expected, that the documentclass (since it is a single-value type) gets overridden and the header-includes (since it is an array) gets extended.
Instead, the values specified in my document got completely ignored.

I compiled the document with the following options:

pandoc --standalone --filter pandoc-citeproc -f markdown+yaml_metadata_block+example_lists+implicit_figures+citations -o ./subject/document.pdf ./pandoc/meta.yaml  ./subject/document.md

Is there a way to achieve my expected behaviour?

[jgm]

When a value is set multiple times in the same document, the
earlier one takes priority. See the MANUAL:

A document may contain multiple metadata blocks. The
metadata fields will be combined through a left-biased
union: if two metadata blocks attempt to set the same
field, the value from the first block will be taken.

Technical explanation for pandoc developers: This is because
the monoid instance for Meta (in Text.Pandoc.Definition)
uses Data.Map.union, which is left-biased. (Monoid append
is used for each new metadata block in the document; see the
end of yamlMetaBlock in the Markdown reader.)

So the solution for you would be to include the common
metadata after your document rather than before.

[gfelbing]

This would work for now.
For List-Typed metadata, you could apply Data.Map.unionWith curried with a (++), so arrays would get extendend.

[tarleb]

Here is an hacked-together prove of concept of how this could be done using lua filters (requires #3584). Would this be sufficient to close this issue?

--- Define ways in which metadata can be merged.
local merge_methods = {
  replace = function(v1, v2) return v2 or v1 end,
  keep = function(v1, v2) return v1 or v2 end,
  extendlist = function(v1, v2)
    local res
    if type(v1) == "table" and v1.tag == "MetaList" then
      res = v1
    else
      res = pandoc.MetaList{v1}
    end
    if type(v2) == "table" and v2.tag == "MetaList" then
      for i = 1, #v2 do
        res[#res + 1] = v2[i]
      end
    else
      res[#res + 1] = v2
    end
    return res
  end
}

--- Merge second metadata table into the first.
function merge_metadata(md1, md2, field_methods)
  for k, v in pairs(md2) do
    -- the default method is to overwrite the current value.
    local method = field_methods[k] or "replace"
    md1[k] = merge_methods[method](md1[k], md2[k])
  end
  return md1
end

return {
  {
    Meta = function(meta)
      local metafile = io.open('/home/albert/tmp/metadata-file.yaml', 'r')
      local content = metafile:read('*a')
      local doc = pandoc.read(content, "markdown")
      metafile:close()
      local field_methods = {
        author = "extendlist",
        title = "replace",
        date = "replace",
        classoptions = "keep",
      }
      return merge_metadata(meta, doc.meta, field_methods)
    end,
  }
}

[mb21]

With the coming release, you'll be able to use the --metadata-file option to achieve this use-case, since metadata values specified inside the document overwrite values specified with this option.

See #4604.