Overriding defaults metadata with the command-line?

[iandol]

We have this in the docs:

Note that, where command-line arguments may be repeated (--metadata-file, --css, --include-in-header, --include-before-body, --include-after-body, --variable, --metadata, --syntax-definition), the values specified on the command line will combine with values specified in the defaults file, rather than replacing them.

So given this test.yaml defaults file:

from: markdown
to: latex
metadata:
  fontsize: 12pt
  colorlinks: true

...this test.md Markdown...

Test content.

This output (only the bits of relevant LaTeX are shown):

pandoc -s -d ./test.yaml test.md

…
\documentclass[
  12pt,
]{article}
…
\hypersetup{
  colorlinks=true,

If we override the defaults metadata with command-line values we get this:

pandoc -s -d ./test.yaml -M fontsize=14pt -M colorlinks=false test.md

…
\documentclass[
  12pt14pt,
]{article}
…
\hypersetup{
  colorlinks=true,

The colorlinks boolean is not overridden (possibly it becomes a string truefalse?) and the fontsize becomes 12pt14pt, i.e. merged. It seems that this behaviour is not ideal for metadata / variables as we would want to merge the keys but not merge the value for the same key? Using variables in the YAML and -V command-line doesn't change this. What is the benefit of merging values for metadata/variable items?

---

note: if the markdown file itself has the metadata:

---
colorlinks: true
fontsize: 12pt
---

Test Content.

Then overriding from the command-line works fine:

pandoc -s -t latex -M fontsize=14pt -M colorlinks=false test.md

\documentclass[
  14pt,
]{article}
…
\hypersetup{
  hidelinks

[jgm]

Note that what happens in these cases is that the value becomes a list (so you could iterate over it using $for(..) in a template. When you try to access it as a string, the values are concatenated as you see.

Why this feature? I am not sure I remember all the reasons, to be honest. But here are two relevant facts:

1. Defaults files work, as much as possible, just like more values specified on the command line. So if -M foo=1 -M foo=2 makes foo a list [1,2], then you should see the same behavior if you have foo: 1 in the defaults file and -M foo=2 on the command line.
2. There really isn't a way to specify a list value for metadata field or variable on the command line, other than this.

[iandol]

Thanks @jgm. Indeed this is a somewhat thorny issue.

1. There are use cases for both concatenation and replacement. I wonder if using some sort of marker could identify an optional replacement, keeping the current concatenation as the default. For example -M foo=!true would replace foo rather than concatenate?
2. I suspect the use case for concatenating booleans [true, false] is definitely not as common as toggling a feature from ON to OFF. So one option could be to change the default behaviour for booleans. Downside: this would be confusing (that concat vs. replacement depends on the item type) and hard to clearly document, sort of magic behaviour.
3. Another possibility: the template engine could allow to specify only one item of a list, for example to specify the last value if it is a list or if it isn't a list the current value: $if(foo)$ $foo[-1]$ $endif$
   …now if foo was overridden on the command-line only that last item in the list would be used. We often use booleans themselves to toggle and if, perhaps $if(foo[-1])$ to use a boolean or if it is a list the last item. I suspect this is a lot of extra complexity for the template engine.