Expand ${HOME}, ${USERDATA} and ${.} In all strings in defaults files

[bpj]

Proposed enhancement and the problem it solves

My request is to make the “variables”

• ${USERDATA}
• ${HOME}
• ${.}

expand to their respective paths in any string in a defaults file, so that you for example can have

variables:
  my_filter_data: '${USERDATA}/data/my-filter.json'

in a defaults file and have it appear expanded in PANDOC_WRITER_OPTIONS.variables.my_filter_data in Lua filters.

Currently I do fake this with the code below, but (a) it feels fragile, (b) it is a lot of boilerplate and (c) it does not work for ${.} anyway.¹

-- Make sure these variables exist and are indexable
local PANDOC_STATE = PANDOC_STATE or { }
local PANDOC_WRITER_OPTIONS = PANDOC_WRITER_OPTIONS or { }
local pandoc_vars = PANDOC_WRITER_OPTIONS.variables or { }

-- Get the directory paths
local user_dirs = {
  HOME = os.getenv('HOME'),
  USERDATA = PANDOC_STATE.user_data_dir
}

-- Check that the paths were set
for _, dir in ipairs{ 'HOME', 'USERDATA' } do
  if nil == user_dirs[dir] then
    error(tostring(dir) .. " directory variable not found")
  end
end

-- Set up error handling for table
setmetatable(user_dirs, {
  __index = function(tab, key)
    return error("Unknown user directory: " .. tostring(key))
  end
})

-- Define utility function
local expand_user_dir = function(s)
  return s:gsub('%$%{([_%w]+)%}', user_dirs)
end

local my_filter_data_path = expand_user_dir(pandoc_vars.my_filter_data or "")
if "" ~= my_filter_data_path then
  local custom_data = read_json_file(my_filter_data_path)
  for k, v in pairs(custom_data) do
    data[k] = v
  end
end

I wouldn’t mind if the code needed in every filter reading in files were reduced to the last block, mutatis mutandis.

Related

#10251

Footnotes

1. And the boilerplate includes the not shown read_json_file function, but that is another issue! ↩

[jgm]

This is probably a good idea. I limited it to paths because, who knows, one might want to include a code block with ${FOOBAR} in your defaults file. But this is probably a low enough probability thing that we can ignore it.

@tarleb thoughts?

[tarleb]

Would it make sense to limit the expansion to the start of metadata values?

Expanding all values everywhere seems a bit much.

[jgm]

Relevant older issues: #9173 #8774 (a dup?) #8024

[jgm]

The code for defaults file parsing is incredibly complicated and it will be hard to make a change like this without simplifying it first.

[bpj]

I think it is even reasonable to limit it to the start of strings without any newlines in them.