How to use "--extract-media" and preserve regular (non-SHA1) filenames?

[jsx97]

I try to download an HTML page from the internet as Markdown and with all the linked images. Here is what I use currently for this:

pandoc http://foo.bar/baz.html -o baz.md --wrap=none --extract-media="./images"

The problem is that the images are saved with SHA1-based filenames, and even though I've read the user's guide,

--extract-media=DIR

Extract images and other media contained in or linked from the source document to the path DIR, creating it if necessary, and adjust the images references in the document so they point to the extracted files. Media are downloaded, read from the file system, or extracted from a binary container (e.g. docx), as needed. The original file paths are used if they are relative paths not containing ... Otherwise filenames are constructed from the SHA1 hash of the contents.

I still don't understand how to preserve regular filenames. What exactly should I do to preserve regular filenames, like foo.png, instead of having SHA1-based ones, like 6f10f22599031f605714f6b3f1153b5c12d96084.png?

[jgm]

See commit eddedbf
I am guessing that on the HTML page, some percent-encoding is used in the image paths. We use SHA1 in that case to avoid a security vulnerability.

[jsx97]

Yes, you are correct, there is percent-encoding there. Actually I would prefer to reduce security temporary, because I trust this website and there are about 200 images there, but most probably this is not possible.

[tarleb]

You could use a Lua filter like the below.

local hex_to_char = function(x)
  return string.char(tonumber(x, 16))
end

local unescape = function(url)
  return url:gsub("%%(%x%x)", hex_to_char)
end

function Image (img)
  -- ignore local images
  if not img.src:match('^https://') then
    return nil
  end

  local mt, contents = pandoc.mediabag.fetch(img.src)
  img.src = pandoc.path.filename(unescape(img.src))
  pandoc.mediabag.insert(img.src, mt, contents)

  return img
end

[jsx97]

Tarleb, many thanks!

[jsx97]

Albert, hello! Maybe this is too much, but could you show how to adjust this filter so that it will handle images such as ../image.png as well, that is, to preserve the original filename of image if there is .. in its path?

[tarleb]

Changing the line if not img.src:match('^https://') then to if not img.src:match('^https://') or img.src:match '%.%.' then should do the trick.

[jsx97]

Hmm, no, this haven't helped. I have also tried to enclose '%.%.' in (), that is,

if not img.src:match('^https://') or img.src:match('%.%.') then

but this doesn't seem to make any difference.

[tarleb]

Ah yes, my bad. It should either be and not instead of or, or everything between not and then should be placed in parentheses.

[jsx97]

This works, many thanks! :)