LaTeX -> Markdown -> LaTeX roundtrip with several issues

[asmaier]

I converted the following test.tex file

\documentclass[12pt,a4paper,twoside]{book}
\usepackage[intlimits]{amsmath}	

% brackets
\newcommand{\lra}[1]{ \left( #1 \right) }
\newcommand{\lrb}[1]{ \left[ #1 \right] }
\newcommand{\lrc}[1]{ \left\{ #1 \right\} }

\begin{document}

Lorem ipsum dolor sit amet.

\begin{align}
a & = \lra{b + c} \label{eq2} \\
c & = \lrb{d + e} \label{eq3} \\
e & = \lrc{f + g} \label{eq4}
\end{align}

Refer to \eqref{eq2} - \eqref{eq4}.

\end{document}

via pandoc

pandoc --wrap=preserve -s -t markdown -f latex -o test.md test.tex

to markdown

Lorem ipsum dolor sit amet.

$$\begin{aligned}
a & =  \left( b + c \right)  \label{eq2} \\
c & =  \left[ d + e \right]  \label{eq3} \\
e & =  \left\{ f + g \right\}  \label{eq4}
\end{aligned}$$

Refer to [\[eq2\]](#eq2){reference-type="eqref" reference="eq2"} - [\[eq4\]](#eq4){reference-type="eqref" reference="eq4"}.

and back again via

pandoc --wrap=preserve -s -f markdown -t latex -o test_reverse.tex test.md 

to latex

...
Lorem ipsum dolor sit amet.

\[\begin{aligned}
a & =  \left( b + c \right)  \label{eq2} \\
c & =  \left[ d + e \right]  \label{eq3} \\
e & =  \left\{ f + g \right\}  \label{eq4}
\end{aligned}\]

Refer to \hyperref[eq2]{{[}eq2{]}} - \hyperref[eq4]{{[}eq4{]}}.

The first issue is that the generated latex fails compiling

$ tectonic test_reverse.tex
Package amsmath Error: Multiple \label's: label 'eq2' will be lost.

The reason seems to be that the conversion of \begin{align} -> $$\begin{aligned} is not reverted when converting from Markdown to LaTeX.
I also don't understand is, why the \eqref commands are not converted back again to \eqref, although the information is stored in markdown as reference-type="eqref". So the information which type of reference it was wasn't lost after converting to markdown. Still pandoc does seem to ignore that, when converting from Markdown to LaTeX again.
And finally it seems strange that the latex macros get expanded when converting to markdown, although this is not necessary, as the latex_macros extension is activated by default for markdown. So the latex macros are lost in the roundtrip.

Are there deeper reasons why a roundtrip for such a simple LaTeX document is not possible with pandoc?

[jgm]

the conversion of \begin{align} -> $$\begin{aligned} is not reverted when converting from Markdown to LaTeX.

Why should it be? Pandoc doesn't know the past history of this conversion. It doesn't know that $$\begin{aligned} was originally \begin{align}. It could have originally been $$\begin{aligned}.

For the references: I think we can do better here. Makes sense to open an issue for this.

You already submitted an issue about the macros, so I've tried to answer your question there.

[asmaier]

the conversion of \begin{align} -> $$\begin{aligned} is not reverted when converting from Markdown to LaTeX.

Why should it be? Pandoc doesn't know the past history of this conversion. It doesn't know that $$\begin{aligned} was originally \begin{align}. It could have originally been $$\begin{aligned}.

But pandoc also doesn't know the history of the LaTeX document. Still it decides to convert \begin{align} -> $$\begin{aligned}. The argument that pandoc doesn't know the history of the document shouldn't stop it from making a reasonable conversion. And pandoc does
know that it usually converts \begin{align} -> $$\begin{aligned} (the ladder being a quite unusual expression for LaTeX), so I believe it would be reasonable to convert $$\begin{aligned} -> \begin{align}when converting from Markdown to LaTeX.

This just follows from the idea of a successful https://en.wikipedia.org/wiki/Round-trip_format_conversion:

A successful round-trip consists of converting a document in format A (docA) to one in format B (docB) and then back again to format A (docA′). If docA and docA′ are identical then there has been no information loss and the round-trip has been successful.

[asmaier]

So we drop down to aligned, which is the "inside math mode" equivalent of align.

Unfortunately aligned and align are not exactly equivalent. aligned is used if you want one equation number for several lines. align can be used to align and label several equations (https://tex.stackexchange.com/questions/103891/multiple-labels-in-aligned-environment). That is the reason why converting the example above back to latex leads to an error. To avoid such an error aligned should be converted back to align in LaTeX (at least if there is more than one label inside).

[jgm]

Unfortunately aligned and align are not exactly equivalent.

I'm well aware of that! It's a lossy conversion because we lose the equation number. But we can't put an aligned environment inside math mode, so that's the best we can do.

To avoid such an error aligned should be converted back to align

No, because we don't know if the aligned in the source document was down-converted from an aligned in an earlier source, or was there in the original. We don't have this information about the history.

[asmaier]

I'm well aware of that! It's a lossy conversion because we lose the equation number. But we can't put an aligned environment inside math mode, so that's the best we can do.

Maybe there is a better way: Would it be possible to convert a single align environment with several labels inside not into one aligned environment, but into several ones? So

\begin{align}
a & = \lra{b + c} \label{eq2} \\
c & = \lrb{d + e} \label{eq3} \\
e & = \lrc{f + g} \label{eq4}
\end{align}

becomes

$$\begin{aligned}
a & = \lra{b + c} \label{eq2}
\end{aligned}$$
$$\begin{aligned}
c & = \lrb{d + e} \label{eq3}
\end{aligned}$$
$$\begin{aligned}
e & = \lrc{f + g} \label{eq4}
\end{aligned}$$

By doing this one would loose the alignment between the equations, but one would preserve the labels. And it also would have the advantage when converting this back to latex, one wouldn't get the error described above.

[jgm]

Losing the alignment isn't nothing. And why would you want a label in aligned, which is an unnumbered equation? There is nothing to reference there.

[asmaier]

As shown in my example at the beginning of this thread, the \label end up in the \aligned environment, because the \align environment from LaTeX is converted into an $$\aligned environment in markdown. So originally it was an equations environment with numbers and labels and references pointing to it.

But the labels are not deleted by the conversion process from LaTeX to markdown. The problem is now, that if one is converting back to LaTeX the multiple labels in the aligned environment cause LaTeX to fail with an error, because it is not allowed to have multiple labels inside a single aligned environment.

So I think pandoc should prevent that multiple '\label' end up in a single \aligned environment, because that is actually not legal. One could either simply delete all the \label in the conversion process. However what should one do with the \ref and \eqref pointing to that labels? Therefor another solution could be, to split an \aligned environment with multiple labels into several \aligned environment with a single label each.

[jgm]

The treatment of math environments like align has long been a rough edge.

I note that texmath (our math conversion library) will convert align environments to other formats. MathJaX also handles them, even between $$ (but not KaTeX).

So, one possibility, I suppose, would be to allow these special math environments (align, equation, etc.) as contents of pandoc Math elements, and move any necessary special treatment to the writers. (E.g. the LaTeX writer would need to render them as raw TeX rather than between $$...$$``, and similarly the HTML writer with KaTeX option would have to downgrade them to aligned`, etc.) This would give a nicer round trip to LaTeX.

[asmaier]

and move any necessary special treatment to the writers

That sounds like a promising idea to make the treatment of math environments more flexible. I like it.

[jgm]

I'll open an issue to track the suggestion.