failed to allocate string buffer exceeds maximum length on a book project when embedding resources

[cderv]

Originally posted by @prs513rosewood in #2462

Sorry to comment on a closed issue (EDIT: #2462), but I'm also encountering this problem. On my machine I had to increase the heap limit past the 8192 default introduced in recent quarto versions. However I also run a quarto render on a gitlab.com runner and the job gets killed, likely because of a memory overrun. Increasing the heap limit does not help there (this used to build successfully six months ago with quarto 1.3.450).

The repository in question is https://gitlab.com/prs513rosewood/projets-numeriques-discrets, here are the steps to reproduce:

git clone https://gitlab.com/prs513rosewood/projets-numeriques-discrets.git
cd projets-numeriques-discrets/
python3 -m venv venv && source venv/bin/activate && pip install -r requirements.txt
quarto render book
# I run with the execute flag, but the error is the same

Here's the output of quarto check:

Quarto 1.6.32
[✓] Checking environment information...
      Quarto cache location: /home/frerot/.cache/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.4.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.46.3: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.6.32
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      Chromium: 869685
      TinyTeX: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /opt/texlive/bin/x86_64-linux
      Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.11.10
      Path: /home/frerot/Documents/repos/test/venv/bin/python3
      Jupyter: 5.7.2
      Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

And here is the failed output (with QUARTO_PRINT_STACK=true):

[ 1/18] index.qmd
[WARNING] Could not fetch resource ./equations_de_mouvement.html
[ 2/18] equations_de_mouvement.ipynb
[WARNING] Could not fetch resource ./interactions.html
[ 3/18] interactions.ipynb
[WARNING] Could not fetch resource ./lj.html
[ 4/18] lj.ipynb
[WARNING] Could not fetch resource ./sph.html
[ 5/18] sph.ipynb
[WARNING] Could not fetch resource ./prep.html
[ 6/18] prep.ipynb
[WARNING] Could not fetch resource ./balistique.html
[ 7/18] balistique.ipynb
[WARNING] Could not fetch resource ./collisions_entre_particules.html
[ 8/18] collisions_entre_particules.ipynb
[WARNING] Could not fetch resource ./lj_tp.html
[ 9/18] lj_tp.ipynb
[WARNING] Could not fetch resource ./thermostat.html
[10/18] thermostat.ipynb
[WARNING] Could not fetch resource ./truss.html
[11/18] truss.ipynb
[WARNING] Could not fetch resource ./plaque.html
[12/18] plaque.qmd
[WARNING] Could not fetch resource ./fissure.html
[13/18] fissure.qmd
[WARNING] Could not fetch resource ./granulaire.html
[14/18] granulaire.qmd
[WARNING] Could not fetch resource ./surface.html
[15/18] surface.qmd
[WARNING] Could not fetch resource ./mosh_pit.html
[16/18] mosh_pit.qmd
[WARNING] Could not fetch resource ./fluides.html
[17/18] fluides.ipynb
[18/18] report.qmd

WARN: lj.html: Unable to resolve crossref @fig-lj-energie
WARN: lj.html: Unable to resolve crossref @fig-lj-energie
WARN: lj.html: Unable to resolve crossref @fig-lj-force
WARN: lj.html: Unable to resolve crossref @fig-lj-cut
WARN: lj.html: Unable to resolve crossref @fig-lj-cut
WARN: lj_tp.html: Unable to resolve crossref @fig-lattice
WARN: truss.html: Unable to resolve crossref @fig-truss

<--- Last few GCs --->

[11478:0x5557c8a15000]   115406 ms: Mark-Compact 8255.6 (8260.8) -> 8255.6 (8260.0) MB, pooled: 6 MB, 31.23 / 0.00 ms  (average mu = 0.859, current mu = 0.429) allocation failure; scavenge might not succeed


<--- JS stacktrace --->



#
# Fatal JavaScript out of memory: Reached heap limit
#
==== C stack trace ===============================

    /opt/quarto/bin/tools/x86_64/deno(+0x2d8b133) [0x5557c2ca5133]
    /opt/quarto/bin/tools/x86_64/deno(+0x2d8a9fb) [0x5557c2ca49fb]
    /opt/quarto/bin/tools/x86_64/deno(+0x2d85f18) [0x5557c2c9ff18]
    /opt/quarto/bin/tools/x86_64/deno(+0x2ddc1ac) [0x5557c2cf61ac]
    /opt/quarto/bin/tools/x86_64/deno(+0x2f90467) [0x5557c2eaa467]
    /opt/quarto/bin/tools/x86_64/deno(+0x2f8e613) [0x5557c2ea8613]
    /opt/quarto/bin/tools/x86_64/deno(+0x2f85211) [0x5557c2e9f211]
    /opt/quarto/bin/tools/x86_64/deno(+0x2f85ce2) [0x5557c2e9fce2]
    /opt/quarto/bin/tools/x86_64/deno(+0x2f64278) [0x5557c2e7e278]
    /opt/quarto/bin/tools/x86_64/deno(+0x30aae11) [0x5557c2fc4e11]
    /opt/quarto/bin/tools/x86_64/deno(+0x30ae780) [0x5557c2fc8780]
    /opt/quarto/bin/tools/x86_64/deno(+0x30ae7a6) [0x5557c2fc87a6]
    /opt/quarto/bin/tools/x86_64/deno(+0x30ae7a6) [0x5557c2fc87a6]
    /opt/quarto/bin/tools/x86_64/deno(+0x30ae7a6) [0x5557c2fc87a6]
    /opt/quarto/bin/tools/x86_64/deno(+0x30ae7a6) [0x5557c2fc87a6]
    /opt/quarto/bin/tools/x86_64/deno(+0x30aaa2f) [0x5557c2fc4a2f]
    /opt/quarto/bin/tools/x86_64/deno(+0x30aa857) [0x5557c2fc4857]
    /opt/quarto/bin/tools/x86_64/deno(+0x2e222bf) [0x5557c2d3c2bf]
    [0x55576412dc76]
/opt/quarto/bin/quarto: line 192: 11478 Trace/breakpoint trap   (core dumped) "${QUARTO_DENO}" ${QUARTO_ACTION} ${QUARTO_DENO_OPTIONS} ${QUARTO_DENO_EXTRA_OPTIONS} "${QUARTO_IMPORT_MAP_ARG}" "${QUARTO_TARGET}" "$@"

And without QUARTO_PRINT_STACK=true:

[ 1/18] index.qmd
[ 2/18] equations_de_mouvement.ipynb
[ 3/18] interactions.ipynb
[ 4/18] lj.ipynb
[ 5/18] sph.ipynb
[ 6/18] prep.ipynb
[ 7/18] balistique.ipynb
[ 8/18] collisions_entre_particules.ipynb
[ 9/18] lj_tp.ipynb
[10/18] thermostat.ipynb
[WARNING] Could not fetch resource ./truss.html
[11/18] truss.ipynb
[WARNING] Could not fetch resource ./plaque.html
[12/18] plaque.qmd
[WARNING] Could not fetch resource ./fissure.html
[13/18] fissure.qmd
[WARNING] Could not fetch resource ./granulaire.html
[14/18] granulaire.qmd
[WARNING] Could not fetch resource ./surface.html
[15/18] surface.qmd
[WARNING] Could not fetch resource ./mosh_pit.html
[16/18] mosh_pit.qmd
[WARNING] Could not fetch resource ./fluides.html
[17/18] fluides.ipynb
[18/18] report.qmd

ERROR: TypeError: failed to allocate string; buffer exceeds maximum length
Path: /home/frerot/Documents/repos/test/book/_book/lj_tp.html

Stack trace:
Path: /home/frerot/Documents/repos/test/book/_book/lj_tp.html
    at readTextFileSync (ext:deno_fs/30_fs.js:864:10)
    at Object.Deno.readTextFileSync (file:///opt/quarto/bin/quarto.js:5124:25)
    at file:///opt/quarto/bin/quarto.js:98911:31
    at Array.map (<anonymous>)
    at websiteOutputFiles (file:///opt/quarto/bin/quarto.js:98910:8)
    at Object.bookPostRender [as postRender] (file:///opt/quarto/bin/quarto.js:100128:26)
    at renderProject (file:///opt/quarto/bin/quarto.js:85784:28)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async Command.actionHandler (file:///opt/quarto/bin/quarto.js:90187:32)
    at async Command.execute (file:///opt/quarto/bin/quarto.js:8086:13)

Thanks for looking at this, the issue is preventing me from updating my class material.

[prs513rosewood]

I did some further testing and upon seeing this comment I fixed the problem by setting embedded-resources to false. I hardly remember why I set it to true in the first place.

Hopefully my repository can help debug memory issues though. The book/ folder is only 42M including figures.

EDIT: commit hash with memory error is b87332a03a7e2136488797039cbddecbf1060cb8

[mcanouil]

I believe you meant embed-resources.
It seems to be another instance of this option being used where it should not (Book/Website projects).

[prs513rosewood]

Yeah sorry for the inconvenience, I somehow missed this when researching the issue before submitting the report.

[mcanouil]

Sorry, I meant, there was some internal discussions about this.
The option is not really meant to be used for multi documents projects such as Book/Website, but there is currently nothing to prevent this, thus a recurring issue. (I think there is an open issue about this but can't put my hands on it)

[cderv]

The option is not really meant to be used for multi documents projects such as Book/Website, but there is currently nothing to prevent this, thus a recurring issue. (I think there is an open issue about this but can't put my hands on it)

Related issue then

• quarto render fails when enabling self-contained or embed-resources in yaml #11206