You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
use camino::{Utf8Path,Utf8PathBuf};use mupdf::error::Error;use mupdf::document::Document;use rayon::prelude::*;fnpdftotext(pdf:Utf8PathBuf) -> Result<String,Error>{let file = Document::open(pdf.as_str())?;letmut content = String::new();for page in file.pages()? {
content.push_str(&page?.to_text()?);}Ok(content)}fnmain(){// populate many pdf paths:let many_pdf_files:Vec<Utf8PathBuf> = vec![];// seg fault:let results:Vec<_> = many_pdf_files.into_par_iter().map(pdftotext).collect();}
pdftotext is called on many pdf files. Whenever I use rayon to parallelize the program, seg fault occurs. The more threads I use, the faster seg fault happens. The program won't crash if I simply iterate over all pdf files in a single thread (e.g., replacing into_par_iter() with into_iter()).
I try the main branch (a7e5f64) today. The seg fault issue disappear. But a new issue occurs. With mupdf-0.4.2 on crates.io, single threaded, it takes 310 seconds to convert 2000+ pdf files to text, 5 of them tagged as containing invalid utf-8. With the main branch, three parallel threads, it takes 450 seconds to convert the same collection of pdf files to text, 30 of them tagged as having the utf8 error. In conclusion, the conversion becomes slower and issues more utf-8 errors than before. I report this issue separately at #86.
I use
mupdf
like this:pdftotext
is called on many pdf files. Whenever I use rayon to parallelize the program, seg fault occurs. The more threads I use, the faster seg fault happens. The program won't crash if I simply iterate over all pdf files in a single thread (e.g., replacinginto_par_iter()
withinto_iter()
).I'm using
mupdf
version 0.4.2 from https://crates.io/crates/mupdf.The text was updated successfully, but these errors were encountered: