PDF document fails to compile when output file argument contains absolute path with tilde in Windows

[LDSamson]

The tilde abbreviation of a folder name can (apparently) happen automatically in Windows.

I noticed it because I created an absolute file path for the output PDF file in the temporary folder from windows, for use in unit tests.
The temporary folder file path will contain a tilde abbreviation if the Windows user name is long (for example, "C:/Users/Johnny with Long Name" will be abbreviated to "C:/Users/JOHNNY~1").

The problem only occurs with creating PDF documents, when attempting to convert the intermediate latex document to PDF.

See reprex below:

EDIT: fixed reprex

# This will error under windows:
local({
  folder_name <- "VERY_L~1" 
  temp_dir <- file.path(tempdir(), folder_name)
  dir.create(temp_dir)
  temp_rmd <- file.path(temp_dir, "test.Rmd")
  temp_report <- file.path(temp_dir, "report.pdf")
  on.exit(unlink(temp_dir, recursive = TRUE))
  writeLines("---\ntitle: 'Test report'\n---\n\ntest report", temp_rmd)
  rmarkdown::render(temp_rmd, output_file = temp_report)
})

# but this will not error:
local({
  folder_name <- "VERY_L~1" 
  temp_dir <- file.path(tempdir(), folder_name)
  dir.create(temp_dir)
  temp_rmd <- file.path(temp_dir, "test.Rmd")
  ## removed report absolute file path:
  temp_report <- "report.pdf"
  on.exit(unlink(temp_dir, recursive = TRUE))
  writeLines("---\ntitle: 'Test report'\n---\n\ntest report", temp_rmd)
  rmarkdown::render(temp_rmd, output_file = temp_report)
})

Output results:
Error: LaTeX failed to compile *****\AppData\Local\Temp\Rtmp2ttLaR/VERY_L~1/report.tex. See https://yihui.org/tinytex/r/#debugging for debugging tips.

tex log created:

This is pdfTeX, Version 3.141592653-2.6-1.40.26 (TeX Live 2024) (preloaded format=pdflatex 2024.5.3)  6 MAY 2024 12:41
entering extended mode
 restricted \write18 enabled.
 %&-line parsing enabled.
****AppData/Local/Temp/Rtmp2ttLaR/VERY_L~1/report.tex
! Emergency stop.
<to be read again> 
                   \protect 
<*> *****/AppData/Local/Temp/Rtmp2ttLaR/VERY_L~
                                                                                  1/report.tex 
Here is how much of TeX's memory you used:
 4 strings out of 476009
 111 string characters out of 5789395
 1925754 words of memory out of 5000000
 22279 multiletter control sequences out of 15000+600000
 558069 words of font info for 36 fonts, out of 8000000 for 9000
 14 hyphenation exceptions out of 8191
 2i,0n,0p,1b,6s stack positions out of 10000i,1000n,20000p,200000b,200000s

!  ==> Fatal error occurred, no output PDF file produced!

Session Info

R version 4.4.0 (2024-04-24 ucrt) Platform: x86_64-w64-mingw32/x64 Running under: Windows 11 x64 (build 22631) Locale: LC_COLLATE=English_Germany.utf8 LC_CTYPE=English_Germany.utf8 LC_MONETARY=English_Germany.utf8 LC_NUMERIC=C LC_TIME=English_Germany.utf8

Package version:
base64enc_0.1.3 bslib_0.7.0 cachem_1.0.8 cli_3.6.2 digest_0.6.35 evaluate_0.23
fastmap_1.1.1 fontawesome_0.5.2 fs_1.6.4 glue_1.7.0 graphics_4.4.0 grDevices_4.4.0
highr_0.10 htmltools_0.5.8.1 jquerylib_0.1.4 jsonlite_1.8.8 knitr_1.46 lifecycle_1.0.4
memoise_2.0.1 methods_4.4.0 mime_0.12 R6_2.5.1 rappdirs_0.3.3 rlang_1.1.3
rmarkdown_2.26.2 sass_0.4.9 stats_4.4.0 tinytex_0.50 tools_4.4.0 utils_4.4.0
xfun_0.43 yaml_2.3.8

Pandoc version: 3.1.11

Checklist

When filing a bug report, please check the boxes below to confirm that you have provided us with the information we need. Have you:

• formatted your issue so it is easier for us to read?

• included a minimal, self-contained, and reproducible example?

• pasted the output from xfun::session_info('rmarkdown') in your issue?

• upgraded all your packages to their latest versions (including your versions of R, the RStudio IDE, and relevant R packages)?

• installed and tested your bug with the development version of the rmarkdown package using remotes::install_github("rstudio/rmarkdown") ?

[cderv]

I can't run your example on Windows, because windows does not allow me to create a file with ~ in the name.

> temp_rmd
[1] "C:\\Users\\chris\\AppData\\Local\\Temp\\Rtmp6nVDJU/VERY_L~1/test.Rmd"
>   writeLines("---\ntitle: 'Test report'\n---\n\ntest report", temp_rmd)
Error in `file()`:
! impossible d'ouvrir la connexion
Run `rlang::last_trace()` to see where the error occurred.
Message d'avis :
Dans file(con, "w") :
  impossible d'ouvrir le fichier 'C:\Users\chris\AppData\Local\Temp\Rtmp6nVDJU/VERY_L~1/test.Rmd' : No such file or directory

Does the reprex shared above it working for you ? (first part of it)

~ has special meaning in file paths on Windows, are they are used for Short Path Name - R has a function to create those path (utils::shortPathName).

Does this happens only when you create some scripting around rmarkdown::render() ?

It seems on windows, when using normalize, it will only works when the path exists.

> xfun::normalize_path(dirname(temp_report))
[1] "C:/Users/chris/AppData/Local/Temp/Rtmp6nVDJU/VERY_LONG_LONG_LONG_PATH"
> xfun::normalize_path(temp_report)
[1] "C:/Users/chris/AppData/Local/Temp/RTC9FB~1/VERY_LONG_LONG_LONG_PATH/report.pdf"

See the difference above. Second path does not exist yet, and is passed through output_file in render.

So maybe this is why the path is not correctly transformed before running LaTeX.

(LaTeX won't handle short path windows syntax)

[cderv]

It seems on windows, when using normalize, it will only works when the path exists.

I don't know if this has changed, but this is definitely the problem it seems.

We should probably normalize output_dir and output_file here

rmarkdown/R/render.R

Line 498 in 3d99b8e

output_dir <- dirname(output_file)

using a wrapper that does unsure that shortpathname are normalized;

Otherwise, we need to patch only the LaTeX rendering here

rmarkdown/R/render.R

Lines 974 to 996 in 3d99b8e

	texfile <- file_with_ext(output_file, "tex")
	# determine whether we need to run citeproc (based on whether we have
	# references in the input)
	run_citeproc <- citeproc_required(front_matter, input_lines)
	# if the output format is LaTeX, first convert .md to .tex, and then convert
	# .tex to .pdf via latexmk() if PDF output is requested (in rmarkdown <=
	# v1.8, we used to call Pandoc to convert .md to .tex and .pdf separately)
	if (output_format$pandoc$keep_tex || knitr::is_latex_output()) {
	# do not use pandoc-citeproc if needs to build bibliography
	convert(texfile, run_citeproc && !need_bibtex)
	# patch the .tex output generated from the default Pandoc LaTeX template
	if (!("--template" %in% output_format$pandoc$args)) patch_tex_output(texfile)
	fix_horiz_rule(texfile)
	# unless the output file has the extension .tex, we assume it is PDF
	if (!grepl('[.]tex$', output_file)) {
	latexmk(texfile, output_format$pandoc$latex_engine, '--biblatex' %in% output_format$pandoc$args)
	file.rename(file_with_ext(texfile, "pdf"), output_file)
	# clean up the tex file if necessary
	if (!output_format$pandoc$keep_tex) {
	texfile <- normalize_path(texfile)
	on.exit(unlink(texfile), add = TRUE)
	}
	}

So that it ensures that the input tex path and others paths are normalized correctly before LaTeX rendering.

@yihui any thoughts ?

[LDSamson]

@cderv sorry, there was an error in the reprex. I fixed it in my initial post. Does it work for you now? The reprex should now hopefully demonstrate that it will also happen when the directory already exists.

Thanks for the quick follow-up.

See below the output of Sys.getenv("TEMP") on the Windows machine where this problem occurred:

`"C:\\Users\\LSAMSO~1\\AppData\\Local\\Temp"`

In a different Windows machine the error did not happen, because I had a shorter username, and a TEMP path without tilde.

I can work around the problem now (use a relative path for output_file); but hopefully someone else finds this report helpful, since it took me a long time to debug this problem.

[cderv]

yes thanks a lot for the report ! This is definitely an issue, and a case we want to handle !

This will be fix - just need to decide the scope of this. Short path on Windows should work with most tools - just not LaTeX. 😞

[yihui]

We should probably normalize output_dir and output_file here

rmarkdown/R/render.R

Line 498 in 3d99b8e

output_dir <- dirname(output_file)

using a wrapper that does unsure that shortpathname are normalized;

Otherwise, we need to patch only the LaTeX rendering here

rmarkdown/R/render.R

Lines 974 to 996 in 3d99b8e

	texfile <- file_with_ext(output_file, "tex")
	# determine whether we need to run citeproc (based on whether we have
	# references in the input)
	run_citeproc <- citeproc_required(front_matter, input_lines)
	# if the output format is LaTeX, first convert .md to .tex, and then convert
	# .tex to .pdf via latexmk() if PDF output is requested (in rmarkdown <=
	# v1.8, we used to call Pandoc to convert .md to .tex and .pdf separately)
	if (output_format$pandoc$keep_tex || knitr::is_latex_output()) {
	# do not use pandoc-citeproc if needs to build bibliography
	convert(texfile, run_citeproc && !need_bibtex)
	# patch the .tex output generated from the default Pandoc LaTeX template
	if (!("--template" %in% output_format$pandoc$args)) patch_tex_output(texfile)
	fix_horiz_rule(texfile)
	# unless the output file has the extension .tex, we assume it is PDF
	if (!grepl('[.]tex$', output_file)) {
	latexmk(texfile, output_format$pandoc$latex_engine, '--biblatex' %in% output_format$pandoc$args)
	file.rename(file_with_ext(texfile, "pdf"), output_file)
	# clean up the tex file if necessary
	if (!output_format$pandoc$keep_tex) {
	texfile <- normalize_path(texfile)
	on.exit(unlink(texfile), add = TRUE)
	}
	}

So that it ensures that the input tex path and others paths are normalized correctly before LaTeX rendering.

@yihui any thoughts ?

Either way seems to be okay to me, but perhaps we should normalize the paths only if they are absolute paths?

[cderv]

but perhaps we should normalize the paths only if they are absolute paths?

Oh that makes sense. Thanks for reminding me this !