Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rmarkdown uses incorrect working directory if filename has spaces and output format is "github_document" #1980

Open
3 tasks done
cboettig opened this issue Dec 14, 2020 · 5 comments · May be fixed by #1982
Open
3 tasks done
Labels
bug an unexpected problem or unintended behavior theme: paths path related improvment / issue

Comments

@cboettig
Copy link
Contributor

cboettig commented Dec 14, 2020

By filing an issue to this repo, I promise that

  • I have fully read the issue guide at https://yihui.org/issue/.
  • I have provided the necessary information about my issue.
    • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
    • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('rmarkdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/rmarkdown').
    • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
  • I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.


As you can in the reprex below, all this does is create a trivial Rmarkdown document in a subdirectory, and then attempt to knit it. Because the RMarkdown document has a space in the name and we have requested github_document, the example errors by having the wrong working directory (the root dir, not the same dir from where the Rmarkdown file is found, as in the usual convention!) Change the output format to html_document, or remove the space in the name, and the error is resolved.

Consider the following reprex:

dir.create("subdir")
download.file("https://gist.githubusercontent.com/cboettig/5ba7013764152c1305bf26d81ec94fbe/raw/0b23bc008f816fdb80f9744da35d7ee78ba5ce4a/Bug%2520Report.Rmd",
"subdir/Bug Report.Rmd")
rmarkdown::render("subdir/Bug Report.Rmd")
#> processing file: Bug-Report.Rmd
#> Quitting from lines 8-10 (Bug-Report.Rmd)
#> Error: file.exists("Bug Report.Rmd") is not TRUE

Created on 2020-12-14 by the reprex package (v0.3.0)

@atusy
Copy link
Contributor

atusy commented Dec 14, 2020

Reproduced. It seems the bug originally comes from md_document which is the base format of github_document.

atusy added a commit to atusy/rmarkdown that referenced this issue Dec 15, 2020
@atusy
Copy link
Contributor

atusy commented Dec 15, 2020

I see issue comes from here, which removes dirname() from input when input has shell characters.

input <- input_no_shell_chars

Then, if intermediates_dir = NULL as default, the following lines considers input file exists in the current directory.

rmarkdown/R/render.R

Lines 355 to 356 in 80f14b2

input_no_shell_chars <- intermediates_loc(
file_name_without_shell_chars(basename(input)))

A workaround is to explicitly specifying intermediates_dir = "subdir".
However, I think intermediates_dir should become explicit much earlier than

rmarkdown/R/render.R

Lines 369 to 372 in 80f14b2

if (is.null(intermediates_dir)) {
intermediates_dir <-
dirname(normalize_path(input_no_shell_chars))
}

I'll open a PR.

@atusy atusy linked a pull request Dec 15, 2020 that will close this issue
@cderv
Copy link
Collaborator

cderv commented Dec 22, 2020

@atusy what I don't understand correctly about this bug is why it works ok with html_document and not with md_document ?

dir.create(tmp_dir <- tempfile())
owd <- setwd(tmp_dir)
dir.create("subdir")
download.file("https://gist.githubusercontent.com/cboettig/5ba7013764152c1305bf26d81ec94fbe/raw/0b23bc008f816fdb80f9744da35d7ee78ba5ce4a/Bug%2520Report.Rmd",
              "subdir/Bug Report.Rmd", quiet = TRUE)
# no error
rmarkdown::render("subdir/Bug Report.Rmd", output_format = "html_document", quiet = TRUE)
browseURL("Bug-Report.html")
# error
rmarkdown::render("subdir/Bug Report.Rmd", output_format = "md_document", quiet = TRUE)
#> Quitting from lines 8-10 (Bug-Report.Rmd)
#> Error: file.exists("Bug Report.Rmd") n'est pas TRUE
# browseURL("Bug-Report.html")
setwd(owd)
unlink(tmp_dir)

The code you pointed out is the same for both formats, isn't it ?

It seems both formats are not doing the same thing. Adding a debug(knitr:::block_exec) to stop the rendering before the chunk is evaluated we can observe this

  • Using html_document we have

    • This during the rendering
       > fs::dir_tree()
       .
       +-- Bug Report.Rmd
       +-- Bug-Report.Rmd
       \-- subdir
           \-- Bug Report.Rmd
    • and this at the end
    > fs::dir_tree()
      .
      +-- Bug-Report.html
      \-- subdir
          \-- Bug Report.Rmd
  • Using md_document

> fs::dir_tree()
.
+-- Bug-Report.Rmd
\-- subdir
    \-- Bug Report.Rmd

So in the second case, the initial Rmd file is not copied and in the first case it is copied.

Also @cboettig the title of the issue is about a wrong working directory. However, the working directory (root dir of knitr chunk) is the same with both html_document and github_document

dir.create("subdir")
xfun::write_utf8(c(
 "```{r}",
 "getwd()",
 "```"
), "subdir/Bug Report.Rmd")
res <- rmarkdown::render("subdir/Bug Report.Rmd", output_format = "html_document", quiet = TRUE)
#>[WARNING] This document format requires a nonempty <title> element.
#>  Defaulting to 'Bug-Report.utf8' as the title.
#>  To specify a title, use 'title' in metadata or --metadata title="...".
content <- xfun::read_utf8(res)
# getwd() folder of knitr chunk
xml2::xml_text(xml2::read_html(content[grepl("## ", content)]))
#>[1] "## [1] \"C:/Users/chris/AppData/Local/Temp/RtmpUN37nz/file969428684ff7\""
res <- rmarkdown::render("subdir/Bug Report.Rmd", output_format = "github_document", quiet = TRUE)
content <- xfun::read_utf8(res)
# Same dir as above
content[grepl("## ", content)]
#>[1] "    ## [1] \"C:/Users/chris/AppData/Local/Temp/RtmpUN37nz/file969428684ff7\""
setwd(owd)
unlink(tmp_dir)

Did I missed something ?

I would like to understand better the real issue here and currently the reprex is not very clear as it mixes the issue presented first here with what it told in the title here.

@cderv
Copy link
Collaborator

cderv commented Dec 22, 2020

So about the issue reported by @cboettig and shown in the initial reprex, which is working for html_document and not for github_document, or any md_document based format:

This is because md_document does not contains any intermediates_generator whereas html_document does. This intermediates_generator in html_document will copy the resources found to the intermediate dir. I think that is why we don't see the error with html_document

However, I believe this is an undesired side effect which is hiding the issue and there is something not right in the way paths and directories are handled when there is a space. And I think you tried to fix that @atusy.

I need to look closer on how this should work and why we get this weird behavior

@atusy
Copy link
Contributor

atusy commented Dec 22, 2020

@cderv Thanks for the follow up. I also guessed intemediates_generator is making the side effect, but skipped to take a closer look.

@cderv cderv added bug an unexpected problem or unintended behavior theme: paths path related improvment / issue labels Jan 6, 2022
@cderv cderv linked a pull request Jan 6, 2022 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug an unexpected problem or unintended behavior theme: paths path related improvment / issue
Projects
Status: Backlog
Development

Successfully merging a pull request may close this issue.

3 participants