From 322ea349a90be0f0008aca848afb55092c7c99ae Mon Sep 17 00:00:00 2001 From: Bob Jansen Date: Fri, 12 Jul 2019 21:49:27 +0200 Subject: [PATCH] Add ccache installation and setup instructions As a reaction to a question raised in the audience during userR! and the discussion on #103, an extra section in the README that explains (hopefully clearly) how to setup ccache for use with R and pak to speed up installation of packages that need compilation. Credits to @eddelbuettel for making me aware of this idea. Settings taken from his blog at http://dirk.eddelbuettel.com/blog/2017/11/27/ Compared to the post I've reduced maximum cache size from 5G to 1G as installation of packages tidyverse, devtools and data.table generates less than 150M of cache files and extended the inline comments somewhat. --- README.Rmd | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/README.Rmd b/README.Rmd index 5cdc91cb62..6d59b923bb 100644 --- a/README.Rmd +++ b/README.Rmd @@ -113,6 +113,59 @@ All dependencies will be installed as well, to the same library. * Package sizes. For CRAN packages pak shows the total sizes of packages it needs to download. +## Caching package compilation + +Some packages require compilation of source files to work. This +compilation process is relatively time consuming and will be repeated +every time package is re-installed or upgraded. Linux and Mac users +can use the tool ccache to cache the results of the initial +compilation and skip most of the compilation time the next time a +package is installed through pak (or other means). + + +### Setting up ccache + +The first step is to install ccache through your package manager or +[manually](https://ccache.dev/documentation.html). Successful +installation can be verified by running the command + +``` +ccache -V +``` + +from the command line. This will show the version of ccache installed +on your system. + +R can be instructed which files to use for compilation of packages +through the Makevars file in `~/.R/Makevars`. This can be done as follows: + +``` +CCACHE=ccache +CC=$(CCACHE) gcc$(VER) +CXX=$(CCACHE) g++$(VER) +CXX11=$(CCACHE) g++$(VER) +CXX14=$(CCACHE) g++$(VER) +FC=$(CCACHE) gfortran$(VER) +F77=$(CCACHE) gfortran$(VER) +``` + +and by adding the following to your ccache configuration in +`~/.ccache/ccache.conf`: + +``` +max_size = 1.0G +# Important for R CMD INSTALL *.tar.gz as tarballs are expanded again and again +# and therefore appear new to ccache. +sloppiness = include_file_ctime +# The packages will be unpacked into temporary directories with random file +# names. Therefore it is important to ignore the file path of the file that is +# being compiled. +hash_dir = false +``` + +ccache will now start building a cache of compiled source files and +then speed up package installation. + ## Roadmap * Support GitLab repositories diff --git a/README.md b/README.md index 290eaff42e..009a8791de 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,53 @@ All dependencies will be installed as well, to the same library. - Package sizes. For CRAN packages pak shows the total sizes of packages it needs to download. +## Caching package compilation + +Some packages require compilation of source files to work. This +compilation process is relatively time consuming and will be repeated +every time package is re-installed or upgraded. Linux and Mac users can +use the tool ccache to cache the results of the initial compilation and +skip most of the compilation time the next time a package is installed +through pak (or other means). + +### Setting up ccache + +The first step is to install ccache through your package manager or +[manually](https://ccache.dev/documentation.html). Successful +installation can be verified by running the command + + ccache -V + +from the command line. This will show the version of ccache installed on +your system. + +R can be instructed which files to use for compilation of packages +through the Makevars file in `~/.R/Makevars`. This can be done as +follows: + + CCACHE=ccache + CC=$(CCACHE) gcc$(VER) + CXX=$(CCACHE) g++$(VER) + CXX11=$(CCACHE) g++$(VER) + CXX14=$(CCACHE) g++$(VER) + FC=$(CCACHE) gfortran$(VER) + F77=$(CCACHE) gfortran$(VER) + +and by adding the following to your ccache configuration in +`~/.ccache/ccache.conf`: + + max_size = 1.0G + # Important for R CMD INSTALL *.tar.gz as tarballs are expanded again and again + # and therefore appear new to ccache. + sloppiness = include_file_ctime + # The packages will be unpacked into temporary directories with random file + # names. Therefore it is important to ignore the file path of the file that is + # being compiled. + hash_dir = false + +ccache will now start building a cache of compiled source files and then +speed up package installation. + ## Roadmap - Support GitLab repositories