Skip to content

Commit

Permalink
Add compression option for non-memory caches
Browse files Browse the repository at this point in the history
Fixes #71
  • Loading branch information
coolbutuseless authored and jimhester committed Oct 9, 2018
1 parent 06d16ec commit 2e6ee17
Show file tree
Hide file tree
Showing 8 changed files with 26 additions and 10 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ Suggests:
covr,
googleCloudStorageR
License: MIT + file LICENSE
RoxygenNote: 6.0.1
RoxygenNote: 6.1.0
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
* Name clashes between function arguments and variables defined when memoising
no longer occur (#43, @egnha).
* Add Google Cloud Storage support via `cache_gcs()` (#59 - @MarkEdmondson1234)
* Add `compress` option for non-memory caches (#71 - @coolbutuseless)

# Version 1.1.0
* Caches now hash the function body along with the arguments, to ensure
Expand Down
6 changes: 4 additions & 2 deletions R/cache_filesystem.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
#' Use a cache on the local filesystem that will persist between R sessions.
#'
#' @param path Directory in which to store cached items.
#' @param compress Argument passed to \code{saveRDS}. One of FALSE, "gzip",
#' "bzip2" or "xz". Default: FALSE.
#'
#' @examples
#'
Expand All @@ -23,7 +25,7 @@
#'
#' @export
#' @inheritParams cache_memory
cache_filesystem <- function(path, algo = "xxhash64") {
cache_filesystem <- function(path, algo = "xxhash64", compress = FALSE) {

if (!dir.exists(path)) {
dir.create(path, showWarnings = FALSE)
Expand All @@ -35,7 +37,7 @@ cache_filesystem <- function(path, algo = "xxhash64") {
}

cache_set <- function(key, value) {
saveRDS(value, file = file.path(path, key))
saveRDS(value, file = file.path(path, key), compress = compress)
}

cache_get <- function(key) {
Expand Down
6 changes: 4 additions & 2 deletions R/cache_gcs.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
#'
#'
#' @param cache_name Bucket name for storing cache files.
#' @param compress Argument passed to \code{saveRDS}. One of FALSE, "gzip",
#' "bzip2" or "xz". Default: FALSE.
#' @inheritParams cache_memory
#' @export
cache_gcs <- function(cache_name = googleCloudStorageR::gcs_get_global_bucket(),
algo = "sha512") {
algo = "sha512", compress = FALSE) {

if (!(requireNamespace("googleCloudStorageR"))) { stop("Package `googleCloudStorageR` must be installed for `cache_gcs()`.") } # nocov

Expand All @@ -33,7 +35,7 @@ cache_gcs <- function(cache_name = googleCloudStorageR::gcs_get_global_bucket(),
cache_set <- function(key, value) {
temp_file <- file.path(path, key)
on.exit(unlink(temp_file))
saveRDS(value, file = temp_file)
saveRDS(value, file = temp_file, compress = compress)
suppressMessages(
googleCloudStorageR::gcs_upload(temp_file, name = key, bucket = cache_name)
)
Expand Down
6 changes: 4 additions & 2 deletions R/cache_s3.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
#'
#'
#' @param cache_name Bucket name for storing cache files.
#' @param compress Argument passed to \code{saveRDS}. One of FALSE, "gzip",
#' "bzip2" or "xz". Default: FALSE.
#' @inheritParams cache_memory
#' @export

cache_s3 <- function(cache_name, algo = "sha512") {
cache_s3 <- function(cache_name, algo = "sha512", compress = FALSE) {

if (!(requireNamespace("aws.s3"))) { stop("Package `aws.s3` must be installed for `cache_s3()`.") } # nocov

Expand All @@ -37,7 +39,7 @@ cache_s3 <- function(cache_name, algo = "sha512") {
cache_set <- function(key, value) {
temp_file <- file.path(path, key)
on.exit(unlink(temp_file))
saveRDS(value, file = temp_file)
saveRDS(value, file = temp_file, compress = compress)
aws.s3::put_object(temp_file, object = key, bucket = cache_name)
}

Expand Down
5 changes: 4 additions & 1 deletion man/cache_filesystem.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion man/cache_gcs.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion man/cache_s3.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 2e6ee17

Please sign in to comment.