Cloudflare R2 - Aws::S3::Errors::NoSuchKey (The specified key does not exist.)

[kubatron117]

Hello,

I am having an issue when uploading to R2 object storage. I tried using S3, which works well, but I need to set up R2.

Here is my configuration:

shrine.rb:
s3_options = {
bucket: ENV["BUCKET_NAME"],
region: "auto",
access_key_id: ENV["ACCESS_KEY_ID"],
secret_access_key: ENV["SECRET_ACCESS_KEY"],
endpoint: ENV["ENDPOINT"],
force_path_style: true,
public: false
}

Shrine.storages = {
cache: Shrine::Storage::S3.new(prefix: "cache", upload_options: {}, **s3_options),
store: Shrine::Storage::S3.new(copy_options: {}, **s3_options)
}

Shrine.plugin :activerecord
Shrine.plugin :cached_attachment_data
Shrine.plugin :restore_cached_data
Shrine.plugin :validation_helpers
Shrine.plugin :validation
Shrine.plugin :determine_mime_type
Shrine.plugin :store_dimensions
Shrine.plugin :remove_invalid
Shrine.plugin :derivatives
Shrine.plugin :pretty_location
Shrine.plugin :add_metadata
Shrine.plugin :backgrounding
Shrine.plugin :presign_endpoint

Error: Aws::S3::Errors::NoSuchKey (The specified key does not exist.)

The problem is that I can upload photos to the cache, but when derivatives are created, I think the process fails during the move from the cache to the regular destination. I cannot resolve the issue. Any help would be greatly appreciated.

[janko]

Could you post a minimal self-contained script that reproduces the error with Cloudflare R2? You can use the following as a template – https://github.com/shrinerb/shrine/blob/master/SELF_CONTAINED_EXAMPLE.md#appendix-b-activerecord-template

[kubatron117]

Thank you for the quick response. I created a "demo code" that returns the same error.

The files are successfully uploaded to the cache, but the process fails when moving them out of it. I have attached a screenshot showing that the files are correctly stored in the cache.

require "bundler/setup"
require "active_record"
require "shrine"
require "shrine/storage/file_system"
require "shrine/storage/s3"
require "down"

s3_options = {
  bucket: "bucket_name",
  region: "auto",
  access_key_id: "acess_key_id",
  secret_access_key: "secret_access_key",
  endpoint: "https://r2.endpoint.domain",
  force_path_style: true,
  public: false
}

Shrine.storages = {
  cache: Shrine::Storage::S3.new(prefix: "cache", upload_options: {}, **s3_options),
  store: Shrine::Storage::S3.new(copy_options: {}, **s3_options)
}

Shrine.plugin :activerecord
Shrine.plugin :pretty_location

class ImageUploader < Shrine
  def generate_location(io, record: nil, **context)
    tenant_id = record[:tenant_id] || "default"
    "tenants/#{tenant_id}/#{super}"
  end
end

ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Base.connection.create_table(:posts) { |t| t.text :image_data }

class Post < ActiveRecord::Base
  include ImageUploader::Attachment(:image)
end

post = Post.create(image: Down.download("https://images.pexels.com/photos/15286/pexels-photo.jpg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=2"))

puts "Image uploaded successfully with data: #{post.image_data}"

Gemfile

source "https://rubygems.org"

gem "bundler", "~> 2.0"
gem "activerecord", "~> 7.1.3.4"
gem "shrine", "~> 3.6.0"
gem 'aws-sdk-s3', '~> 1.159.0'
gem "down"
gem 'image_processing', '~> 1.13'
gem 'mini_exiftool', '~> 2.11.0'
gem "sqlite3", "~> 1.4"
gem 'nokogiri'

PS: In actual use, I am using Ruby on Rails.

[kubatron117]

Finally, I found a mistake in my endpoint URL configuration. I had also included the bucket name in the URL.