Adding Direct S3 Uploads (original) (raw)
This walkthrough shows how to add asynchronous uploads to a Rails app. The flow will go like this:
- User selects file(s)
- For each file a presign request is made to fetch AWS S3 upload parameters
- Files are uploaded asynchronously to AWS S3
- Uploaded file JSON data is written to a hidden field
- Form is submitted instantaneously as it only has to submit the JSON data
- JSON data is assigned to the Shrine attachment attribute
AWS S3 setup
You'll need to create an AWS S3 bucket, which is where the uploads will be stored. See this walkthrough on how to do that.
Next you'll need to configure CORS for that bucket, so that it accepts uploads directly from the client. In the AWS S3 Console go to your bucket, click on the "Permissions" tab and then on "CORS configuration". There paste in the following:
https://my-app.com GET POST PUT 3000 Authorization x-amz-date x-amz-content-sha256 content-type ETag * GET 3000Replace https://my-app.com
with the URL to your app (in development you can set this to *
). Once you've hit "Save", it may take some time for the new CORS settings to be applied.
1. Installation
Add Shrine and aws-sdk-s3 to the Gemfile:
Gemfile
gem "shrine", "> 3.0"
gem "aws-sdk-s3", "> 1.14"
and run bundle install
.
2. Initializer
Add your S3 credentials to your application:
s3: bucket: "" region: "" access_key_id: "" secret_access_key: ""
...
Then create an initializer which configures your S3 storage with those credentials and loads default plugins:
config/initializers/shrine.rb
require "shrine" require "shrine/storage/s3"
s3_options = Rails.application.credentials.s3
Shrine.storages = { cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options), store: Shrine::Storage::S3.new(**s3_options), }
Shrine.plugin :activerecord # load Active Record integration Shrine.plugin :cached_attachment_data # for retaining cached file on form redisplays Shrine.plugin :restore_cached_data # refresh metadata for cached files
3. Migration
Add the <attachment>_data
text or JSON column to the table to which you want to add the attachment:
$ rails generate migration add_image_data_to_photos image_data:text # or :jsonb
This should generate the following migration:
class AddImageDataToPhotos < ActiveRecord::Migration def change add_column :photos, :image_data, :text # or :jsonb end end
Run rails db:migrate
to apply the migration.
4. Attachment
Create an uploader for the types of files you'll be uploading:
app/uploaders/image_uploader.rb
class ImageUploader < Shrine end
and add an attachment attribute to your model:
app/models/photo.rb
class Photo < ApplicationRecord include ImageUploader::Attachment(:image)
validates_presence_of :image end
5. Form
In your form you can now add form fields for the attachment attribute, and an image tag for the preview:
<%= form_for @photo do |f| %>
In your controller make sure to allow the attachment param:
app/controllers/photos_controller.rb
class PhotosController < ApplicationController
...
def create @photo = Photo.new(photo_params)
if @photo.save
redirect_to @photo
else
render :new
end
end
...
private
def photo_params params.require(:photo).permit(..., :image) # permit attachment param end end
Now you should have working synchronous file uploads.
6. Direct upload
We can now add asynchronous direct uploads to the mix. We'll be using Uppyand its AwsS3 plugin, which will upload selected files directly to S3 using params fetched from Shrine's presign endpoint.
6a. Presign endpoint
We'll first configure the presign_endpoint
plugin and mount it in our routes:
config/initializers/shrine.rb
...
Shrine.plugin :presign_endpoint, presign_options: -> (request) {
Uppy will send the "filename" and "type" query parameters
filename = request.params["filename"] type = request.params["type"]
{ content_disposition: ContentDisposition.inline(filename), # set download filename content_type: type, # set content type (defaults to "application/octet-stream") content_length_range: 0..(1010241024), # limit upload size to 10 MB } }
config/routes.rb
Rails.application.routes.draw do
...
mount Shrine.presign_endpoint(:cache) => "/s3/params" end
6b. Uppy
Now we can setup Uppy to do the direct uploads. First we'll add the package to our bundle (we're assuming you're using webpacker):
Now we can setup direct uploads to S3, where upload params request will be fetched from Shrine's presign endpoint, and upload result will be written to the hidden attachment field:
// app/javascript/fileUpload.js
import 'uppy/dist/uppy.min.css'
import { Core, FileInput, Informer, ProgressBar, ThumbnailGenerator, AwsS3, } from 'uppy'
function fileUpload(fileInput) { const hiddenInput = document.querySelector('.upload-data'), imagePreview = document.querySelector('.upload-preview img'), formGroup = fileInput.parentNode
// remove our file input in favour of Uppy's formGroup.removeChild(fileInput)
const uppy = Core({
autoProceed: true,
})
.use(FileInput, {
target: formGroup,
})
.use(Informer, {
target: formGroup,
})
.use(ProgressBar, {
target: imagePreview.parentNode,
})
.use(ThumbnailGenerator, {
thumbnailWidth: 600,
})
.use(AwsS3, {
companionUrl: '/', // will call the presign endpoint on /s3/params
})
uppy.on('thumbnail:generated', (file, preview) => { // show preview of the image using URL from thumbnail generator imagePreview.src = preview })
uppy.on('upload-success', (file, response) => { // construct uploaded file data in the format that Shrine expects const uploadedFileData = { id: file.meta['key'].match(/^cache/(.+)/)[1], // object key without prefix storage: 'cache', metadata: { size: file.size, filename: file.name, mime_type: file.type, } }
// set hidden field value to the uploaded file data so that it's submitted
// with the form as the attachment
hiddenInput.value = JSON.stringify(uploadedFileData)
}) }
export default fileUpload
You may wish to initialize on turbolinks (or turbo) loading: or alternatively, using a stimulus controller:
(A) Initialize on Turbo load
// app/javascript/packs/application.js // ... import fileUpload from 'fileUpload'
// if you are using turbo, listen on turbo:load instead // listen on 'turbolinks:load' instead of 'DOMContentLoaded' if using Turbolinks document.addEventListener('DOMContentLoaded', () => { document.querySelectorAll('.upload-file').forEach(fileInput => { fileUpload(fileInput) }) })
(B) Initialize using a Stimulus Controller
- Multiple Upload Controller and see the ruby code needed in the view.
- Single Upload Controller and see the ruby code in the view.
And add the following CSS:
/* app/assets/stylesheets/application.css / / ... */ .upload-preview img { display: block; max-width: 100%; }
.upload-preview { margin-bottom: 10px; display: inline-block; height: 300px; }
img[src=""] { visibility: hidden; }
And that's it, now when a file is selected it will be asynchronously uploaded to your app, accompanied by an image preview and a nice progress bar.
Restricting Access to Presign Endpoint
You may want to restrict the access to your presign endpoint to protect your s3 bucket.
One way to do this (if you are using Rails) is to use constraints on your rails route (e.g., example using headers, or via plain ol' ruby):
Rails.application.routes.draw do
# need a true statement with the constraint - in this case that a header exists using a lambda
mount Shrine.presign_endpoint(:cache) => "/s3/params", constraints: -> (req) {
req.headers["HEADER_THAT_SHOULD_EXIST"]
}
# or you can use a ruby class
mount Shrine.presign_endpoint(:cache), :constraints => AdminConstraint.new # only admins should get access
# or you can route requests to a separate controller where you can return whatever rack response you design
end
./app/lib/admin_constraint.rb
class AdminConstraint def matches?(request) return false unless request.session[:user_id] user = User.find request.session[:user_id] user && user.site_admin? # only logged in admin users should be able to upload end end