Rails asset pipeline is quite a familiar topic for Ruby on Rails developers. It was first introduced in Rails 3.1 and it is now implemented by the sprockets-rails gem, and is enabled by default. In this tutorial, we’ll explore further into rails asset pipeline and the benefit is has when using JavaScript and CSS frameworks.

Let’s begin.

1. First, What is the Asset Pipeline?

As the name suggests, Asset Pipeline in Rails is used to process the assets before pushing them to the production environment.

Here, we will use the command: rails assets: precompile to process the assets (such as CSS and Javascript files) and save it in the public/assets (as default) folder on your Rails application. It essentially allows assets in your application to be automatically combined with other assets.

But what is behind this command? How does Rails process it?

Let’s get started!

2. Definitions

Sprockets::Environment

According to Sprockets, we will need an instance of the Sprockets:: Environment class to access and process assets. This instance is pre-defined in Rails. See below:

You can view how it is implemented in the source code of Rails here.

Load Path

Load Path is an ordered list of asset folders. By using Load Path, Sprockets can easily find assets and integrate with each other. Below are load paths in this application:

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/select2-rails-3.5.9.3/vendor/assets/images

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/select2-rails-3.5.9.3/vendor/assets/images

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/select2-rails-3.5.9.3/vendor/assets/images

/Users/duchoang/workspaces/rails/crb/app/assets/javascripts

/Users/duchoang/workspaces/rails/crb/app/assets/stylesheets

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/bootstrap-sass-3.2.0.0/assets/fonts

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/bootstrap-sass-3.2.0.0/assets/fonts

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/bootstrap-sass-3.2.0.0/assets/fonts

/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/bootstrap-sass-3.2.0.0/assets/font

A logical path is a path of assets in a load path. For example: the absolute path of assets /Users/duchoang/.rvm/gems/ruby-2.1.5/gems/select2-rails-3.5.9.3/vendor/assets/images/select2.png then logical path is select2.png and the load path is /Users/duchoang/.rvm/gems/ruby-2.1.5/gems/select2-rails-3.5.9.3/vendor/assets/images. 

Below is a list of logical paths in this application:

select2.png

select2-spinner.gif

select2x2.png

application.js

application.css

bootstrap/glyphicons-halflings-regular.eot

bootstrap/glyphicons-halflings-regular.svg

bootstrap/glyphicons-halflings-regular.ttf

bootstrap/glyphicons-halflings-regular.woff
#<Sprockets::Asset:3fd0a8de1724 "file:///Users/duchoang/workspaces/rails/crb/app/assets/javascripts/application.js?type=application/javascript&id=b0974b318282a6c93731434f474281d8eecb80fdb58a919a9195f9183a1e5bfb">

All the logical paths are located under a specific folder called mount point. With Rails, mount point of Sprockets environment is the assets folder. So, when a client requests the file /assets/application.js, Sprockets environment will search across all the load paths for the file named application.js to return it back to the client. Sprockets environment use find_asset or [path] to search for assets. For example, in order to find the asset named application.js, use Rails.application.assets.find_asset “application.js” or Rails.application.assets[“application.js”] and the result will be turn out like below:

You can find out more about find_asset function in Rails source code on GitHub here.

Processors

With Javascript assets, apart from using Javascript normally, we can also use coffee script. And for CSS, apart from using standard CSS, we could also use SCSS or SASS. Processor is an engine to convert the coffee script code back to normal Javascript and SCSS to standard CSS. This is because web browsers understand standard Javascript and CSS only.

Below are some processors in Sprockets:

Compressor

The name suggests its functionality. Compressors are used to compress assets to smaller sizes for faster downloads from client side but not readable. We can check the compressor Javascript and CSS in Rails by using:

Rails.application.config.assets.js_compressor => :uglifier

And

Rails.application.config.assets.css_compressor => :sass

We can use Rails.application.assets.compressors to view other compressors of Sprockets use

=> {

   "text/css"=>{

       :sass=>Sprockets::SassCompressor, :scss=>Sprockets::SassCompressor, :yui=>Sprockets::YUICompressor

   },

   "application/javascript"=>{

       :closure=>Sprockets::ClosureCompressor, :uglifier=>Sprockets::UglifierCompressor, :uglify=>Sprockets::UglifierCompressor, :yui=>Sprockets::YUICompressor

   }

Fingerprinting

Fingerprinting is a useful way in Rails to create file names by using the hashed value of file content. By using this technique, whenever the file content is changed, the hashed value and the file name of that content will also change.

Sprockets that use fingerprinting will append an MD5 hashed value after the file name. For example: File application.css after fingerprinting will have the name like:

application-a39441177efd45e6ac4sd35a66a2re41e33434rf72f8gf5ga45d4r53beac34e8.css

The appended hash value after the file name is done by:

  • Digest based on filepath then save to metadata on each asset
  • Hashing metadata[:digest]

You can refer to the source code of Sprocket digest_utilsSprocket assets and Sprocket base.

Manifest Files and Directives

Manifest files are files that store assets. For example, application.js is a Manifest file. Manifest files store some directive’s information which helps Sprockets to process it in case it’s needed, and then the multiple files are chained or linked into one file (.css, .js) to reduce the number of requests. Note that the Rails.application.config.assets.compress true config must be turned on.

Now that we’ve covered some important aspects in Sprocket and precompile assets, let’s dive into how assets pipeline actually works.

Pre-compile assets processing

When we execute rake assets:precompile command, the flow of compiling assets can be completed in a step-by-step process:

Step 1: Call to the precompile task which is defined in the file named task.rb of sprockets-rails. The content of rake task file will look like this:

Step 1

Step 1: Call to the precompile task which is defined in the file named task.rb of sprockets-rails. The content of rake task file will look like this:

desc "Compile all the assets named in config.assets.precompile"

task :precompile => :environment do

 with_logger do

   manifest.compile(assets)

 end

end

manifest is an instance of Sprockets::Manifest and it has the responsibility of saving the compile asset process to a specific compile folder. It will save some basic information about the assets for a fast search without the need of re-compiling. This file is a json data file, so that the search can be conducted extremely fast. The content of the file should look like this:

{

 "files":{

   "full_calendar-94f2588c650c3dbc9d62f5r64ef25167a4432wsd6ddb091fce912as186255637.js":{

     "logical_path":"full_calendar.js",

     "mtime":"2015-07-22T21:32:24+07:00",

     "size":1102019,

     "digest":"94f1327c650c3dbc6tf4f7cd6ef25167a4407e9d6ddc43efce946dd186255637",

     "integrity":"sha256-lPJYjGUMPbydYvfNbvJRZ6RAfp1t2wkfzpRt0YYlVjc="

   },

   "select2-1qw5d8d83dbc18eer577c8761d331cd9e5123c96849576550406e982wsdc5ae8.png":{

     "logical_path":"select2.png",

     "mtime":"2015-05-23T23:53:35+07:00",

     "size":613,

     "digest":"d6b5d8d83dfdg5fb8d77c8761d355ty9e5123c9684950cxcd406e98a24ac5ae8",

     "integrity":"sha256-1rSDSE28GPuNd8h3REDc2eUSPJaElQurBAbpiiSsWug="

   },

   ...

 }

}

assets is an array used to store information of the assets that are to be compiled. We can check the information of assets by using Rails.application.config.assets.precompile command. The result should look like this:

=> [#<Proc:[email protected]/Users/duchoang/.rvm/gems/ruby-2.1.5/gems/sprockets-rails-2.3.1/lib/sprockets/railtie.rb:60 (lambda)>, /(?:\/|\\|\A)application\.(css|js)$/, "schedules.js,", "full_calendar.js", /bootstrap\/glyphicons-halflings-regular\.(?:eot|svg|ttf|woff)$/, "select2.png", "select2-spinner.gif", "select2x2.png", "rails_admin/rails_admin.js", "rails_admin/rails_admin.css", "rails_admin/jquery.colorpicker.js", "rails_admin/jquery.colorpicker.css"]

Step 2

Use the compile function in the Sprockets::Manifest class in order to compile. There will be other sub-process steps in this step.

def compile(*args)

 unless environment

   raise Error, "manifest requires environment for compilation"

 end

 filenames = []

 find(*args) do |asset|

   files[asset.digest_path] = {

     'logical_path' => asset.logical_path,

     'mtime'        => Time.now.iso8601,

     'size'         => asset.bytesize,

     'digest'       => asset.hexdigest,

     # Deprecated: Remove beta integrity attribute in next release.

     # Callers should DigestUtils.hexdigest_integrity_uri to compute the

     # digest themselves.

     'integrity'    => DigestUtils.hexdigest_integrity_uri(asset.hexdigest)

   }

   assets[asset.logical_path] = asset.digest_path

   target = File.join(dir, asset.digest_path)

   if File.exist?(target)

     logger.debug "Skipping #{target}, already exists"

   else

     logger.info "Writing #{target}"

     asset.write_to target

   end

   filenames << asset.filename

 end

 save

 filenames

end

Instance Sprockets::Manifest will call the initial function. This function is responsible for finding json data file that was mentioned earlier. You can view the file content from this on GitHub.

As we can see, the above compile function will call to the

find(*args)

function, and the

find(*args)

function is an instance method of

Sprockets::Manifest

this function will use the instance of

Sprockets::CachedEnvironment

to find assets by using the logical path and then return with an Enumerator of Assets. The process is laid out like so:Instance

Sprockets::Manifest

will call the initial function. This function is responsible for finding json data file that was mentioned earlier. You can view the file content from this on GitHub.

As we can see, the above compile function will call to the find(*args) function, and the find(*args) function is an instance method of Sprockets::Manifest, this function will use the instance of Sprockets::CachedEnvironment to find assets by using the logical path and then return with an Enumerator of Assets. The process is laid out like so:

  • Take the function argument and classify logical paths from paths
  • Find assets based on the above classified logical paths by using environment.find_all_linked_assets(path) function, then return the corresponding Enumerator objects.

For example, for an asset returned with logical path: full_calendar.js will be converted to an array like this:

[#<Sprockets::Asset:3fe46495f71c "file:///Users/duchoang/workspaces/rails/crb/app/assets/javascripts/full_calendar.js?type=application/javascript&id=3e4r13759f41de45ac4cfd865658234123da09642b044adfbd3a55a3fcc02wst">]

This is a Sprockets::Asset object, and it is built while executing the load(uri) function of loader module. There are two cases here:

  • Load from cache if the asset you are looking for already exists in cache. That’s the reason why when we re-compile an asset without any modification, it will be really fast.
  • Calling the load_asset_by_uri(uri, filename, params) function. By using this, Rails will use processors to compress, build and save asset to cache.

The definition of a load function:

def load(uri)

 filename, params = parse_asset_uri(uri)

 if params.key?(:id)

   unless asset = cache.get("asset-uri:#{VERSION}:#{uri}", true)

     id = params.delete(:id)

     uri_without_id = build_asset_uri(filename, params)

     asset = load_asset_by_uri(uri_without_id, filename, params)

     if asset[:id] != id

       @logger.warn "Sprockets load error: Tried to find #{uri}, but latest was id #{asset[:id]}"

     end

   end

 else

   asset = fetch_asset_from_dependency_cache(uri, filename) do |paths|

     if paths

       digest = digest(resolve_dependencies(paths))

       if id_uri = cache.get("asset-uri-digest:#{VERSION}:#{uri}:#{digest}", true)

         cache.get("asset-uri:#{VERSION}:#{id_uri}", true)

       end

     else

       load_asset_by_uri(uri, filename, params)

     end

   end

 end

 Asset.new(self, asset)

end

Step 3

Write assets to the mount point folder. All the assets will be saved to the mount point folder when the write_to function gets called like so:

 

def write_to(filename)
FileUtils.mkdir_p File.dirname(filename)

 PathUtils.atomic_write(filename) do |f|

   f.write source

 end

 nil

end

And there you have it! The entire process behind precompiling assets. We hope this helps you to better understand how Rails assets pipeline works under the hood. Comment below if you have any questions!