My new rails application won’t display the “Yay!” page!

I was doing some recent environment testing with rails with the same application created on each of OSX, CentOS, and Ubuntu and was surprised when the application gave an error about no root route available, instead of the “Yay!” page:

I, [2020-02-12T16:51:06.845723 #46720] INFO -- : [8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] Started GET "/" for at 2020-02-12 16:51:06 -0800
F, [2020-02-12T16:51:06.846251 #46720] FATAL -- : [8d1ab83a-b4e4-45bf-a734-a6494c4d15c2]
F, [2020-02-12T16:51:06.846316 #46720] FATAL -- : [8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] ActionController::RoutingError (No route matches [GET] "/"):
F, [2020-02-12T16:51:06.846353 #46720] FATAL -- : [8d1ab83a-b4e4-45bf-a734-a6494c4d15c2]
F, [2020-02-12T16:51:06.846390 #46720] FATAL -- : [8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] actionpack ( lib/action_dispatch/middleware/debug_exceptions.rb:65:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] actionpack ( lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] railties ( lib/rails/rack/logger.rb:38:in `call_app'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] railties ( lib/rails/rack/logger.rb:26:in `block in call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] activesupport ( lib/active_support/tagged_logging.rb:71:in `block in tagged'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] activesupport ( lib/active_support/tagged_logging.rb:28:in `tagged'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] activesupport ( lib/active_support/tagged_logging.rb:71:in `tagged'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] railties ( lib/rails/rack/logger.rb:26:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] actionpack ( lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] actionpack ( lib/action_dispatch/middleware/request_id.rb:27:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] rack (2.2.2) lib/rack/method_override.rb:24:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] rack (2.2.2) lib/rack/runtime.rb:22:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] activesupport ( lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] actionpack ( lib/action_dispatch/middleware/executor.rb:14:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] rack (2.2.2) lib/rack/sendfile.rb:110:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] railties ( lib/rails/engine.rb:524:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] puma (3.12.2) lib/puma/configuration.rb:227:in `call'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] puma (3.12.2) lib/puma/server.rb:674:in `handle_request'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] puma (3.12.2) lib/puma/server.rb:476:in `process_client'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] puma (3.12.2) lib/puma/server.rb:334:in `block in run'
[8d1ab83a-b4e4-45bf-a734-a6494c4d15c2] puma (3.12.2) lib/puma/thread_pool.rb:135:in `block in spawn_thread'

I had assumed that any new rails application would serve out the, every so happy, “Yay!” page, regardless of environment. It turns out not to be the case. To fix the issue, set the RAILS_ENV to development and restart the rails server:

> RAILS_ENV=development rails server
=> Booting Puma
=> Rails application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.2 (ruby 2.6.0-p0), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop

Easy fix!

Rails new fails to create all of the content…

I recently deployed a Rails app on a CentOS 8 container and was surprised to see that the ‘rails new’ command did not create the entire application directory structure that I would expect, ie:

[root@607ebc931c48 app]# rails _5.2.3_ new centos_app_2
      create  Rakefile
      create  .ruby-version
      create  .gitignore
      create  Gemfile
         run  git init from "."

I didn’t know that git was a requirement now for rails apps, so the issue was resolved by installing the git client.

# dnf install -y git

To get a rails application created using a CentOS 8 docker container, use the following steps. This will assume rails 5.2.3 and ruby 2.6.3:

docker run -i -t --rm -v ${PWD}:/usr/src/app centos:8 bash
dnf install curl git gnupg2 which -y
gpg2 --keyserver hkp:// --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
curl -sSL | bash -s stable
source /etc/profile.d/
rvm requirements
rvm install 2.6.3
gem install rails -v 5.2.3
rails _5.2.3_ new centos_app

Note that the container actually takes quite a bit to build, so making an image would be more useful as subsequent rails commands could be run more easily.

Why is rails 5.2 joining to the same table twice?

I was working on a Rails project for a client recently where I was trying to figure out why a model search operation was joining to the same table twice, resulting in a huge performance hit. The answer turned out to be an incomplete understanding of the ‘has_many’ ‘:through’ relationship.

Given a model with the following relationships (using the rails documentation example):

class Physician < ApplicationRecord
  has_many :appointments
  has_many :patients, through: :appointments

I added a search method to return results based on web form input of a begin timestamp, end timestamp, and search term. I am also using the will_paginate gem. Also, my dataset is large enough that I do not want to return results without input parameters:

  def, current_page, begin_timestamp, end_timestamp)
    if search
      if search.length > 0 
        Physician.includes(:patients, :appointments).where('patients.updated_at BETWEEN ? AND ?  and LIKE ?', "#{begin_timestamp}", "#{end_timestamp}", "%#{search}%").references(:patients, :appointments).paginate(page: current_page, per_page: 15).order(' ASC').distinct

This resulted in a query (as seen in the mysql console and the rails application logs) that joined to the appointments table twice. Once for the has_many, through association, and once for the includes, references as added to the model.

The answer ended up being to remove the reference to appointments in the Physician model.

  def, current_page, begin_timestamp, end_timestamp)
    if search
      if search.length > 0 
        Physician.includes(:patients).where('patients.updated_at BETWEEN ? AND ?  and LIKE ?', "#{begin_timestamp}", "#{end_timestamp}", "%#{search}%").references(:patients).paginate(page: current_page, per_page: 15).order(' ASC').distinct

This optimization increased performance by a huge margin. The application is now usable.

Rails timestamp search using datetime_field_tag

I recently worked on an issue when creating a Rails form to search and display objects by timestamp and spent far too much time troubleshooting an issue with the datetime_field_tag form helper.

The issue began with the following two lines in my view:

    <%= datetime_field_tag(:begin_timestamp, params[:begin_timestamp]) %>
    <%= datetime_field_tag(:end_timestamp,   params[:end_timestamp]) %>

Evidently, you cannot place more than one space between the name assigned to the form helper and the value to populate in the form helper. The above two lines don’t format well in this wordpress blog post, but there are 3 spaces between ‘:end_timestamp’ and ‘params[:end_timestamp]’. The parameter was passed to the controller, and the code worked, but it would not display the current selection in the datetime_field_tag on submit.

The fix was to remove the additional formatting spaces, ie:

    <%= datetime_field_tag(:begin_timestamp, params[:begin_timestamp]) %>
    <%= datetime_field_tag(:end_timestamp, params[:end_timestamp]) %>

Hope this helps somebody. I spent far too much time on this.

More info:

> rails server
=> Booting Puma
=> Rails 5.2.3 application starting in production
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.1 (ruby 2.6.0-p0), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: production
* Listening on tcp://
Use Ctrl-C to stop

ruby aws-sdk strikes again…

When using ruby to upload files to S3 and trying to use multipart upload, beware the following ArgumentError:

...param_validator.rb:32:in `validate!': unexpected value at params[:server_side_encryption] (ArgumentError)
	from /var/lib/jenkins/.gem/ruby/gems/aws-sdk-core-3.6.0/lib/seahorse/client/request.rb:70:in `send_request'
	from /var/lib/jenkins/.gem/ruby/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/client.rb:3980:in `list_parts'

The options passed to list_parts must not include “server_side_encryption”. I always forget to remove this parameter.

A good way that I have found to solve this issue is:

      input_opts = {
        bucket:                 bucket,
        key:                    key,
        server_side_encryption: "AES256",

      if defined? mime_type
        input_opts = input_opts.merge({
          content_type: mime_type,
      input_opts.delete_if {|key,value| key.to_s.eql?("content_type") }
      input_opts.delete_if {|key,value| key.to_s.eql?("server_side_encryption") }

      input_opts = input_opts.merge({
          :upload_id   => mpu_create_response.upload_id,

      parts_resp = s3.list_parts(input_opts)

You can see here that I delete values that may have been added so that the final options hash will work with the list_parts call.

Puppet deprecation in stdlib module…

As part of the long upgrade to become fully compatible with puppet 4 and drop puppet 3 support — version 4.13+ of the stdlib module introduced some breaking changes for other modules that I use. I recently upgrade some individual modules using the ‘puppet module upgrade’ method.

Upon upgrading, I received the following message:

Error: Evaluation Error: Error while evaluating a Function Call, undefined method `function_deprecation' ...

The solution, for now, until the modules that I use are upgraded to work with the newer version of stdlib, is to downgrade to version 4.12 of puppetlabs-stdlib.

Before downgrading, check to see if there are other modules which require a version greater than 4.12, ie:

> for file in $(find modules/ -type f -name metadata.json); do echo -n ${file}; echo -n ":  "; egrep -i stdlib ${file} || echo ""; done
modules//apt/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"}
modules//archive/metadata.json:        "name": "puppetlabs/stdlib",
modules//concat/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 4.2.0 < 5.0.0"}
modules//ec2tagfacts/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"},
modules//epel/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 3.0.0"}
modules//java/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 2.4.0 < 5.0.0"}
modules//jenkins/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 4.6.0 < 5.0.0"},
modules//lvm/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">=4.1.0 < 5.0.0"}
modules//mysql/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"},
modules//python/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">= 4.6.0 < 6.0.0"},
modules//rvm/metadata.json:      {"name":"puppetlabs/stdlib","version_requirement":">=4.2.0"},

The following modules require a version of puppetlabs-stdlib greater than 4.12:

  • apt – 4.5+
  • concat – 4.2.0+
  • jenkins- 4.6.0+
  • python4.6.0+
  • rvm – 4.2.0+

Sting with the apt module, the apt module is version 4.1.0. According to the puppetlabs-apt change log page, they recommend staying on version 2.3.0 unless you’re ready for the latest puppet 4 changes (they actually say any version 2 release but we’ll let that go after figuring it out…), so we need to downgrade that module first:

> puppet module upgrade puppetlabs-apt --version 2.3.0 --modulepath=modules/
Notice: Preparing to upgrade 'puppetlabs-apt' ...
Notice: Found 'puppetlabs-apt' (v2.4.0) in .../puppet/modules ...
Notice: Downloading from ...
Error: Could not upgrade module 'puppetlabs-apt' (v2.4.0 -> v2.3.0)
  Downgrading is not allowed.

Ok, so you can’t use the upgrade command to downgrade modules, so I need to be a little more heavy handed. I removed the apt module manually:

> rm -rf modules/apt/

Next, install version 2.3.0 of puppetlabs-apt:

> puppet module install puppetlabs-apt --version 2.3.0 --modulepath=modules/
Notice: Preparing to install into .../puppet/modules ...
Notice: Downloading from ...
Notice: Installing -- do not interrupt ...
└─┬ puppetlabs-apt (v2.3.0)
  └── puppetlabs-stdlib (v4.19.0)

Now that I’m deep into this problem, I see great value in using something like r10k or Code Manager (which uses r10k) to manage these modules and dependencies.

Another potentially useful tip – I removed the puppetlabs-stdlib module and ran a module list command, and it then told me which modules were dependent upon the missing module, and which versions:

> puppet module list --modulepath=modules/
Warning: Missing dependency 'puppetlabs-stdlib':
  'puppetlabs-apt' (v2.3.0) requires 'puppetlabs-stdlib' (>= 4.5.0 < 5.0.0)
  'puppet-archive' (v2.0.0) requires 'puppetlabs-stdlib' (>= 4.13.0 < 5.0.0)
  'puppetlabs-concat' (v2.2.0) requires 'puppetlabs-stdlib' (>= 4.2.0 < 5.0.0)
  'bryana-ec2tagfacts' (v0.2.0) requires 'puppetlabs-stdlib' (>= 3.2.0 < 5.0.0)
  'stahnma-epel' (v1.2.2) requires 'puppetlabs-stdlib' (>= 3.0.0)
  'puppetlabs-java' (v1.5.0) requires 'puppetlabs-stdlib' (>= 2.4.0 < 5.0.0)
  'rtyler-jenkins' (v1.7.0) requires 'puppetlabs-stdlib' (>= 4.6.0 < 5.0.0)
  'puppetlabs-lvm' (v0.7.0) requires 'puppetlabs-stdlib' (>=4.1.0 < 5.0.0)
  'puppetlabs-mysql' (v3.10.0) requires 'puppetlabs-stdlib' (>= 3.2.0 < 5.0.0)
  'stankevich-python' (v1.14.2) requires 'puppetlabs-stdlib' (>= 4.6.0 < 6.0.0)
  'maestrodev-rvm' (v1.13.1) requires 'puppetlabs-stdlib' (>=4.2.0)

Moving on to using r10k..

gem install r10k

I then created a Puppetfile using my modules and am using r10k to manage them.

Note: converting to using r10k took around 20 minutes — if you’re not using r10k (or Code Manager), it’s time to start.

Throttling Requests with the Ruby aws-sdk

A common problem of late is throttling requests when using the ruby aws-sdk gem to access AWS services. Handling these exceptions is fairly trivial with a while loop like the following:

retry_count   = 0 
retry_success = 0 

while retry_success == 0
  retry_success = 1

    # enter code to interact with AWS here

  rescue Aws::APIGateway::Errors::TooManyRequestsException => tmre

  # note that different AWS services have different exceptions
  # for this type of response, be sure to check your error output

    sleep_time = ( 2 ** retry_count )
    retry_success = 0 
    sleep sleep_time
    retry_count = retry_count + 1 


Note that there are different exceptions for different services that might indicate a throttling scenario so be sure to check the output received or the documentation around which exception to handle. Also note that additional exceptions should be handled around bad requests, missing, duplicate, unavailable, or mal-formed objects.

Multipart uploads to s3 using aws-sdk v2 for ruby…

The Ruby guys over at AWS have done a great job at explaining file uploads to S3 but they left out how to perform multipart uploads citing reservation over “advanced use cases“.


  • identify an S3 bucket to upload a file to — use an existing bucket or create a new one
  • create or identify a user with an access key and secret access key
  • install version 2 of the aws-sdk gem
  • this example uses ruby 2.2.0, other versions may not be supported or work the same way

A multipart upload consists of four steps: setup a client connection to S3, a call to create_multipart_upload, one or more calls to upload_part, and finally, and if all works out, a call to complete_multipart_upload. Otherwise, the final step would be an abort_multipart_upload.

Before starting the upload, I will first establish a constant to define the file size before an upload will be executed using multipart or whether I will simply call upload_file.

# 100MB

Next, over-ride the File class and setup a method to return parts of the file:

class File
  def each_part(part_size=PART_SIZE)
    yield read(part_size) until eof?

The rest of this script assumes that between 4 and 5 parameters were passed into the script from the command line. The reason we pass all of these in is so that we can keep credentials out of source control and so we can use this script with any bucket and any object. The optional ‘prefix’ parameter would be text that would be prepended to the key and facilitate organizing key objects into a directory structure.

(access,secret,bucket,localfile,prefix) = ARGV

Next, we need to establish an authenticated client connection to S3:

s3 =
  region: 'us-east-1',

This will establish a connection to the us-east-1 region using the access key and secret access key variables listed.

Next, I will setup the key with any path information removed and with the optional prefix prepended:

  filebasename = File.basename(localfile)

  if prefix
    key = prefix + "/" + filebasename
    key = filebasename

The next step is to open the File and determine whether or not it’s large enough to warrant a multipart upload:, 'rb') do |file|
    if file.size > PART_SIZE
      puts "File size over #{PART_SIZE} bytes, using multipart upload..."

Next, we use create_multipart_upload to start the upload and return an upload_id needed to manage the rest of the process. Replace bucket and key with your s3 bucket and the intended name of the file on S3 (aka key).

      input_opts = {
        bucket: bucket,
        key:    key,

      mpu_create_response = s3.create_multipart_upload(input_opts)

If all worked well there, we can then upload the parts. I like to give some level of progress as the parts are uploaded so let’s add some code to do that as well:

      total_parts = file.size.to_f / PART_SIZE
      current_part = 1 

Next, we can iterate through the file parts as we upload them:

      file.each_part do |part|

        part_response = s3.upload_part({
          body:        part,
          bucket:      bucket,
          key:         key,
          part_number: current_part,
          upload_id:   mpu_create_response.upload_id,

        percent_complete = (current_part.to_f / total_parts.to_f) * 100 
        percent_complete = 100 if percent_complete > 100 
        percent_complete = sprintf('%.2f', percent_complete.to_f)
        puts "percent complete: #{percent_complete}"
        current_part = current_part + 1 


This part also computed the complete percentage which prints out a progress line after each part is uploaded. That part is optional but I’ve found my clients really appreciate it.

Once all of the parts are uploaded, we can finish the multipart upload with a call to complete_multipart_upload:

      input_opts = input_opts.merge({
          :upload_id   => mpu_create_response.upload_id,

      parts_resp = s3.list_parts(input_opts)

      input_opts = input_opts.merge(
          :multipart_upload => {
            :parts =>
     do |part|
              { :part_number => part.part_number,
                :etag        => part.etag }

      mpu_complete_response = s3.complete_multipart_upload(input_opts)

Note that we called list_parts to get a some required information about each part. This would allow us to make a multi-threaded upload client but that would require a different approach relative to what I have done here and we will leave that for advanced use cases (ha!).

Finally, be sure to close the if statement and provide the put_object method for files smallert than PART_SIZE:

      s3.put_object(bucket: bucket, key: key, body: file)

The script can then be called with the following format:

./upload_file.rb $access_key $secret_key $bucket $upload_file $optional_prefix

Now that completes the multipart upload. A critical piece that was not covered here is that if you do not complete an upload, it is consuming space that you are paying for and you either need to abort it or complete it to free up this space. I will share a utility that I have to clean these up in a future article.