Ruby, Rackspace Cloudfiles, Performance, and Fog

I work quite a bit with Rackspace cloudfiles and use ruby to upload and download files using the fog gem. There have been some major performance issues with apps that use the fog gem where there is a lack of understanding around how the container get, new, and save methods work. The basic issue is that the get and save methods will be very slow:

“the get method makes an HTTP call that returns metadata for up to the first 10,000 files”

With this knowledge, it is possible to make things work very fast, even if a container is not known to exist before writing to it, with the use of an exception handling block.

require 'rubygems'
require 'fog'

service = Fog::Storage.new({
    :provider            => 'Rackspace',
    :rackspace_username  => 'username',
    :rackspace_api_key   => 'api_key',
    :rackspace_region    => :ord,
    :connection_options  => {}
})

container = service.directories.new :key => "container-name"
file = container.files.create :key => 'test.txt', :body => File.open("test.txt")

begin
  file.save
rescue Fog::Storage::Rackspace::NotFound => e
  container.save 
  file.save
end

This piece of code will try to write a file to a container and if that container does not exist, the Fog::Storage::Rackspace::NotFound exception will be caught to allow the container to be created, and then the file saved.