I’ve been working on upgrading an app from Rails 2.3 to 3.0. As one of the first steps, I’m converting the app to use Bundler. Currently, all of the gems are in the vendor/gems directory, except for gems which need to be compiled — those are installed as system gems. I ran into some unexpected challenges with the upgrade. Thankfully, a friend had some advice.
wrestling with Bundler. and losing.
— Dan Manges (@dan_manges) February 3, 2013
To introduce Bundler as incrementally as possible, I wanted to try to have Bundler load the system gems and the gems from the vendor/gems directory without making any other changes. I didn’t want to have to install Bundler as a system gem, or change Capistrano scripts to run any additional commands as part of the deployment process. I was also hoping that the production runtime environment would remain as much the same as possible. I ran into two issues, both related to groups.
For an example, I’m going to use this Gemfile.
source :rubygems
gem "rails", "2.3.16"
group :development do
gem "faker", "1.1.2"
gem "thin", "1.5.0"
end
I was able to follow Bundler's instructions for Rails 2.3 successfully. If you remove the error handling the instructions, you end up with:
require "rubygems"
require "bundler"
Bundler.setup
I wanted to make sure that development gems weren’t loaded in production, so I added a group to the setup call to only load the default gems and the gems for the current environment.
Bundler.setup(:default, Rails.env)
The first problem that I noticed is that my development gems were available on the load path in a non-development irb console. Since the Faker gem is only in the development group, it shouldn’t be available.
:001 > $LOAD_PATH.grep(/faker/)
=> ["~/.rvm/gems/ruby-1.8@blog-bundler/gems/faker-1.1.2/lib"]
It took me a while to figure out why this was happening. I learned that Bundler adds -r bundler/setup to the RUBYOPT environment variable. Rails starts its console by making an exec call to irb. The require of bundler/setup is equivalent to calling Bundler.setup without any argument, so Bundler adds all gems to the load path. It doesn’t actually require the gems, but I’d still prefer to not have development gems on the load path in production.
require "rubygems"
require "bundler"
Bundler.setup(:default)
p $LOAD_PATH.grep(/faker/)
#=> []
exec "irb"
:001 > $LOAD_PATH.grep(/faker/)
=> ["~/.rvm/gems/ruby-1.8@blog-bundler/gems/faker-1.1.2/lib"]
The next problem that I ran into was that Bundler was raising an exception stating that gems needed to be installed in production even if they were only declared in the development group. Using the Gemfile from above and uninstalling thin, I’d get an error when calling Bundler.setup(:default).
# gems/bundler-1.2.0/lib/bundler/spec_set.rb:90:in `materialize':
# Could not find thin-1.5.0 in any of the sources (Bundler::GemNotFound)
Intuitively, I wouldn’t think that Bundler.setup(:default) (before the exec irb) would require development gems to be installed since they’re not actually used.
After digging through the Bundler source code for a while, I figured out the problem. A side effect of running bundle install is that Bundler writes a .bundle/config file. If you run bundle install --without development, then the file looks like:
---
BUNDLE_WITHOUT: development
Setting BUNDLE_WITHOUT makes the irb exec and the Bundler.setup call work as expected. Development gems no longer get added to the load path inside irb, and bundler no longer requires development gems to be installed even though they’re not loaded. You can test this by adding this line to the script above:
ENV["BUNDLE_WITHOUT"] = "development"
I think this issue happens because Bundler expects bundle install to be run to install gems. Because I already had system gems installed and all other gems unpacked into the vendor/gems directory, I didn’t think that I needed to run it. More importantly, I specifcally didn’t want to run it in production yet because I wanted to roll this release out without modifying the Capistrano scripts. It’s unintuitive that Bundler.setup(:some_group) will require gems in other groups to be installed unless BUNDLE_WITHOUT is also set. Or, if Bundler does expect install to run before any calls to setup, then Bundler could raise an exception to communicate that. I’d like to submit a patch to make this more clear, although changes may cause issues with backwards compatibility. I’ll either work around this issue for now, or go ahead and update deployment scripts to run bundle install when deploying, though it should be unnecessary.