Convention over Configuration

Will Ruby Last?

A conversation just struck me the other day. Are people really leaving the Ruby community? Are we really "lead footed" and imploding in a "version hell"?

I'm not going to talk about either of those things, but they made me think. When I first got started with programming I used Flex... and everything was XML... and everyone good used Java. And everything was configurable. The motto:

XML is like violence. If it doesn't solve your problem, you're not using enough of it.

Such a mistake...

Ruby Apps are becoming too Configurable

Now that Rack is so popular, a lot of Rack gems are being pumped out.

• The Rack Middleware Directory

And there's a hundred other gems being pumped out every day.

• Rubygems' Twitter Feed
• Github has over 1 Million Repositories

Rack and Rubygems aren't the problems though, neither is the sheer number of them. In fact, the more gems the better in my opinion, because the reality is every piece of functionality, once you get deep in it, should be solved thoroughly and will result in something like a gem. That's a good thing.

The problem is, we are building our gems to be too Configurable. Configuration is punching Convention in the face.

Convention Over Configuration, Take Two

You guys all know about Configuration Over Configuration, it's what made Rails popular. It's what DDH keeps saying. It's on the front page of his blog.

Convention over Configuration is the reason people start programming. It's the reason why Ruby == Agile.

.

But as Ruby and Rails got more established, and more and more people started depending on it, and more and more people needed their custom "configuration", we developers started catering to them. We started building our gems so that, by default, there are no dependencies. If you want to have "x" feature, require it. Then all of a sudden this

1

require 'rubygems'

turns into this

1
2
3

require 'rubygems'
require 'bundler'
Bundler.setup

And this

1
2

require 'rubygems'
require 'sinatra'

Becomes this

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

require 'rubygems'
require 'bundler'
Bundler.setup
begin
  gem 'rack', '~> 0.2.0'
  require 'rack'
  require 'rack/utils'
rescue Exception => e
  puts "wrong version of rack, run `bundle install`"
end
require 'sinatra/base'
# ...

You know it is, you can feel it. Maybe not exactly that code snippet, but like it. There's too many versions and the dependencies are too configurable. If things keep going like this, Ruby will be like Java.

Ruby was a Disruptive Innovation, but like all things that mature, they lose the ability to adapt.

Configuration means:

• We have to have more knowledge of the product, which means
  • We increase the time spent configuring
  • We increase the room for error in managing version conflicts
  • We decrease the time spent innovating
• The product requires more time and resources to support, which means
  • The product has to grow and grow to handle more configuration options
  • The people who invented the project are either stuck there or have to drop it
  • The community's growth stagnates, which inevitably means
  • The technology will be replaced by a disruptive innovation.

Enter Node.js, and that conversation...

Convention over Configuration: How-to

1. Users should only have to require one path

Even if your rubygem is packed with functionality and could possibly be divided into smaller 'sub-gems', the end-user should not have to know about that. The gem should be a black box.

Take building an Oauth/OpenID wrapper for example, along the lines of OauthPlugin, AuthlogicConnect, Passport, or OmniAuth, this is something I'm working with lately. Say your gem is called 'magic-auth'. If you wanted to include oauth and openid in a simple Sinatra app, with support for MyOpenID, Google, Facebook, Twitter, and LinkedIn, the trend is to do something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

require 'rubygems'
require 'sinatra'
require 'magic-auth'
# if openid
gem "ruby-openid", ">= 0.5.0"
require 'ruby-openid'
require 'rack-openid'
require 'magic-auth/openid'
require 'openid/store/filesystem'
# if oauth
require 'oauth' # version 1
require 'oauth2' # version 2
gem "rack-oauth", "~> 0.3.1"
require 'rack-oauth'
require 'magic-auth/oauth'

enable :sessions

use MagicAuth::Twitter, 'key', 'secret'
use MagicAuth::Facebook, 'key', 'secret'
use MagicAuth::LinkedIn, 'key', 'secret'
use MagicAuth::Google, 'key', 'secret'
use MagicAuth::MyOpenID, OpenID::Store::Filesystem.new('/tmp')

get "/" do
  "..."
end

That is, the user must specify exactly what libraries they want that your library uses internally.

Now all of a sudden I have to know about the OpenID gem, about Rack, about Oauth, and about MagicAuth. But I just wanted to Login with Facebook!

On one hand, requiring this configuration complexity is great. It requires you to dig into the code and figure it out, which means you'll learn how great coders code. And it means by the end you'll probably understand the whole Oauth protocol and have written a few tutorials on it. But you'll have shaved a month off your life.

A much simpler solution is to handle that stuff in the background, so the user only has to do this:

1
2
3
4
5
6
7
8
9

require 'rubygems'
require 'sinatra'
require 'magic-auth'

use MagicAuth, "credentials.yml"

get "/" do
  "..."
end

Then inside your gem, you solve those gem dependency problems really well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

.
|-- lib
|   |-- magic_auth.rb # base dependencies
|   |-- core
|   |-- support # env specific configuration
|     |-- sinatra.rb
|     |-- rails.rb
|     |-- active_record.rb
|     |-- mongo_db.rb
|     |-- redis.rb
|   |-- ...
`-- README.markdown # convensions, simple setup

And inside MagicAuth, you can process credentials.yml do do exactly what that verbose use MagicAuth::Twitter chunk did.

Leaving it so we have to manually require and configure everything for your gem:

1. Increases the amount of code I must write to get started
2. Increases the depth of knowledge I must have to get started
3. Increases the risk for error
4. Increases the amount I have to maintain

Instead, encapsulate all the problems and quirks inside your gem and leave it so all your user has to do is

1

require 'your-gem'

The end result is that the end user, the one who uses your gem, can get started as easy as it was to get started with Rails.

2. Use Interfaces

Define interfaces up front. Use Design Patterns. Create Interface Gems.

• How do you want the gem to be used?
• How do I decrease the barrier to entry and have it require as little knowledge as possible?

Solve these problems right away, they make Test Driven Development exciting. And in the end you have something extremely simple to use.

I'll use Oauth to illustrate. Oauth and OpenID are pretty convoluted, and all of the solutions up until now have been pretty convoluted. Almost every Oauth rubygem has been built for basically 1 service, and there is no common pattern to it. The reason for this is clear, it's not possible without using design patterns.

I do this too, it's a hard habit to get used to. It's hard to remember design patterns' solution to such multifaceted problems. But if we keep this in mind - use design patterns - from the start, we get these benefits:

• Interfaces allow us to solve the same problem for more cases.
• Once we solve the first problem, the subsequent problems are infinitely easier.

So for Oauth + OpenID + BasicAuth, we can boil it down to something like this:

• All of these services act the same if you really think about it, so
• Use the Strategy Pattern to make it so all services have the same Interface.
• Authentication protocols can be boiled down to a request and callback phase (from omniauth)
• And they all seem to boil down to having the same core properties: key, token, and secret. So even if Twitter calls it something different than FreshBooks, find some way to make them look the same.

Once we establish an interface, we can easily build more just like it, even before testing.

So first, we make a Facebook strategy for connecting to oauth, and for well-being's sake, we want the developer api to be as simple as possible, so we come up with something like this:

1
2
3
4
5
6
7
8
9

class Facebook < Oauth
  key do |access_token|
    JSON.parse(access_token.get("/me"))["id"]
  end

  settings "https://graph.facebook.com",
    :authorize_url => "https://graph.facebook.com/oauth/authorize",
    :scope         => "email, offline_access"
end

Then you write some tests, spend a lot of time making sure you can actually write the next service like that, get it up on Github, and you're golden.

The next one is a piece of cake.

1
2
3
4
5
6

class Twitter < Oauth
  key :user_id

  settings "http://api.twitter.com",
    :authorize_url => "http://api.twitter.com/oauth/authenticate"
end

And they both would have this API:

1
2
3
4
5
6

service = [Twitter, Facebook][rand(1) + 1]
service.key
service.token
service.secret
service.request
service.response

Once that first pattern is put into place, writing tests and extending your gem is easy. And to the end user, they don't need to know "oh I need x version of this dependency, and I need to configure it like y". No, we should solve that problem in our gem. Otherwise, we create a community that's dependent on configuration, which will always lead to over complexity in the long run. And overly dependent systems have a very hard time adapting to change; i.e, they're not Agile.

So to sum up, I'm very excited to be seeing so many great gems being pumped out daily. But in order to choose a fate different that Java, I suggest we make our gems require near-zero configuration. Convention is a core goal, as should be User Experience.

What are your thoughts? Agree, disagree? I'd love to hear your comments.