Life On The Edge With Merb, DataMapper & RSpec

This book is work in progress, your help is requested @ github

Life On The Edge With Merb, DataMapper & RSpec

Foreword

(TODO) the foreward

Preface

At New Bamboo we've been building our internal applications in Merb for quite some time now. This is a collaborative effort to document the features of Merb and DataMapper, while also providing example Merb applications.

Who is this for?

The target reader is someone who has some experience with writing Ruby on Rails applications, and is looking to try out a new framework. So if your looking for the AWDR equivalent for Merb, you may be disappointed as not all topics will be covered (yet).

Copyright © 2008 Respective Authors.

Authors


This work is licensed under a Creative Commons Attribution-Noncommercial 2.0 UK: England & Wales License.

Creative Commons License

Source code of the applications are dual licensed under the MIT and GPL licenses:

What's Merb, DataMapper & RSpec?

If you're not living on the edge, you're taking up too much room. - Alice Bartlett

Merb, DataMapper and RSpec are all open source projects that are great for building kick-ass web applications. They are all in active development and although it can be hard, we'll try our best to keep up-to-date.

Merb

It's a relatively new framework (a bit like Ruby on Rails) and was created by Ezra Zygmuntowicz. Merb stands for Mongrel + ERB although now it supports the rack webserver interface so it can use any web server that has rack support (Mongrel, Thin, ebb, etc).

If you know Ruby and have used Rails you're likely to get the hang of Merb quite easily. Noticeable differences between Merb and Rails include its stance on being less opinionated and its approach to modularisation. (TODO) - Clarify "opinionated?" doesn't force a particular style? (TODO) - Add more differences, or change the structure of this sentence to just reflect on one major difference.

Merb consists of a number of gems: merb-core, merb-more and merb-plugins. This means that it is possible to pick and choose the functionality you need, instead of cluttering up the framework with non-essential features. The merb gem installs both merb-core and merb-more; all you need in order to get started straight away. The benefit of this modularity is that the framework remains simple and focussed with additional functionality provided by gems.

Thanks to Merb's modularity, you are not locked into using any particular libraries. For example, Merb ships with plugins for several popular ORMs and provides support for both Test::Unit and RSpec.

merb-core alone provides a lightweight framework (a la camping) that can be used to create a simple web app such as an upload server or API provider where the functionality of an all-inclusive framework is not necessary.

DataMapper

DataMapper is an Object-Relational Mapper (ORM) written in Ruby by Sam Smoot. We'll be using DataMapper with Merb. It's possible to use the same ORM as Rails (ActiveRecord), but as there are plenty of examples of using ActiveRecord already so we've chosen to focus on DataMapper.

DataMapper has some nice features that make it faster than ActiveRecord in some cases. What really stands out for me is the way it handles database attributes. The schema, migrations and attributes are all defined in one place: your model. So you no longer have to look around in your database or other files to see what is defined.

While DataMapper has similarities to ActiveRecord we will be highlighting the differences as we go along.

RSpec

RSpec is a Behaviour Driven Development framework for Ruby. Merb currently supports the Test::Unit and RSpec testing frameworks. As the specs for Merb and Datamapper are written in RSpec, we will be covering some aspects of RSpec but it will not be our main focus.

RSpec is (TODO) Beef up this section a little. As this section is entitled "What's Merb, DataMapper & RSpec", we should probably spend a little time describing RSpec

What About Ruby On Rails?

[Merb is] Harder, Better, Faster, Stronger, to quote Daft Punk - Max Williams

So what's the big deal? We have Ruby on Rails and that's enough, isn't it? There is little doubt that Ruby on Rails has rocked the web application development world. You have to give credit where credit's due, but it can be unforgiving if you don't want to do things 'the Rails way'.

Where Rails is opinionated, Merb is agnostic. You can easily use your favourite ORM (ActiveRecord, DataMapper, Sequel) or none at all, and choose the Javascript library and template language.

Merb also has super-fast routing and is thread-safe (If performant were a word, Merb would be it). The core functionality is kept separate from the other plugins and it uses less Ruby 'magic', which makes it easy to understand and hack.

Rails (and consequently Ruby) has received a lot of criticism for not being suitable for large scale web applications, which isn't necessarily true. Merb has been built from the outset to prove that Ruby is a viable language for building fast and scaleable web applications.

At the end of the day it's about choice. There are many new Ruby frameworks springing up, undoubtedly helped by the success of Rails, but in my opinion Merb shows the most promise as a Rails competitor.

If you'd like to take a look at some other frameworks these links should get you started:

Communities

i'm going to become rich and famous after i invent a device that allows you to stab people in the face over the internet - <[SA]HatfulOfHollow>

The internet is a scary place, but fortunately the Ruby community is very friendly. All these open source projects rely on the contributions from the community, if something needs fixing consider helping out.

Websites

These are the first places to go for help. Check out the API documentation and see if you can find your answer there.

IRC Channels - freenode.net

If you can't find what you were looking for in the API docs then you could join the respective IRC channel on FreeNode and ask your question in there, you may need to wait for a response.

Mailing Lists

The mailing lists are another good way to get help, the response time isn't as fast as asking in an irc channel but it can be useful to do a search to see if someone else has had your problem before.

Bug Trackers

Your problem may or may not be a known bug. Search the bug trackers and submit a ticket if its not there already (don't forget to include a description and test cases, or better yet: a patch!). You may find the ticket is solved in the edge version.

Wikis

Getting Started

XKCD - Compiling

Before we get started I'm going to assume you have the following installed:

Installing Merb

If you have an older version of Merb (<0.9.2) you should remove the all the gems before continuing. Use gem list to see your installed gems.

Installing the merb gems should be as simple as:

sudo gem install merb --source http://merbivore.org

or for JRuby:

jruby -S gem install merb mongrel

Unfortunately we are living right on the edge of development so we'll need to get down and dirty with building our own gems from source. Luckily this is much easier than it sounds...

Start by installing the gem dependancies:

sudo gem install rack mongrel json_pure erubis mime-types rspec hpricot \
    mocha rubigen haml markaby mailfactory ruby2ruby

Then download the merb source:

git clone git://github.com/wycats/merb-core.git
git clone git://github.com/wycats/merb-plugins.git
git clone git://github.com/wycats/merb-more.git

Then install the gems via rake:

cd merb-core ; rake install ; cd ..    
cd merb-more ; rake install ; cd ..
cd merb-plugins; rake install ; cd ..

The json_pure gem is needed for merb to install on JRuby (Java implementation of a Ruby Interpreter), otherwise use the json gem as it's faster.

Merb is ORM agnostic, but as the title of this book suggests we'll be using DataMapper. Should you want to stick with ActiveRecord or play with Sequel, check the merb documentation for install instructions.

Installing DataMapper

DataMapper is splitting into dm-core and dm-more so datamapper 0.3 will be outdated soon. If you have an older version of datamapper, data_objects, or do_mysql, merb_datamapper (< 0.9) you should remove them first. Remove the merb_datamapper gem before installing dm-merb within dm-more.

We will use MySQL in the following example, but you can use either sqlite3 or PostgreSQL, just install the appropriate gem. You will also need to ensure that MySQL is on your system path for the gem to install correctly.

(TODO) - JDBC_do install

To get the gems from source:

git clone git://github.com/sam/do.git

cd do
cd data_objects
rake install ; cd ..
cd do_mysql  # || do_postgres || do_sqlite3
rake install

git clone git://github.com/sam/dm-core.git
git clone git://github.com/sam/dm-more.git

cd dm-core ; rake install ; cd ..
cd dm-more
cd merb_datamapper ; rake install ; cd ..    
cd dm-migrations ; rake install ; cd ..
cd dm-validations ; rake install ; cd ..

To update a gem from source, run git pull and rake install again.

Install RSpec

The rspec gem was installed in the Merb section above. However, if for some reason you didn't install it there, or want to grab the it from source, run the following commands:

gem install rspec
svn checkout http://rspec.rubyforge.org/svn/trunk rspec_trunk

(TODO) RSpec moved to http://github.com/dchelimsky/rspec/tree/master

The Easy Way

If you're on a *nix or OSX operating system then keeping upto date with all the edge versions of these gems can be made really easy by using the Edgy sake tasks.

All you need to run to get RSpec, merb-core, merb-more, dm-core, dm-more, data_objects & all the other dependent gems installed automgically is...

    sudo gem install sake
    sake -i 'http://edgy.4ninjas.org/edgy.sake'
    sake edgy:install packages="merb-stack"

And then to keep upto date you just need to execute

    sake edgy:update

Creating an App

Now that we've got all of that installed, time to create a test Merb application.

Merb more comes with a gem called merb-gen, this gives you a command line tool by the same name which is used for all of your generator needs. You can think of it as script/generate done the Merb way. Running merb-gen from the command line with no arguments will show you all of the generators that are available.

Merb follows the same naming convention for projects as rails, so 'my_test_app' and 'Test2' are valid names but 'T 3' is not (they need to be valid SQL table names).

merb-gen app test

This will generate an empty Merb app, so lets go in and take a look. You'll notice that the directory structure is similar to Rails, with a few differences.

# expected output
RubiGen::Scripts::Generate
  create  app
  create  app/controllers
  create  app/helpers
  create  app/views
  create  app/views/exceptions
  create  app/views/layout
  create  autotest
  create  config
  create  config/environments
  create  public
  create  public/images
  create  public/stylesheets
  create  spec
  create  app/controllers/application.rb
  create  app/controllers/exceptions.rb
  create  app/helpers/global_helpers.rb
  create  app/views/exceptions/internal_server_error.html.erb
  create  app/views/exceptions/not_acceptable.html.erb
  create  app/views/exceptions/not_found.html.erb
  create  app/views/layout/application.html.erb
  create  autotest/discover.rb
  create  autotest/merb.rb
  create  autotest/merb_rspec.rb
  create  config/rack.rb
  create  config/router.rb
  create  config/init.rb
  create  config/environments/development.rb
  create  config/environments/production.rb
  create  config/environments/rake.rb
  create  config/environments/test.rb
  create  public/merb.fcgi
  create  public/images/merb.jpg
  create  public/stylesheets/master.css
  create  spec/spec.opts
  create  spec/spec_helper.rb
  create  /Rakefile

Configuring Merb

Before we get the server running, you'll need to edit the init.rb file and un-comment the following line (this is only necessary if you need to connect to a database, which we do in our case):

config/init.rb

use_orm :datamapper

Typing merb now in your command line will try and start the server.

Started merb_init.rb ...
No database.yml file found in /Users/work/merb/example_one/config.
A sample file was created called database.sample.yml for you to copy and edit.

As you can see, we forgot to set up the database. A sample file has helpfully been generated for us. Edit this and rename it to database.yml:

# This is a sample database file for the DataMapper ORM
development:
   adapter: mysql
   database: test
   username: root
   password: 
   host: localhost
   socket: /tmp/mysql.sock

Don't forget to specify your socket, if you do not know it's location, you can find it by typing:

mysql_config --socket

Starting Merb again shows that everything is running okay.

The following command will give you access to the Merb console:

merb -i

You'll notice Merb runs on port 4000, but this can be changed with flag -p [port number]. More options can be found by typing:

merb --help

You can even run Merb with any application server that supports rack (thin, evented_mongrel, fcgi, mongrel, and webrick):

merb -a thin

The Framework

The directory structure of the project created should look like the following. We'll give brief overview of the framework here and go into further details of each component in subsequent chapters.

test
  |--> app
  |--> autotest
  |--> config
  |--> log
  |--> public
  `--> spec

The app folder contains your models, views (including exception pages and layouts) and controllers, helpers. It also has Parts, they inherit from AbstractController and similar to the old Rails components, but are lightweight and are useful for sidebars, widgets etc. Mailers, which also inherit from the AbstractController have their own folder where the controllers and views live. 

app
  |--> controllers
  |--> models (generated with a model)
  |--> helpers
  |--> mailers (generated with a mailer)
  |--> helpers
  |--> parts (generated with a parts controller)
  `--> views

The config folder has all the configuration files and environments. It's important to edit the init.rb and database.yml files in here before running Merb. The Merb router, which maps the incoming requests to the controllers is also here. The rack.rb file is the rack handler and you can pass options to merb -a to change rack adapter.

config
  `--> environments

RSpec specs can be found in the spec folder.

spec

In addition to these folders you can have a gem directory, which stores frozen gems (see Freezing Gems for more info), and a lib folder to store other ruby files.

A little blog

In the examples we'll be developing a small blogging application. It's a good idea to grab the source code from http://github.com/deimos1986/book_mdar/tree/master/code, so you can follow along with the examples.

First of all let's define some of the functionality we would expect from any blogging application.

We're going to call our app golb, think of it as a backward blog. Feel free to change the name of your app but remember to change the word golb for the name of your app.

To make a new app we'll use the command

merb-gen app golb

We're going to use the Linguistics gem later on, you can install it with:

gem install Linguistics

Set up the configuration files for your application, this lets Merb know what gems to load for plugins and generators.

config/init.rb

use_orm :datamapper

use_test :rspec

dependencies "Linguistics", "dm-validations"

Now add a config/database.yml file with the following:

---
# Edit this file:
:development: &defaults
    # These are the settings for repository :default
    :adapter:  mysql
    :database: golb
    :host: localhost
    :encoding: utf8
    :username: root
    :password:
    :socket: /opt/local/var/run/mysql5/mysqld.sock

test: &defaults
    # These are the settings for repository :default
    :database: golb_test

Now we're ready to rock and roll ...

Models

(TODO) - rewrite for DM 0.9, and make this section clearer

Getting started

Having discussed the functionality we can deduce that we will need the following models, Post, Comment, Tag, User and Image.

DataMapper has a model generator just as rails does:

merb-gen model post

This will make a post model for you, provided that you have defined an orm and the database golb, in the previous steps.

When you run rake dm:db:automigrate, it will create the database table and all the properties, but take care this is a destructive method!

You can set the name of the database table in your model if it is called something different with:

set_table_name 'list_of_posts'

This is only necessary if you are using an already existing database.

Properties

So DataMapper models differ a bit from ActiveRecord models as previously stated. Defining the database columns is achieved with the property method.

app/models/post.rb

property :title,  String, :lazy => false

This is the title property of the post model. As we can see, the parameters are the name of the table column followed by the type and finally the options.

Some of the available options are: (TODO) - cover more properties

:public, :protected, :private, :accessor, :reader, :writer,
:lazy, :default, :nullable, :key, :serial, :field, :size, :length,
:format, :index, :check, :ordinal, :auto_validation, :validates, :unique,
:lock, :track, :scale, :precision

:key          - Set as primary key
:serial       - auto-incrementing key
:lazy         - Lazy load the specified property (:lazy => true).
:default      - Specifies the default value
:column       - Specifies the table column
:nullable     - Can the value be null?
:index        - Creates a database index for the column
:accessor     - Set method visibility for the property accessors. Affects both
                reader and writer. Allowable values are :public, :protected, :private.
:reader       - Like the accessor option but affects only the property reader.
:writer       - Like the accessor option but affects only the property writer.
:protected    - Alias for :reader => :public, :writer => :protected
:private      - Alias for :reader => :public, :writer => :private

(TODO) - talk about accessors and overriding them

DataMapper supports the following properties:

(TODO) - creating your own custom properties

Associations

Like ActiveRecord, DataMapper has associations which define relationships between models. There is difference in syntax but the underling idea is the same. Continuing with the Post model we can see a few of the associations defined:

one_to_many :comments
many_to_one :author, :class => 'User', :foreign_key => 'author_id'

(TODO) the alternate has n..n syntax for relationships

Pretty straight forward. A few things you should note however, you do not need to specify the foreign key as a property if it's defined in the association, and currently has_one is implemented as has_many (so it returns an array with one object instead of just the object itself, but this is will likely change!).

You also don't have to specify a relationship at all if you don't want to, as models can have one way relationships.

Polymorphic associations

http://pastie.textmate.org/private/mrvx3qmuagypwukrri9jq (TODO) -polly assoc (pastie link is broken)

Where is my has_many :through?!

has_many :through is in the pipes, but it is not currently in DM. You can however mimic that behaviour by specifying the tables to join on, in the join model.

# has_many :categories, :through => :categorizations
has_many :categorizations

has_and_belongs_to_many :categories,
  :join_table => "categorizations",
  :left_foreign_key => "post_id",
  :right_foreign_key => "category_id",
  :class => "Category"

You still have access to .categorizations and you now have access to .categories as well… plus no new tables or nothing. If categorizations had, say, a score column on it which stores how strongly your categorization process thinks this post belongs to this category, you could tack on :order => 'score desc' to the has_and_belongs_to_many just fine.

Validation

(TODO) - custom validation, and validatable gem It’s a known fact that users are stupid. They screw up; it happens. They enter information in the wrong format, leave required fields blank, or even enter in completely horrid data because they’re idiots and that’s what idiots do. I point you at YouTube video comments, Digg (as a whole), and MySpace as proof of web users’ collective idiocy.

But, alas, they’re how we make our money online. Thus, we need to guard against user error by validating anything that we need to save out to our persistence layers. Sometimes that means guarding against hack attempts, but most of the time it means guarding against invalid data and accidents.

Both ActiveRecord and DataMapper have a concept called Validations, which is ultimately a set of callbacks which fire right before an object gets saved out to our persistence layer and interrupt things when it detects something awry.

A problem arises when your website has users creating content and content being created automatically from scrapers or some sort of automated background process (be it from RSS feeds, an FTP server or a web service). No idiots are involved in the creation of content when it’s imported into the system and you likely really want that content to appear in your system. This is where Group Validations come in to play.

Group Validations are callbacks which kick-in as a subset, rather than all validations running at once. You might want to make sure that a user enters the title for a blog post in your system, but you don’t really want such a check for when that blog post comes in off of your RSS scraping system. Maybe you’d send those imported blog posts into a holding pen somewhere so that they can be rescued later, rather than preventing their save and never importing them in at all.

With ActiveRecord, if you declare a validates_presence_of on :title, that’s it - game over. The only way to bypass that validation is to save_without_validations and that skips all of your validations, rather than just this one.

But with DataMapper and it’s use of Validatable, you can check for the validity of an object depending on the circumstance you’re in. Here’s what that blog post model would look like if we wanted to validate blog posts by idiots, but not from our not-so-idiotic scrapper:

If this shit doesn’t work, consider it pseudo-code. If it does work, I’m a badass (quoted! -bj)

class Post
  include DataMapper::Persistence

  property :title, :string, :length => 0..255
  property :body, :text
  property :original_uri, :string, :length => 0..255
  property :created_at, :datetime
  property :can_be_displayed, :boolean, :default => false

  # user creation
  validates_presence_of :title,
    :groups => [:manual_entry, :display]
  validates_presence_of :body,
    :groups => [:manual_entry, :save, :display]

  # automated import
  validates_presence_of :original_uri, :groups => [:import]

  alias_method :__save, :save

  def save(context = :valid_for_save?)
    self.__save if self.send(context)
  end

  before_save do |instance|
    instance.can_be_displayed = true if instance.valid_for_display?
  end
end

Running quickly through my sample here, you’ll spot this odd :groups => [...] argument to a few of the validations. These define which group these validations are a part of. Validatable uses these to give us a few dynamic methods like valid_for_display? and valid_for_manual_entry?, which is the mechanism used to check if an instance is valid in one context or another.

Using a model setup like this, we could call @post.valid_for_manual_entry? when we need to verify that the idiot’s blog post can be added into our persistence layer safely. By overloading the save() method so that you pass in the group of validations to be executed (like :valid_for_manual_entry or :valid_for_import) to use when checking validity, we’ve effectively made it possible to choose which validation callbacks get fired and which don’t when saving out the model. NOTE: As I wrote this, a discussion was occurring in #datamapper on irc.freenode.net about making save() smarter so it respects grouping. Having to overload save() may not be needed in the future. As usual, your results may vary.

You’ll notice that I gave :body a validates_presence_of for both the :manual_entry group and the :save group. This means that, no matter what, that validation callback will kick in.

Also of note is the can_be_displayed boolean and the before_save manual callback I defined. Here, I’m helping myself out later on so that it’s easy to pull out valid blog posts that can be displayed without worrying about nil field values and such:

@posts = Post.all(
  :title.not => nil,
  :slug.not => nil,
  :order => 'created_at desc',
  :limit => 10
)

Becomes…

@posts = Post.all(
  :can_be_displayed => true,
  :order => 'created_at desc',
  :limit => 10
)

Pretty sexy, no? I can’t off-hand think of a way to get this functionality from ActiveRecord objects without manually mixing in Validatable and then fighting the battle between AR’s validations and Validatable’s validations. (I likely just need to think harder, though….maybe using single-table inheritance and then tacking on different validations for different subclasses…maybe?)

With the proper use of Group Validations, you end up saving yourself a lot of headache and work later on down the line, as well as supporting different scenarios where a post might be valid or might not–all without having to hack-around. How enterprise-y!

validates_true_for

The second outstanding feature of Validatable that I’m oh-so-in-love with is validates_true_for. Think of it like overloading valid? only capable of the full power of real validations behind it.

Say, for example, you’ve got an Event model that needs to make sure the end_date for the event is greater than the start_date. Wouldn’t want to break the laws of physics, so we’d do something like:

class Event < ActiveRecord::Base
  def valid?
    start_time < end_time
  end
end

Yup, it’s pretty simple with ActiveRecord. Just toss in our own valid? method and we’re done. With DataMapper, things are a touch more complicated, but overall not brutally difficult, and buy you the full power of Validatable validations:

class Event
  include DataMapper::Persistence

  # properties here

  validates_true_for :start_time, :logic => lambda {
    start_time < end_time
  }
end

So, a couple of things are going on here. First, we’re declaring the check to make sure start_time being less than end_time on the start_time property. We could have easily done it on the end_time property as well (take your pick). Secondly, we’re passing in a block (lambda) to be called when the check occurs. As long as our lambda returns true, we’re golden, the validation passes, and the object can be saved out to the persistence layer.

Say we want to do much more complicated logic, though.

class Event
  include DataMapper::Persistence

  # properties here

  scary_validation = lambda do
    # freakish logic here for particularly complicated
    # validations.
  end

  validates_true_for :start_time, :logic => scary_validation
end

Ruby’s support for closures is so damn-skippy that we can pass blocks of code around, assign them to variables, execute them later, totally forget about them, whatever we want. We don’t need to overload a method or anything.

Plus, we’ve elevated our advanced validation logic into a real Validation which we can then assign to groups, pass around from object to object, what-have-you. Had we just stuck this code in an overloaded .valid? method, we wouldn’t get our spiffy Group Validation stuff as well as a few other things that make Validatable so-very-sexy.

In Conclusion

Validations with Validatable (in DataMapper) are that much more powerful than their counter-parts in ActiveRecord, and therefore, you should switch to DataMapper (or Validatable)

Callbacks

(TODO) list of available call backs

Migrations

There is a rake task to migrate your models, but be warned migrations are currently destructive!

rake dm:auto_migrate    # Automigrates all models

You can also create databases from the Merb console (merb -i)

database.save(Posts)

This does the same job as the rake task migrating all your models.

DataMapper::Persistence.auto_migrate!

Migrations in the sense of AR migrations, don't exist yet, so you'll have to manually alter your database if you want to retain your data. There are plans however to include migrations in a future version of DataMapper.

CRUD

Creating

To create a new record, just call the method new on a model and pass it your attributes.

@post = Post.new(:title => 'My first post')

There is also an AR like method to find_or_create which attempts to find an object with the attributes provided, and creates the object if it cannot find it.

There is another way to create an object, which is to save it after the attributes have been set like this:

@post = Post.new
@post.attributes = {:name => 'Hi!',:body => 'This is just awesome!'}
@post.save

Reading (aka finding)

The syntax for retrieving data from the database is clean an simple. As you can see with the following examples.

Finding a post with one as its primary key is done with the following:

Post[1]

To get an array of all the records for the post model:

Post.all

NOTE: you can also do Post.find(:all), like the AR syntax but this is just a synonym for Post.all

To get the first post, with the condition author = 'Matt':

Post.first(:author => 'Matt')

When retrieving data the following parameters can be used:

#   Posts.all :order => 'created_at desc'              # => ORDER BY created_at desc
#   Posts.all :limit => 10                             # => LIMIT 10
#   Posts.all :offset => 100                           # => OFFSET 100
#   Posts.all :include => [:comments]

If the parameters are not found in these conditions it is assumed to be an attribute of the object.

You can also use symbol operators with the find to further specify a condition, for example:

Posts.all :title.like => '%welcome%', :created_at.lt => Time.now

This would return all the posts, where the tile was like 'welcome' and was created in the past.

Here is a list of the valid operators:

If you require a custom find, you can use SQL with the method, find_by_sql. This will return an array of Structs (which are read-only) with the result of the query.

Count

DM provides a count method to count the number of records of a model, you can also pass search conditions to count:

comment_count = Post[1].comments.count
Each

Each works like like expected iterating over a number of rows and you can pass a block to it. The difference between Comments.all.each and Comments.each is that instead of retrieving all the rows at once, each works in batches instantiating a few objects at a time and executing the block on them (so is less resource intensive). Each is similar to a finder as it can also take options:

Comments.each(:date.lt => Date.today - 20).each do |c|
    c.destroy!
end

Updating

Updating attributes has a similar syntax to ARs update_attributes:

Post.update_attributes(:title => 'Opps the title has changed!')
Post.save

Post will only update the attributes which it persists and have changed, so changing virtual attributes will require marking the object as dirty to force a save.

Destroying

You can destroy database records with the method destroy!, this work much like AR.

bad_comment = Comment[6]
bad_comment.destroy!

Should you want to delete all the records of a model, you can do the following:

Comment.delete_all

Routing

Routing is similar to rails

Strings/Regex

Hashes

Restful Routes

Nested Routes

URLs

(TODO) - Defining routes, and resources (TODO) - Nested routes (TODO) - Namespaces (TODO) - Show routes, merb.show_routes in merb's irb console (merb -i)

Controllers

filters

(TODO) - filters, how the chaining works and :throw

Merb filters are quite powerful, etc..

In Rails:

before_filter :find_post
after_filter

In Merb:

before :login_required, :exclude => [:index, :show]
after  :send_email, :only => :create

skip_before is used to skip a before filter

Note: it's exclude not except

(TODO) - how params get passed in controllers (TODO) - Exception Controller (TODO) - explain provides (TODO) - usecase for a part, explain what they are (possibly comments?) - or a side bar of some sorts (TODO) - Admin controller (TODO) - specify a layout (TODO) - rest (TODO) - content_type (TODO) - flash?

Views

(TODO) - form helpers (TODO) - mention you can use other template languages

Partials

Use the partial method to render a partial from the current directory. If you pass a hash as the second argument the contents will be made available as local variables in the partial.

partial :post, {:comments => @post.comments}

To display the latest posts on our blog's front page, we use the :with and :as arguments to render a collection.

partial :post, :with => @posts, :as => post

Mailers

(TODO) - sending mail (TODO) - mail templates in /views

Authentication

(TODO) - Rolling your own (TODO) - integrating RESTful auth (merbful)

Attachments

(TODO) - attachment_pu (TODO) - image resize/crop (TODO) - downloading

RSpec

way of writing human readable tests, build it mocking stubing not important, changes the language and syntax to its more readble. works fine no change in merb.

When using stubs with RSpec you can roughly categorise the methods you are going to use into two categories. On one side you have the sub! and should_receive methods which refine what methods you expect to be called with what parameters and potentially what they should return in the case of the test being run. On the other side you have assertions which test the output and value or variables. The should method is primarily used when asserting things.

What to test?

(TODO) - how to write good test and what should just trust works

Stories

(TODO) - finish stories section

RSpec Stories are use to replace the specification phase in requirements gathering, in the form of scenarios. so we have both a spec and a integration tests. human/customer readable.

Add this line to your app's init.rb:

dependency "merb_stories" if Merb.environment == "test"

Now generate your story:

merb-gen story mystory

Now run your story:

rake story[mystory]

Yes, you must include the square brackets, and you have to escape them.

Now fill out your story. There are some differences to Rails' versions. The best places to look for help are in the Merb code itself:

spec/public/test/controller _matchers _spec.rb
lib/merb-core/test/helpers
lib/merb-core/test/matchers
To start you off, here are the steps for a simple integration test:

steps_for(:homepage) do
  When("I visit the root") do
    @mycontroller = get("/")
  end
  Then("I should see the home page") do
    @mycontroller.should respond_successfully
    @mycontroller.body.should contain("Hello") 
  end    
end

Spec'ing Models

(TODO) - How to spec models, use example merb/dm test talk through them, mocking

Spec'ing Views

(TODO) - What they should test

For more information, check Merb's wiki

Spec'ing Controllers

Getting started

Testing controllers typically involves stubbing out some methods, making a fake request and then ensuring the right variables are assigned, exceptions are raised and views rendered.

A good start is testing the show action in our Posts controller.

class Posts < Application
  provides :html

  def show(id)
    @post = Post[id]
    render @post

    rescue DataMapper::ObjectNotFoundError
    raise NotFound
  end
end

Our first test will ensure that Post[1] is called when /posts/1 is visited, and when the post exists the response code is 200 OK.

describe Posts, "show action" do
  it "should find post and render show view" do
    Post.should_receive(:[]).with("1")
    get('/posts/1', :yields => :controller) do
      controller.stub!(:render)
    end
    status.should == 200
  end
end

The first should_receive ensures that Post[1] is called, we could mock out a Post instance to return here, but in this case we're only interested in it being called and not raising an exception.

Next we use the get method to make a request to the controller, the :yields option allows us to set what the get request returns. Here we want to grab the controller and then stub out the render method before the request is made. Anything inside you get method's block will be executed before the request is dispatched.

After the request has been dispatched, several methods are available to return the results from the request: body, status, params, cookies, headers, session, response and route. Note that these all just call the same method on controller (so status is the same as controller.status).

This test was fairly simple, and it's likely you won't need to such tests if your controllers are as simple as ours. But once you have more than a few lines in your controller, simple response status checks can be useful for ensuring the overall integrity of your app.

A more important test would be ensuring that a 404 is returned when the post cannot be found in the database. When Datamapper cannot find a record is raises Datamapper::ObjectNotFoundError. Merb has several useful exception classes which will set the correct status and then call the relevant action in your Exceptions controller. Raising NotFound will set the status to 404 and then call the not_found action, which can return a much nicer.

it "should return 500 if post doesn't exist" do
  post = mock(Post)
  Post.should_receive(:[]).with("1").and_raise(DataMapper::ObjectNotFoundError)
  get('/posts/1')
  status.should == 404
end

Unlike the last test there was no need for us to stub the render method because DataMapper::ObjectNotFoundError is raised before it is reached.

Testing multipart forms

(TODO: Make and example of uploading assets in the simple blog)

The multipart_post method allows you to include files in a fake request. There must however be an actual file to be opened and submitted. If you put the file in the same directory as your spec, use File.dirname(FILE) to ensure the full path is used.

If you are going to open the tempfile which is uploaded, remember to stub out File.open. Watch out though, if you use simply open instead of File.open it won't be the File.open you stubbed out. The other issue here is within the spec we have no way of knowing what the filename of the tempfile is, so we have to assume it's correct and use aninstanceof(String) so any filename is accepted.

(TODO: test code)

describe Posts, "create action" do 
    it "should receive file" do
      File.should_receive(:open).with(an_instance_of(String))
      multipart_post("/posts", {:image => File.open(File.join( File.dirname(__FILE__), "picture.jpg"))})
      controller.assigns(:filename).should == "picture.jpg"
    end
end

Your controller would look something like this.

class Posts < Application
    def create
        fp = File.open(params[:image][:tempfile].path)
        @filename = params[:image][:filename]
    end
end

More ways to dispatch a request

There are several other ways to dispatch a request in your test. Look at Merb's Wiki for more information

Caching

(TODO) - session cache (TODO) - Query/Mem cache

Check Merb's wiki for more information.

Gotchas

Merb

The Rails way The Merb way
script/server merb
script/console merb -i
script/generate merb-gen
redirect_to blog_path(@blog) redirect url(:blog, @blog)
respond_to provides :xml, :js, :yaml
format content_type
format.html only_provides :html
render :xml => @post render @post
render :file => ‘public/404.html’, :status => 404 raise NotFound
logger Merb.logger
before_filter before
render :partial partial
f.text_field :name text_control :first_name
RAILS_ENV Merb.environment

Freezing Gems

As Merb is spilt up into various gems, and it's hard to keep update with each one it's a good idea to freeze them into your application, so an update to one gem doesn't break your app.

The easiest way to freeze a gem is to add -i gems as a command line option to specify the location for the installed gem. And then add the gem as a dependency in your init.rb.

gem install aquarium -i gems

When running this command from the root of your merb application, it will install the gem inside the gem directory

If you want to freeze the version of the gem that you have installed which is from trunk, you'll need to find where your gems are located and pass that parameter to the gem install command.

gem environment gemdir

As I have installed Ruby via port my gem folder is located at /opt/local/lib/ruby/gems/1.8. To freeze the aquarium gem I have from trunk I would need to run:

gem install /opt/local/lib/ruby/gems/1.8/cache/aquarium-0.4.1/ -i gems

If you want to freeze merb itself you need to add this to your init.rb, then run the following:

require 'merb-freezer'

rake freeze:core
rake freeze:more
rake freeze:plugins

Once the merb gem is frozen, you can run merb with frozen-merb. If you want to update your frozen gem version, pass the update parameter to the rake task:

rake freeze:core UPDATE=true

DataMapper

(TODO) - DM / AR diffs

RSpec

Submitting a patch

http://www.gweezlebur.com/2008/2/1/so-you-want-to-contribute-to-merb-core-part-1

(TODO) - example patch

Diffs

(TODO) - where to send diffs

Docs

(TODO) - doc convention

Specs

(TODO) - write specs

Hacking Merb

(TODO) - Hacking Merb

Changing the directory structure

# Build the framework paths.
#
# By default, the following paths will be used:
# application:: Merb.root/app/controller/application.rb
# config:: Merb.root/config
# lib:: Merb.root/lib
# log:: Merb.root/log
# view:: Merb.root/app/views
# model:: Merb.root/app/models
# controller:: Merb.root/app/controllers
# helper:: Merb.root/app/helpers
# mailer:: Merb.root/app/mailers
# part:: Merb.root/app/parts
#
# To override the default, set Merb::Config[:framework] in your initialization file.
# Merb::Config[:framework] takes a Hash whose key is the name of the path, and whose
# values can be passed into Merb.push_path (see Merb.push_path for full details).
#
# ==== Note
# All paths will default to Merb.root, so you can get a flat-file structure by doing
# Merb::Config[:framework] = {}
# 
# ==== Example
# {{[
#   Merb::Config[:framework] = {
#     :view => Merb.root / "views"
#     :model => Merb.root / "models"
#     :lib => Merb.root / "lib"
#   }
# ]}}
# 
# That will set up a flat directory structure with the config files and controller files
# under Merb.root, but with models, views, and lib with their own folders off of Merb.root.
class Merb::BootLoader::BuildFramework < Merb::BootLoader
  class << self

    def run
      build_framework
    end

    # This method should be overridden in merb_init.rb before Merb.start to set up a different
    # framework structure
    # DOC: Yehuda Katz FAILED
    def build_framework
      unless Merb::Config[:framework]
        %w[view model controller helper mailer part].each do |component|
          Merb.push_path(component.to_sym, Merb.root_path("app/#{component}s"))
        end
        Merb.push_path(:application,  Merb.root_path("app/controllers/application.rb"))
        Merb.push_path(:config,       Merb.root_path("config"), nil)
        Merb.push_path(:environments, Merb.dir_for(:config) / "environments", nil)
        Merb.push_path(:lib,          Merb.root_path("lib"), nil)
        Merb.push_path(:log,          Merb.log_path, nil)
        Merb.push_path(:public,       Merb.root_path("public"), nil)
        Merb.push_path(:stylesheet,   Merb.dir_for(:public) / "stylesheets", nil)
        Merb.push_path(:javascript,   Merb.dir_for(:public) / "javascripts", nil)
        Merb.push_path(:image,        Merb.dir_for(:public) / "images", nil)        
      else
        Merb::Config[:framework].each do |name, path|
          Merb.push_path(name, Merb.root_path(path.first), path[1])
        end
      end
    end
  end
end

Deploying a Merb Application

"Yes, Rails scales just like everything else scale." - Ezra Zygmuntowicz

The most satisfying experience of building a web application is having others use it. Implementing a robust deployment plan is essential to ensure each release of your project goes off with out a hitch.

The Pieces

Subversion

Version control is an essential piece of any software development cycle. There are several options available, including CVS, Git, Subversion, and many more. This guide assumes you have either a Subversion or Git repo that holds your application.

Nginx

A proxy is required to handle incoming HTTP requests. Here there are several options including Apache, Lighttpd, Swiftiply, and many more. A favorite in the community for performance and simplicity has become Nginx, which we'll use for this example. Developed in Russia by Igor Sysoev, the goal of Nginx is to provide a lightweight, high performance web server.

Capistrano

Merb deployment's base lies with Capistrano. Originally developed to ease the process of pushing Rails applications into production, it has been improved upon to more generally provide automating tasks via SSH on remove servers. It can be used for software installation, application deployment, configuration management, and much more.

Mongrel

For our merb instances, there are a variety of Ruby web servers. The de facto standard has been Mongrel, which has also spawned improved stacks such as Thin and Swiftiply. It is extremely easy to change, so examples for all 3 will be provided.

Preparing Your Production Server

Nginx

The stable version of Nginx at the time of writing is 0.5.35 with the latest development of 0.6.29. There is also a branch that supports a newer balancer for handing out requests. For this example we'll compile the latest development version.

./configure --prefix=/usr/local --with-http_ssl_module

Create a configuration file, this example has been maintained by Ezra and is availble at http://brainspl.at/nginx.conf.txt

# user and group to run as
user  ez ez;

# number of nginx workers
worker_processes  6;

# pid of nginx master process
pid /var/run/nginx.pid;

# Number of worker connections. 1024 is a good default
events {
  worker_connections 1024;
}

# start the http module where we config http access.
http {
  # pull in mime-types. You can break out your config 
  # into as many include's as you want to make it cleaner
  include /etc/nginx/mime.types;

  # set a default type for the rare situation that
  # nothing matches from the mimie-type include
  default_type  application/octet-stream;

  # configure log format
  log_format main '$remote_addr - $remote_user [$time_local] '
                  '"$request" $status  $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

  # main access log
  access_log  /var/log/nginx_access.log  main;

  # main error log
  error_log  /var/log/nginx_error.log debug;

  # no sendfile on OSX
  sendfile on;

  # These are good default values.
  tcp_nopush        on;
  tcp_nodelay       off;
  # output compression saves bandwidth 
  gzip            on;
  gzip_http_version 1.0;
  gzip_comp_level 2;
  gzip_proxied any;
  gzip_types      text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;


  # this is where you define your mongrel clusters. 
  # you need one of these blocks for each cluster
  # and each one needs its own name to refer to it later.
  upstream mongrel {
    server 127.0.0.1:4000;
    server 127.0.0.1:4001;
    server 127.0.0.1:4002;
  }

  # the server directive is nginx's virtual host directive.
  server {
    # port to listen on. Can also be set to an IP:PORT
    listen 80;

    # Set the max size for file uploads to 50Mb
    client_max_body_size 50M;

    # sets the domain[s] that this vhost server requests for
    # server_name www.[engineyard].com [engineyard].com;

    # doc root
    root /data/ez/current/public;

    # vhost specific access log
    access_log  /var/log/nginx.vhost.access.log  main;

    # this rewrites all the requests to the maintenance.html
    # page if it exists in the doc root. This is for capistrano's
    # disable web task
    if (-f $document_root/system/maintenance.html) {
      rewrite  ^(.*)$  /system/maintenance.html last;
      break;
    }

    location / {
      # needed to forward user's IP address to rails
      proxy_set_header  X-Real-IP  $remote_addr;

      # needed for HTTPS
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect false;
      proxy_max_temp_file_size 0;

      # If the file exists as a static file serve it directly without
      # running all the other rewite tests on it
      if (-f $request_filename) { 
        break; 
      }

      # check for index.html for directory index
      # if its there on the filesystem then rewite 
      # the url to add /index.html to the end of it
      # and then break to send it to the next config rules.
      if (-f $request_filename/index.html) {
        rewrite (.*) $1/index.html break;
      }

      # this is the meat of the rails page caching config
      # it adds .html to the end of the url and then checks
      # the filesystem for that file. If it exists, then we
      # rewite the url to have explicit .html on the end 
      # and then send it on its way to the next config rule.
      # if there is no file on the fs then it sets all the 
      # necessary headers and proxies to our upstream mongrels
      if (-f $request_filename.html) {
        rewrite (.*) $1.html break;
      }

      if (!-f $request_filename) {
        proxy_pass http://mongrel;
        break;
      }
    }

    error_page   500 502 503 504  /500.html;
    location = /500.html {
      root   /data/ez/current/public;
    }
  }

  # This server is setup for ssl. Uncomment if 
  # you are using ssl as well as port 80.
  server {
    # port to listen on. Can also be set to an IP:PORT
    listen 443;

    # Set the max size for file uploads to 50Mb
    client_max_body_size 50M;

    # sets the domain[s] that this vhost server requests for
    # server_name www.[engineyard].com [engineyard].com;

    # doc root
    root /data/ez/current/public;

    # vhost specific access log
    access_log  /var/log/nginx.vhost.access.log  main;

    # this rewrites all the requests to the maintenance.html
    # page if it exists in the doc root. This is for capistrano's
    # disable web task
    if (-f $document_root/system/maintenance.html) {
      rewrite  ^(.*)$  /system/maintenance.html last;
      break;
    }

    location / {
      # needed to forward user's IP address to rails
      proxy_set_header  X-Real-IP  $remote_addr;

      # needed for HTTPS
      proxy_set_header X_FORWARDED_PROTO https;

      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect false;
      proxy_max_temp_file_size 0;

      # If the file exists as a static file serve it directly without
      # running all the other rewite tests on it
      if (-f $request_filename) { 
        break; 
      }

      # check for index.html for directory index
      # if its there on the filesystem then rewite 
      # the url to add /index.html to the end of it
      # and then break to send it to the next config rules.
      if (-f $request_filename/index.html) {
        rewrite (.*) $1/index.html break;
      }

      # this is the meat of the rails page caching config
      # it adds .html to the end of the url and then checks
      # the filesystem for that file. If it exists, then we
      # rewite the url to have explicit .html on the end 
      # and then send it on its way to the next config rule.
      # if there is no file on the fs then it sets all the 
      # necessary headers and proxies to our upstream mongrels
      if (-f $request_filename.html) {
        rewrite (.*) $1.html break;
      }

      if (!-f $request_filename) {
        proxy_pass http://mongrel;
        break;
      }
    }
    error_page   500 502 503 504  /500.html;
    location = /500.html {
      root   /data/ez/current/public;
    }
  }
}

Building a Deployment Recipe with Capistrano

Install Capistrano.

gem install capistrano

Navigate to your Merb repository directory and run the capify command to create the skeleton for your deployment recipe.

$ capify .
[add] writing `./Capfile'
[add] writing `./config/deploy.rb'
[done] capified!

Tailor your deploy.rb to meet the requirements of your application

set :application, "YOUR_APPLICATION_NAME"

# Set the path to your version control system (Subversion assumed)
set :repository, "http://something.com/svn/yourapplication/trunk"

# Set your SVN and SSH User
set :user, "your_ssh_user"
set :svn_user, "your_svn_user"
#Set the full path to your application on the server
set :deploy_to, "/PATH/TO/YOUR/#{application}"

#Define your servers
role :app, "your.appserver.com"
role :web, "your.webserver.com"
role :db, "your.databaseserver.com", :primary => true

desc "Link in the production extras and Migrate the Database ;)"
task :after_update_code do
  run "ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml"
  run "ln -nfs #{shared_path}/config/merb.yml #{release_path}/config/merb.yml"
  run "ln -nfs #{shared_path}/log #{release_path}/log"
  #if you use ActiveRecord, migrate the DB
  #deploy.migrate
end

desc "Merb it up with"
deploy.task :restart do
  run "cd #{current_path};./script/stop_merb"
  run "cd #{current_path};env EVENT=1 merb -c 3"
# If you want to run standard mongrel use this:
# run "cd #{current_path};merb -c 4"
end

#Overwrite the default deploy.migrate as it calls: 
#rake RAILS_ENV=production db:migrate
#desc "MIGRATE THE DB! ActiveRecord with Merb Love"
#deploy.task :migrate do
#  run "cd #{release_path}; rake db:migrate MERB_ENV=production"
#end

Use Capistrano to initiate the environment, setting up the necessary directories on the server.

$ cap deploy:setup

Next, install the Gems you need on the Production server.

#Ensure you have the latest version of the gem system
sudo gem update --system

sudo gem install merb
sudo gem install rspec

# For ActiveRecord
sudo gem install merb_activerecord

# For DataMapper
sudo gem install datamapper
sudo gem install do_mysql

# For Evented Mongrel
sudo gem install swiftiply

# For Standard Mongrel
sudo gem install mongrel

# For Thin
sudo gem install thin

Create the directories and files that will be linked in.

mkdir /YOURDEPLOYPATH/shared/config
touch /YOURDEPLOYPATH/shared/config/database.yml
touch /YOURDEPLOYPATH/shared/config/merb.yml

Edit the .yml files to your liking and then be sure to create your database in MySQL.

Deploy your app:

cap deploy

You should now have your application deployed to your server with 3 Mongrel instances running being proxied to by Nginx.

Advanced Recipes

Introduction

So you've come this far and are feeling pretty confident about your Merb abilities. This chapter is all about sharing some top tips and code examples that you may find to be time savers in real life situations.

Contributing

Most of the examples here are taken from real world projects or blog posts. We are always looking for contributions so if you have something to share let us know.

Loading The Merb Environment Outside Of Merb

Sometimes you will need to load up a Merb environment (and it's frozen gems) for things such as RSpec stories or cron tasks. This little snippet does just that. 

env = ENV['MERB_ENV'] || 'test'

require 'rubygems'
Gem.clear_paths
Gem.path.unshift(File.join(File.dirname(__FILE__), "gems"))

require 'merb-core'
Merb.load_dependencies(:environment => env)

require 'spec'
Merb.start_environment(:testing => true, :adapter => 'runner', :environment => env)

Here we have assumed that the script is running from the root of your app. In practice though you will most likely want it in a different location so be sure to adjust the Gem.path accordingly.

Freezing Gems

This recipe was contributed by Michael Klishin

One thing Merb community gets right is gems bundling. config/init.rb in Merb apps has the following magic line that shows idea of independency of the application from environment it runs in is baked into the core:

Gem.path.unshift(Merb.root / "gems")

Yay, no need to reinvent Gem plugin. It is that simple. A note here: to actually get up and running with Merb and Edge ActiveSupport and ActiveRecord bundled under /gems directory you have to specify installation directory with -i option:

sudo gem install -i ~/dev/workspace/some-merb-application/gems

This sets up a directory structure RubyGems' custom require expects to see.

Another thing Merb community gets right is freezing of Merb itself. I want to run this app on Edge Merb, I want to be independent from what Merb gems are on the box I deploy to. Rails does it by exporting a tarball since it moved to Git so you absolutely cannot track the tree you currently use.

In Merb there is a nice plugin merb-freezer. What it does is using either gems unpack strategy or Git submodules strategy if you use Git for your Merb application. This is very cool. Git submodules is like Subversion's externals but adapted to distributed nature of Git and packed with features Subversion lacks.

With git modules freezing you can track what commit hash app is frozen to, what recent log messages say, update it one by one or all at once, use the branch you want from repository as a submodule, see meaningful submodules state summary. Compare this to tarballs management.

To use merb freezer all you have to do is to install merb-freezer from merb-more repo and include a line

require 'merb-freezer'

into your config/init.rb. Then run

rake freeze:core

if you want to use Git submodules or

MODE=gems rake freeze:core

if you want to go with installed gems.

freeze:more and freeze:plugins do freezes of merb-more and merb-plugins, respectively.

If you choose submodules, make sure you start with a clean branch. Submodules meta information file (.gitmodules) and frameworks directory where Merb is frozen to have to be commited after run of Rake task that does the freeze.

To update Merb use the same Rake task with UPDATE env variable set to true. To see what commits application is frozen to, use

git submodule status

To see N recent commits in Merb core installed as a submodule use

git submodule summary -n <N> frameworks/merb-core

Plugins

(TODO) - Go over some cool merb plugins