Capybara with Given/When/Then steps in acceptance testing

Capybara with Given/When/Then steps in acceptance testing.

Many of us use cucumber to write acceptance tests.
We must to say it’s very nice tool that allows you to write features in English plain text.
The main goal is provide interface that should be clear for Customer(non-programmer) and Developer.

Cucumber example:

Feature: User signup
  As a user
  I want to sign in
  So I can use service features
 
  Background:
    Given user with "jack@daniles.com" email and "qwerty" password
 
  Scenario: Signing in with correct credentials
    When I go to sign in page
    And I fill in "email" with "jack@daniles.com"
    And I fill in "password" with "qwerty"
    And I click "Login" button
    Then I should see "Welcome, jack@daniles.com!"
 
  Scenario: User tries to sign in with incorrect password
    When I go to sign in page
    And I fill in "email" with "jack@daniles.com"
    And I fill in "password" with "bla"
    And I click "Login" button
    Then I should see "Invalid credentials"

It’s very readable. But cucumber has one big downside:

Steps maintenance

All steps are global and a instance variable from one step is accessible in another step.
In big project it can easy lead to mess.
Also you can read some rants about cucumber:

• Does cucumber suck?
• Why Cucumber sucks and I still love it
• Why Steak (over Cucumber)

Previously we also use cucumber as tool for user stories.
Lately we found that our customers NEVER read cucumber features.
So do we need it anymore? Let’s check alternatives.

As you probably know, cucumber use internally Capybara for steps implementation. And there is even more – latest version of Capybara comes with a built in DSL for creating descriptive acceptance tests!

So we decided to switch to pure rspec+capybara testing.
We moved our features to spec/acceptance directory.
Let’s look at rspec+capybara example.

RSpec+Capybara example

$ cat spec/acceptance/signup_feature_spec.rb

feature "User signup" do
  background do
    @user = Factory(:user, :email => 'jack@daniles.com', :password => 'qwerty')
  end
 
  scenario "Signing in with correct credentials" do
    page.visit "/sessions/new"
    page.fill_in "email", :with => "jack@daniles.com"
    page.fill_in "password", :with => "qwerty"
    page.click_button "Login"
    page.should have_content("Welcome, jack@daniles.com!")
  end
 
  scenario "User tries to sign in with incorrect password" do
    page.visit "/sessions/new"
    page.fill_in "email", :with => "jack@daniles.com"
    page.fill_in "password", :with => "bla"
    page.click_button "Login"
    page.should have_content("Invalid credentials")
  end
end

Thus approach has one big benefit:

Feature description and feature implementation are located together.

You don’t need anymore to jump between feature and step files to guess where is implementation for a step.
It’s nice but there are also two downsides.

First downsides is that RSpec documentation output will be too poor and it’s not enough informative:

$ rspec -fs -c spec/acceptance/signup_feature_spec.rb

User signup
  Signing in with correct credentials
  User tries to sign in with incorrect password

Finished in 13.55 seconds
2 examples, 0 failures

Second downside is that scenario does not have anymore exact boundary blocks or steps.
It makes harder to understand the scenario flow.

Real user story often has big scenarios and we realized that cucumber steps were cool.

There were two ways:

• switch back to cucumber
• go further and invent

Railswarians never go back

Our Solution

If you decided to switch from Cucumber to Capybara acceptance testing try rspec-example steps gem.

With this gem you may use Given/When/Then steps into rspec example!

RSpec+Capybara+Steps example

feature "User signup" do
  background do
    @user = Factory(:user, :email => 'jack@daniles.com', :password => 'qwerty')
  end
 
  Steps "Signing in with correct credentials" do
    When "I go to sign in page" do
      page.visit "/sessions/new"
    end
    And "I fill in right email" do
      page.fill_in "email", :with => @user.email
    end
    And "I fill in  right password" do
      page.fill_in "password", :with => "qwerty"
    end
    And "I click login" do
      page.click_button "Login"
    end
    Then "I should see greeting" do
      page.should have_content("Welcome, #{@user.email}!")
    end
  end
 
  Steps "User tries to sign in with incorrect password" do
    When "I go to sign in page" do
      page.visit "/sessions/new"
    end
    And "I fill in right email" do
      page.fill_in "email", :with => @user.email
    end
    And "I fill in wrong password" do
      page.fill_in "password", :with => "bla"
    end
    And "I click login" do
      page.click_button "Login"
    end
    Then "I should see error" do
      page.should have_content("Invalid credentials")
    end
  end
end

And documentation output will be:

$ rspec -fs -c spec/acceptance/signup_feature_spec.rb

User signup
  Signing in with correct credentials
    When I go to sign in page
    And I fill in right email
    And I fill in right password
    And I click login" do
    Then I should see greeting
  User tries to sign in with incorrect password
    When I go to sign in page
    And I fill in right email
    And I fill in wrong password
    And I click login
    Then I should see error

Finished in 13.55 seconds
2 examples, 0 failures

Now it’s much better. Isn’t it?

RSpec Example Steps features

• Does not hack RSpec Example. It only extends it.
• Brings Given/When/Then/And/But steps support into rspec example block that allow you to better organize scenario
• Adds shared_steps DSL
• Adds ability to mark step as pending
• Reports with color about passed/pending/failed step into documentation formating output

Installation

$ gem install rspec-example_steps

Usage

See README for more details.

Just add to spec/spec_helper.rb

require 'rspec/example_steps'

And enjoy Given/When/Then step usage!

References

• rspec-example_steps
• cucumber
• rspec-steps

Accessing application session in capybara

Trouble

There are some cases when you may need to access application session in your tests:

• User signup or registration flow is too long.
• Application use another backend and store result into session.

In each story you must repeat the same steps. You can actually use some shared code but it does not speed-up tests anyway.

Nowadays if you want to access session in cucumber/capybara acceptance test you are in trouble.

Capybara’ README says:

  Access to session and request is not possible from the test ...

And it’s true

Rescue

So is it actually possible to access session? Actually yes

Instead of trying to hack capybara you may just extend your application in test environment!
Just add some code that modify session according to given request parameters.

If you use rack based application then your are lucky –
try rack_session_access gem.

If it’s another application you need implement concept yourself

This gem should work with any rack application. We covered it with acceptance testing against:

• rack builder application
• sinatra application
• rails3 application

Usage

  $ gem install rack_session_access

See README for usage examples.

Rails + Rspec + Capybara example

Add to spec/spec_helper.rb:

require 'capybara/rspec'
require 'rack_session_access/capybara'
 
Rails.application.config do
  config.middleware.use RackSessionAccess::Middleware
end

And use set_rack_session helper in acceptance test:

require 'spec_helper'
 
feature "My feature" do
  background do
    @user = Factory(:user)
  end
  scenario "logged in user goes to profile page" do
    # read your authorization engine manual how it store user into session
    page.set_rack_session(:user_id => @user.id)
    page.visit '/profile'
    ...
  end
end

Conclusion

Enjoy faster testing!

References

• capybara
• rack_session_access

Testing gem integration with multiple ruby frameworks

Preface

When you develop some cool gem that should work in different ruby frameworks
you definitely should write acceptance tests.

Nowadays it’s pretty easy to do with RSpec+Capybara.

Your goal

Assume you develop MyRackMiddleware gem that should work in Sinatra and Rails application.

Your actions

Create gem layout

To create layout use well-known bundle gem command:

$ bundle gem my_rack_middleware

It’s generate something like:

├── lib
│   ├── my_rack_middleware
│   │   └── version.rb
│   └── my_rack_middleware.rb
├── Gemfile
├── my_rack_middleware.gemspec
└── Rakefile

Add to Gemfile :

source "http://rubygems.org"
 
gemspec
 
gem 'rspec'
gem 'capybara'
 
gem 'sinatra'
gem 'rails'

Now start use BDD Before start implementation we need two sample applications.
Let’s put they into apps directory.

Creating Sinatra test application

Thus Sinatra is simple framework it’s trivial to create test application:

$ cat apps/test_sinatra_app.rb

require 'sinatra'
class TestSinatraApp < Sinatra::Base
  enable :sessions
 
  get '/login' do
    body "Please log in"
  end
 
  post '/login' do
    session[:user_email] = params[:user_email]
    redirect to('/profile')
  end
 
  get '/profile' do
    if user_email = session[:user_email]
      body "Welcome, #{user_email}!"
    else
      redirect to('/login')
    end
  end
end

Creating Rails3 test application

When you think that with rails it will be hard then your are wrong
Thanks to rails3 design its also rather simple! Let’s see:

$ cat apps/test_rails_app.rb

require 'rails'
require 'action_controller/railtie'
 
module TestRailsApp
  class Application < Rails::Application
    config.secret_token = '572c86f5ede338bd8aba8dae0fd3a326aabababc98d1e6ce34b9f5'
 
    routes.draw do
      get  '/login'   => 'test_rails_app/sessions#new'
      post '/login'   => 'test_rails_app/sessions#create'
      get  '/profile' => 'test_rails_app/profiles#show'
    end
  end
 
  class SessionsController < ActionController::Base
    def new
      render :text => "Please log in"
    end
 
    def create
      session[:user_email] = params[:user_email]
      redirect_to '/profile'
    end
  end
 
  class ProfilesController < ActionController::Base
    def show
      if user_email = session[:user_email]
        render :text => "Welcome, #{user_email}!"
      else
        redirect_to '/login' 
      end
    end
  end
end

So you have whole rails application in one file. Nice!

Write integration tests

Now is most interesting part of this article.
Capybara gem itself has rspec helpers that allows you to write rspec examples
in way of acceptance testing (aka feature and scenario).

Actually internally feature is just alias to ExampleGroup (describe) and scenario is alias to Example (it).
But this syntax follows you to think different.

Put in spec/spec_helper.rb

require 'capybara/rspec'
 
require 'rack_session_access'
require 'rack_session_access/capybara'
 
# load test applications
Dir[File.expand_path('../../apps/*.rb', __FILE__)].each do |f|
  require f
end
 
# configure sinatra application to use MyRackMiddleware
TestSinatraApp.configure do |app|
  app.use MyRackMiddleware
end
 
# configure rails application to use MyRackMiddleware
TestRailsApp::Application.configure do |app|
  app.middleware.use MyRackMiddleware
end

Write acceptance spec for example in spec/middleware_spec.rb :
Follow DRY principle and use shared_examples!

require 'spec_helper'
 
shared_examples "common scenarios" do
  scenario "test application itself" do
    page.visit "/login"
    page.should have_content("Please log in")
 
    page.visit "/profile"
    page.should have_content("Please log in")
  end
 
  scenario "MyRackMiddleware related scenario ..." do
    ...
  end
end
 
feature "MyRackMiddleware", %q(
  As ROLE
  I want FEATURE
  So I have BENEFIT
) do
 
  context "with Sinatra application" do
    background do
      Capybara.app = TestSinatraApp
    end
 
    include_examples "common scenarios"
  end
 
  context "with Rails application" do
    background do
      Capybara.app = TestRailsApp::Application
    end
 
    include_examples "common scenarios"
  end
end

Run tests

Before implementation run specs

$ bundle exec rspec -fs -c spec/middleware_spec.rb

Now you may add scenarios that cover your MyRackMiddleware influence on applications and start your middleware implementation!

Conclusion

We hope this article will help you to write better tests for your gems.

References

• capybara
• rspec

PivotalBooster – Keep your stories up to date!

The Railsware company is proud to show you our brand new free product – PivotalBooster.

The PivotalBooster is a simple and fast Mac OS X client for the popular Pivotal Tracker service. Railsware put great efforts to make your work with Pivotal Tracker more comfortable and faster. PivotalBooster features allow you concentrate on all the stories you have, get all the information you need, be always up to date, and more.

We have lots of ideas and features to implement in future releases and we need your active help to make our product even better. We’d appreciate any comment or feedback as your opinion is very important for us. Please feel free to share your thoughts!

For sure you can also follow us on Twitter and like us on Facebook to get more information and news.

Read the rest of this entry »

Mailtrap: Save your customers from getting test emails.

We assume you’re a web developer, QA engineer, product manager, etc — somebody who’s doing internet software projects. Have you ever needed to test e-mail notifications in your web application? Today we’ll speak about common issues while testing e-mail notifications and best solutions for that issues.

Common issues with manual e-mail notifications testing

We believe most critical issue here is accidentally sending dummy e-mails to real customers while testing web applications at staging/development servers. But let’s start with easier once at first.

Let’s say you’re doing users registration where every new users gets registration confirmation e-mail. In order to register yet another user for testing purposes you need to have new unique email address that should be registered somewhere. That’ll become problematic unless you’ll use service like Mailinator which seems good enough as a solution for our problem.

Now let’s make things a bit harder. Imagine our web application became successful and now we need to send weekly e-mail notifications to our users: a bit different for different users. How will you test that to know which e-mails were sent to which customers? Will you create tons of virtual users with tons of mail boxes and check each one? But that’s just so stupid and time-consuming!

In real life things are even worse. In my experience it is usually a good idea to copy production database to your staging servers or development machines for test purposes — so you’ll work with a real data instead of something synthetic. But this also brings you a risk of sending test e-mails to the real customers from your development/staging environment. My personal score is about 2500 e-mails sent by mistake at a time. I know people whose personal record is more than 2 millions emails sent by mistake! Share your personal fuck up stories in comments

Existing solutions for keeping your customers from test e-mails

If you’re lazy or just don’t have enough time, you may skip this chapter and just scroll down to Mailtrap: our solution to common problem because our solution is really much better that others, believe me!

So which possible solutions are most popular?
Read the rest of this entry »

Happy 5th Birthday to Us!

All HPB pictures on Flickr

Railsware at Rubyshift and RubyC 2011 conferences

During last month railswarians took a part in a really great Ruby events: Rubyshift in Dnepropetrovsk, Ukraine and RubyC in Kyiv, Ukraine.

RubyShift October 22-23 2011

Conference gathered more than 250 participants and speakers from all over the world including Pat Allan (@pat), Sven Fuchs (@svenfuchs), Josh Kalderimis (@joshkalderimis), Jon Leighton (@jonleighton), Thorben Schröder.

This is the first time we decided to film such events and not only because 3 Railswarian heroes (Sergey Korolev, Sergey Boiko, Alexander Mishyn) had cool speeches on RubyShift stages. We really wanted to share good content with Ruby community and interview various participants on specifics of ukrainian development culture, their plans, issues and to understand the way we all have to move on better.

Conference videos are available at Vimeo (Credits to Dmitry Larkin — our super-camera-man!). We will definitely share more speeches and interviews with Ruby stars soon.

More photos can be found at: our Flickr stream.

RubyShift  was organized great: venue and afterpartiies (yeah, everybody really enjoyed it) where smart, creative people met and socialized together.

Thanks again to organizers crew @beai, @ifesdjeen and @mariya_o5. Good luck to you, guys with more #RubyShifts in the future!

RubyC November 5-6 2011

RubyC conference taken a place in Kiev November 5-6.

Ruby guys like Joinas Nicklas (@jonicklas), Steve Klabnik (@steveklabnik) , Ryan Bigg (@ryanbigg), Darcy Laycock (@sutto) and Pat Allan (@pat) rocked during these weekend. Railswarian Bogdan Gusiev made a great speech about “Fighting Fat Models”.

And again we shoot some nice videos.

Finally, I’d like to wish Ukrainan developers to be more active, talk to guests, they are really willing to chat, that’s one of main reasons they’ve decided to attend these conferences. Talk to peopl like you during coffee breaks, afterparties. That’s the main event purpose: socialize, exchange, share your great ideas! Do not be scared of video cam as well It’s your opportunity to pass the message to a wide range of Ruby developers, business people, help in establishing community. Rock on guys!

Conclusion

Both conferences inspired us to be more active in introducing our work style, approaches, do more open source products and exchange innovations with community. Blogging, filming, interviewing participation and conferences organizintrg (yes, we have a lot of cool stuff to talk about!) and produce internal products so that we can make life of Ruby community and companies a bit easier — that’s our next step. And we will do it of course with lots of fun!

Stay tuned!

RubyC 2011 Conference Videos

We’ve shoot videos at recent european RubyC conference held in Kiev, Ukraine.

Wider than Rails

Author: Alexey Nayden (lightweight Ruby solutions)
Language: English

How to be awesome at Rails

Author: Ryan Bigg
Language: English

Read the rest of this entry »

CapHub – multiple applications deployment with Capistrano

Today we’ll talk about deploying multiple applications via single Capistrano project, managing its deployment configurations in a single place.
This approach is a must for projects with multiple applications deployment and recommended for any other project using Capistrano for deployment.
We’ve created a tool called Caphub for simplifying creation of multi-deployment Capistrano projects. Before showing it in details let’s review original Capistrano deployment methods: default and multi-stage.

Preface

We’re pretty sure you’re already familiar with basic capistrano setup. Let’s just repeat it briefly:

$ cd blog
$ gem install capistrano
$ capify .

Then you put project-specific configuration to config/deploy.rb

Another thing that people usually use for deployment is multistage extension originally written by capistrano author.

This extension allows you to specify particular configuration related to specific environments easily. For example, production or qa environments. Common configuration is located in config/deploy.rb and specific configuration in config/deploy/{stage_name}.rb So any of your tasks are always executed as:

$ cap CONFIGURATION_TASK NAMESPACE:TASK_NAME

E.g.:

$ cap production deploy:setup

In our opinion this approach works fine… even extremely fine! Thank you, Jamis Buck!

Decentralized multi-deployment approach

Following examples assume you’re working on large or just complex project that contains several applications. For example you use SOA approach. So deployment code is located in each of its applications.

There are serious disadvantages in this method:

1. Multiple directories: You must enter each project and start application deployment procedure there. So you either use cd command or open each project’s directory in a new terminal console.
2. Copy-n-Paste: Some shared configuration and Capistrano recipes are just copy-n-pasted between projects. It’s not DRY.
3. Deployment and Application mixing: We believe application code must be independent from deployment code — that is just different projects. So why would we put deployment code directly into main application repository? It makes a mess at least in SCM commit history.

As an alternative you may use centralized deployment approach.

Centralized multi-deployment approach

Let’s create another repository and call it mysite-deploy for example. Create repository layout similar to Capistrano multistage convention:

[mysite-deploy]$ tree --dirsfirst
.
├── config
│   ├── deploy
│   │   ├── billing
│   │   │   ├── production.rb
│   │   │   └── qa.rb
│   │   ├── core
│   │   │   ├── production.rb
│   │   │   └── qa.rb
│   │   └── customer
│   │       ├── production.rb
│   │       └── qa.rb
│   ├── keys
│   └── deploy.rb
├── recipes
├── Capfile
└── Gemfile

What is the benefit?

1. Centralized location: mysite-deploy is your deployment control repository now. All deployment code located in single place and you can easily support or refactor it.
2. Reuseable code: You don’t need to copy-n-paste Capistrano recipes or configuration files anymore. Such approach allows you to include shared configurations to certain subprojects. You can include external recipes via gems or put your own ones into recipes directory. You can easy configure/override particular recipe for certain configuration.
3. Security policy: You may restrict access to deployment project so only specific people/departments would be able to deploy the application. Or if you use ssh keys you can create yet another repository deployment-keys and add it as git submodule or symlink it. There are lots of ways how you may organize your security policy. Anyway it is much easier with centralized deployment project location.

Read the rest of this entry »

Posting delivery events to Google Calendar

Introduction

One day our client asked us about product deployment notification. Initially we wanted to use mail-list or create special skype chat, but it turned out to be unreliable and inconvenient:

• hard to filter notification emails
• hard to track deploment history
• newly added person can’t observe old events

So we “thinked different”© and made next simple solution.

Google Calendar

What’s about using Google Calendar as event storage? It has many useful features:

• stores any event history
• provides chronological event representation for free
• we can easily subscribe/unsubscribe to/from notification feed
• calendar can be easily customized in different ways(e.g. colors, size)
• you can embed calendar directly into any html-page via iframe
• you can access your calendar on any android/iphone device

Capistrano Calendar

We extracted our calendar recipes and released as
capistrano-calendar gem.

Read the rest of this entry »