The Modest Rubyist

Recently, I have been spending some of my extra time writing a little command-line application for interacting with the O'Reilly Bookworm Public API. I remembered reading about Aruba, a Cucumber ‘add-on’ similar to cucumber-rails but for the command-line, on TheChangeLog. Aruba’s own step definitions are self-documented in its own features and scenarios but I will cover some of the basics here. I will also cover some useful parts of the API defined by Aruba that you can use to write your own steps.

Simply put, Aruba is a gem that defines steps and helpers you can use to drive out your command-line application. You can use Aruba to test not only applications written in Ruby but also C, Perl or any other script that you can run from your command prompt. For Ruby applications Aruba also supports RVM so you can test your applications in specific environments. Use Aruba to control not only the Ruby version but also what gems are available to the system. If your developing a command-line app and use Cucumber, Aruba is for you.

If you would like to get started with Aruba install it with

> gem install aruba

You will need Cucumber installed as well to get going.

Starting your Aruba Vacation (Setting up the Environment)

Although you can use Aruba to test your C or Java application, this is a Ruby blog so I will discuss how to use it while writing an executable contained in a Ruby gem. I setup my gem directory like so:

- some_gem/
  - bin/
    -executable
  - lib/
    - some_gem.rb
    - some_gem/
  - features/
    - support/
      - env.rb
    - step_definitions/
      - some_steps.rb
      - more_stems.rb
    - feature1.feature
    - feature2.feature

By convention, I use the file features/support/env.rb to get Aruba up and running in our Cucumber environment. The first and only thing we need to do, for now, is include the library. Add

require 'aruba'

to env.rb. Make sure everything is ok by running cucumber on the empty features directory.

> cucumber 
0 scenarios
0 steps
0m0.000s

Running Executables

For the purposes of this post lets continue to work with the previous example and assume we are to drive out the functionality of the application bin/executable. When we write our scenarios we would like to address the running of this application, possibly with various options, as we would from the command-line. We would, also, like to test the output our application writes to STDOUT. The first few steps you need to know about are defined as,

When /^I run "(.*)"$/ do |cmd|
  run(unescape(cmd))
end

Then /^I should see "([^\"]*)"$/ do |partial_output|
  combined_output.should =~ compile_and_escape(partial_output)
end

Then /^I should see:$/ do |partial_output|
  combined_output.should =~ compile_and_escape(partial_output)
end

Then /^I should not see "([^\"]*)"$/ do |partial_output|
  combined_output.should_not =~ compile_and_escape(partial_output)
end

Then /^I should not see:$/ do |partial_output|
  combined_output.should_not =~ compile_and_escape(partial_output)
end

For our purposes we would write something like

Scenario: Running our Executable
  When I run "executable --someoption"
  Then I should see:
    """
    Hello World!
    This is my executable
    """
  And I should not see "something I don't wanted printed"

If you run this scenario you will probably see a similar error to this one in the output, even if you have created the file bin/executable.

 Then I should see:                 # 1.7/lib/aruba/cucumber.rb:66
   """
   Hello World!
   This is my executable
   """
  expected: [removed this for presentation purposes],
       got: "\n------------------------------------------\
    ----------------------------\nsh: executable: not found\n" \
    (using =~)

Aruba allows you to play with files as well. For that reason it must do things in a temporary place. As a result the load path isn’t as straight forward as you might or I did assume. Changing

 When I run "executable --someoption"

to

When I run "bin/executable --someoption"

will not work either. You could install the gem each time but that would be a hassle. Instead, we need to rewrite one of Aruba’s helper methods to work with our executable. I stole the following idea from RSpec-2’s aruba setup. For our example we need to add this so we can start going from red to green.

# features/support/env.rb
module ArubaOverrides
  def detect_ruby_script(cmd)
    if cmd =~ /^executable /
      "ruby -I../../lib -S ../../bin/#{cmd}"
    else
      super(cmd)
    end
  end
end

World(ArubaOverrides)

This code is pretty straight forward. We define a module ArubaOverrides; you could have named it anything. Aruba defines a helper, detect_ruby_script, which we override. This method is called by another helper, run (see the step definitions above). If the command we run starts with ‘executable’, in this case, we run it explicitly using the -S option and add our library to the load path with -I. Finally, we use World(ArubaOverrides) to include the helper in the Cucumber World.

Now if we write the following little script,

# bin/executable
#!/usr/bin/ruby
puts "Hello World!"
puts "This is my executable"

and do a little permission modification,

> chmod +x bin/executable

we should be in the green when we run our scenario again.

Aruba Tags

Aruba uses four special tags to make running scenarios a bit more detailed or to help with debugging. The tags and their side-effects are,

@announce-cmd    # prints a command called by 'run'
@announce-stdout # prints the final output written to STDOUT
@announce-stderr # prints the final output written to STDERR
@announce        # all of the above

Adding the tag @announce-cmd to our scenario would result in the following output when run.

When I run "executable --someoption"
$ /usr/bin/ruby1.8 -I../../lib -S ../../bin/executable --someoption

Some more useful steps

If you would like to include steps about your application’s exit status Aruba makes it easy. Here are some scenario’s from Aruba’s own features:

Scenario: exit status of 0
  When I run "ruby -h"
  Then the exit status should be 0

Scenario: non-zero exit status
  When I run "ruby -e 'exit 56'"
  Then the exit status should be 56
  And the exit status should not be 0

There is also a step definition that helps test an expected successful exit. The first step will pass and the second will fail.

Scenario: Success
  When I successfully run "ruby -e 'exit 0'"
  And I successfully run "ruby -e 'exit 10'"

Some other useful exit status related steps are defined as,

Then /^the exit status should be (\d+)$/ do |exit_status|
  @last_exit_status.should == exit_status.to_i
end

Then /^the exit status should not be (\d+)$/ do |exit_status|
  @last_exit_status.should_not == exit_status.to_i
end

Then /^it should (pass|fail) with:$/ do |pass_fail, partial_output|
  Then "I should see:", partial_output
  if pass_fail == 'pass'
    @last_exit_status.should == 0
  else
    @last_exit_status.should_not == 0
  end
end

If you would like to test the output written to standard out you can use the I should (not) see steps we used above. You can also test STDOUT and STDERR with,

Then /^the stderr should contain "([^\"]*)"$/

Then /^the stderr should not contain "([^\"]*)"$/

Then /^the stdout should contain "([^\"]*)"$/

Then /^the stdout should not contain "([^\"]*)"$/

I mentioned above that Aruba lets you work with temporary files. Here are some more examples right from Aruba’s own feature set.

Scenario: create a dir
  Given a directory named "foo/bar"
  When I run "ruby -e \"puts test ?d, 'foo'\""
  Then the stdout should contain "true"

Scenario: append to a file
  Given a file named "foo/bar/example.rb" with:
    """
    puts "hello world"
    """
  When I append to "foo/bar/example.rb" with:
    """
    puts "this was appended"
    """
  When I run "ruby foo/bar/example.rb"
  Then I should see "hello world"
    And I should see "this was appended"

I mentioned Aruba has support for RVM and control over gems installed when testing your script. The Aruba README has some great examples, so I point you there. You can also check out all of the defined steps here.

Writing your own steps with the Aruba API

If you need something more than the steps defined for you by Aruba you can use its API to write your own. You have already seen how Aruba uses its helpers in its own steps, above. I will address some of the useful ones.

The run(cmd) helper runs a command passed to it. This string should be the same as if you were running it from your command prompt. run uses IO.popen to run your command as a sub-process, storing the results of STDOUT and STDERR. You can check these variables in your steps. They are named @last_stdout and @last_stderr, respectively. @last_exit_status, also, gets defined in this helper.

You can move around directories with the helper cd(dir). This simulates calling ‘cd’ from a prompt. You can create a directory, with create_dir(dir_name), before changing into it. If you would like to create or append to a file use the helpers create_file(filename, file_content) or append_to_file(filename, file_content). Don’t feel the need to create the entire path to a temporary file, however. create_file does this for you with FileUtils#mkdir_p.

check_file_presence(paths, expect_presence) is a useful one for your ‘Then’ steps. It loops through the array of given paths. If expect_presence takes on a true value, it will pass if each path given points to a file. If expect_presence is false, it will pass, otherwise.

create_dir('some/dir') cd('some/dir')
create_file(File.expand_path('~/.dotfile', 'someprop: some_val\n')
check_file_presence(['file1', 'dir/file2'], true)

One final useful helper you can use in your tests is check_file_content(file, partial_content, expect_match). This helper passes if the contents of file, a path, matches (using regular expressions) the given partial_content when expect_match is true. If expect_match takes on a false value, the helper passes when the partial_content is not contained in file.

Well, I hope you enjoyed our little vacation to Aruba as much as I did. If you already use Cucumber for your code/gems or Rails applications, using Aruba is a nice addition to the toolbox. There are some things to note, however. Remember, Aruba runs your code in a sub-process. This changes a few things. For example, even though you should not be stubbing in Cucumber, if you need to, don’t be surprised when a stub doesn’t work when a method is called inside your executable. This is because your code is run in an entirely separate context from the Cucumber runner. It is important to take this into consideration when writing your own acceptance tests.