Some thoughts on designing a test framework with Ruby

Here are some of my thoughts after actively working on a test framework in Ruby and scripting and running test scripts for the last couple of months.

Start writing tests first, and then grow your framework. I tried to design a framework before I had a solid idea of what the testing needs were. I had just joined the company, and didn't really understand the use cases and test procedures very well. So, I just made assumptions of what I thought would be good common code and what should be in a "good" test framework. These assumptions came from other test frameworks I had used or designed in the past. I felt like my experience would make up for my lack of product knowledge. In a lot of cases that was true, but I could have delved more into the testing side, instead of just the framework. Sometimes to test something, you just need a very simple script, and a sophisticated framework can just get in the way.

Don't overdesign features before you really need them. For example, I'm thinking of ripping out the logging stuff I implemented, because now it is annoying me. I had designed the test framework so that when you include the test harness class in the test class, a log will be made for any test run. Well, I found out I wasn't really reading the logs too much, I was just relying on the output from Ruby's Test::Unit class. And I hadn't implemented a cleanup mechanism to clear the logs after a certain age was reached. So, I'm constantly typing "rm -f *.log". I think it is wiser to not force the logging in a test class, and have the test developer decide what logging is necessary and the mechanism to do that.

The DRY (Don't Repeat Yourself) concept is great, but don't get too fancy, though, with code reuse and that magical metaprogramming (if you are using a dynamic language like Ruby, as I am). It probably doesn't matter much when you need to quickly crank out tests, since this is just testware, not shippable code. In most cases, I think repeating yourself a couple of times is okay, especially when you are trying to automate as much of your tests by the end of a sprint. But once you get to the magical number of 3, then you should think about your modularity and move your repeated code to a common library.

Don't get hung up about trying to implement tests in only one language. This is probably a good reason to not spend too much time working on a framework, because, you might need a different language or tool to perform a subset of your tests. For example, there are some things that Java is better at than Ruby and vice versa. When it comes to manipulating large amounts of XML, performance-wise, Java would be my choice over Ruby.

Using Ruby and ERB to dynamically create XML files from templates

Here is a simple way to dynamically create XML files from templates. Actually, it could be any type of file. I chose XML format, because I work with plenty of XML at work. And I use this technique in a test setup.

For our example template, here is a very basic XML file, called ERB_example.xml.

<email>
 <to><%= $name %></to>
 <from><%= $me %></from>
 <date><%= Time.now %></date>
 <subject><%= $hello %></subject>
 On your way home, please pick up the following from the store:
% $list.each do |thing|
     * <%= thing %>
% end
 <%= $signoff %>
</email>

As you can see, it has the basic elements of an email message, I have tags surrounded by <%= and %>. Anything inside these tags are treated like a Ruby expression and are evaluated. I also have two occurrences of just %. These are treated as the other tags if they are at the start of the line. (Note that I did not indent those lines!) This is useful for us because we cannot nest the <%= and %> tags. (Try it!)

Just to make things easier for this example, I am using global variables for most of these tokens. I also have an expression Time.now which will substitute the date and time. Now to our Ruby script, ERB_example.rb.

require 'rubygems'
require 'erb'

# set up some variables that we want to replace in the template
$hello = "Hola"
$me = "Tu mama"
$name = "Mi'jito"
$list = [ "milk", "eggs", "bread"]
$signoff = "Te quiero mucho."

# method update_tokens takes template_file, expecting globals
# to be set, and will return an updated string with tokens replaced.
# you can either save to a new file, or output to the user some
# other way.
def update_tokens(template_file)
 template = ""
 open(template_file) {|f|
   template = f.to_a.join
 }
 updated = ERB.new(template, 0, "%<>").result

 return updated
end

new_xml=update_tokens(Dir.getwd+"/ERB_example.xml")
puts new_xml

Note that in our third argument to ERB.new, we are telling ERB two things. First, the % says to process any % at the beginning of a line. Second, the <> says to omit any newline for lines beginning with <% and ending with %>.

So, our final output when we run this little script is as follows:

<email>
 <to>Mi'jito</to>
 <from>Tu mama</from>
 <date>Tue Oct 07 14:22:10 -0500 2008</date>
 <subject>Hola</subject>
 On your way home, please pick up the following from the store:
     * milk
     * eggs
     * bread
 Te quiero mucho.
</email>

You can find out more about ERB at the Ruby Standard Library Documentation.

How to compile and install Ruby on RHEL 4

This is one of those posts where I'm mostly writing this so I can remember how to do it in the future. But maybe it will help someone else, too. I'm not a Unix sysadmin, by any means, but here is what I did to get, compile, and install Ruby on a RHEL 4 system at work. I realized later that I could have found and installed an RPM, but after I started down this path, I wanted to finish it.

First of all, this may seem backwards, but I set these environment variables first. This is because I messed up the first time I tried to install Ruby, so these were left hanging around.

export RUBYLIB=/opt/ruby/lib:/usr/lib/site_ruby
export PATH=$PATH:/opt/ruby:/opt/ruby/bin

Next I downloaded the Ruby source code:

wget ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.6.tar.gz

I unarchived it:

tar -zxvf ruby-1.8.6.tar.gz

I made a symbolic link which will match the paths in the RUBYLIB and PATH environment variables I set up previously. Also, it's just a lot easier to type /opt/ruby instead of /opt/ruby-1.8.6-p287:

ln -s ruby-1.8.6-p287 ruby

Now, we build and install:

cd ruby
./configure
make
make install

I next installed RubyGems. First we download the archive and unpack:

wget http://rubyforge.org/frs/download.php/43985/rubygems-1.3.0.tgz
tar -zxvf rubygems-1.3.0.tgz

Then we run the setup:

cd rubygems-1.3.0
ruby setup.rb

And that is it!

Book review: Everyday Scripting with Ruby

Everyday Scripting with Ruby (for Teams, Testers, and You), by Brian Marick, is a good introduction to the Ruby language. Marick has a good conversational style that is not too cutesy like some other instructional texts I've read. The book is very textbook in that it develops everything step-by-step. It is geared toward the reader with very little knowledge of programming or scripting.

I also like how the scripts and projects that are developed are actually useful scripts to a software tester: an uninstaller checker, a script to gauge the amount of churn in a source control system, a website scraper script that extracts and data and reformats it to CSV, and a watchdog script. Interspersed between the chapters pertaining to the projects are chapters that go more in depth to the concepts that are presented in the project chapters. For example, in the discussion of CSV files, Hashes are introduced. In the following chapter, Marick shows all kinds of things that one can do with a Hash that was not covered.

Marick also introduces quite well the concept and practice of test driven development in his discuss of Test::Unit. He demonstrates the thought process of writing the basic script assuming Ruby provides everything, then writing tests for methods that need to be constructed, and then scripting the methods themselves.

I have finished reading the book but I am still working the exercises. The publisher's website has discussion and errata that have been very helpful.

This book is definitely a good way to get started in Ruby. I've definitely gotten interested in the language because of it, and I can't wait to start honing my skills more. 9 out 10.

My first Ruby script: finding duplicate photo files

I've started on the large task of organizing my digital photo files. My disorganization stems from a couple of things. First, when I began using a digital camera, I used iPhoto. I don't really use it now. But I've noticed that iPhoto makes some duplicate files. Second, I think I have duplicate files from backups and from migrating from old machines to new ones. Compounded with the fact that most of the files begin with DSCN or DCP, it is a mess. So, to help me get started, I thought I would write a script that would create a report of any duplicate files and their locations. Also, since I'm about one third of the way through the Everyday Scripting with Ruby by Brian Marick, I know just enough to be dangerous. After a couple of evenings of blundering my way through this, I finally have my first Ruby script.



To call this script, you can pass in a list of directories, or it will default to the current directory.

Learning Ruby

I just started learning the scripting language Ruby. I've started reading the book Mr. Neighborly's Humble Little Ruby Book by Jeremy McAnally. You can download the pdf version of the book at InfoQ.com to try before you buy.

As it turns out, I've already got Ruby installed on my iBook G4 since I'm running OS X 10.4. If you are on Windows or some other machine that does not have Ruby already installed, you can get it at http://www.ruby-lang.org, where they also have plenty of instructional information.

So far, I've only read the first chapter and have done the requisite "Hello, World!" program:

puts "Hello, World!"

Pretty simple, huh? Sure beats the pants off of Java for doing simple IO. As a software tester by profession, I think Ruby will be handy in my toolbag. And since it has gotten quite popular, I might as well try it as an alternative to Perl for my scripting and testing needs.