Test Portability

"Is your test portable?" - Generic Manager

"Huh?" - Todd

Test cases are dependent on interfaces.  A test case is a test of something and that something has a specification.  If the interfaces required by a test case exist across multiple testbeds, the test case will be portable.  A well documented test case is specific and includes the testbed specification.  A test case will be runnable when its testbed requirements are met.  If not, not.

If a testbed doesn't meet the required specifications for a test, why would we want to run it there?

If we want to test the bash 'pwd' command, our test cases should specify that the test must be run where the command exists, namely on the Unix/Linux command line (the dependency exists whether we understand it or not).  The 'pwd' command doesn't exist on the Windows command line.  What does it mean to make the value judgment that the test should be portable to Windows?

Like I said: "Huh?"

I could force our test cases to be portable by creating a virtual shell layer that runs on both Linux and Windows.  Yeah, I could do that.  It'd be fun.  The virtual shell could offer its own 'pwd' command.  It would just pass inputs and outputs to and from the Unix/Linux command line.  On Windows, it would simulate the Linux command's behavior.  The resulting test cases would be portable.  Silly, but portable.  Windows DOS still doesn't have a 'pwd' command!  So why are we testing it there?

Tests cannot be run where their testbed requirements aren't met.  Nor should they be.  Trying to force test cases to be portable is founded on a misguided value judgment.  The portability goal is driven by the laudable desire for efficient reuse, but is based on an imprecise understanding of test case dependencies.  There is a mapping between test cases and their required testbeds.  It's a better use of our time to improve our understanding of that mapping to assure we're running tests where they should be run than it is to try to contrive tests to run where they shouldn't.


Todd Shoenfelt
http://www.linkedin.com/in/toddshoenfelt