[Israel.pm] Testing problem

[Jason Elbaum]

This testing question raises a broader question: How to unit-test system 
scripts? Especially in Perl, which has so many commands which access the 
system, including: system(), open(), backtick, -X, <>, unlink, readdir, 
etc...

The options, as I see them:

1) Set up the system environment correctly before each test and run the 
code under test with direct system access

Advantage: Don't have to change the code under test.

Disadvantage: Test code can be complex to set up and slow to run - needs 
to set up and access (for example) filesystem, sockets, etc.


2) Mock all the system commands the code uses, with the code calling the 
mock functions

Advantage: Tests can run fast, since they don't need system access.

Disadvantage: Script code won't be idomatic Perl, since syntactic 
constructs like backtick and -X will be replaced with function calls.


3) Hook into the Perl core to replace the implementations of the system 
functions.

Advantage: Tests can run fast. Script code doesn't have to change.

Disadvantage: Implementing the Perl core hooks can be ugly.


Also, with solutions 2 and 3, it can be difficult to mock an entire 
system - it's a bit like implementing a mini operating system. You need 
to be able to create, write and read files and directories, execute 
commands - which need access to those files - etc.! In the general case, 
this might be nearly impossible.


Does anyone have experience with testing system scripts? Have you used 
any of these approaches, or another?


Regards,

Jason Elbaum