Continuous integration - CI tools

http://staf.sourceforge.net/ -- Salesforce.com uses this and this STAF is top choice in the article below

Functional Testing Tools, the Open-Source Wave
By Edward J. Correia

Last week, I wrote about “The Forrester Wave: Functional Testing Solutions, Q3, 2008,” a recent report that evaluated the six leading test-tool vendors offering manual and automated testing along with test management in a single solution. Qualifying vendors also earned at least US$10 million in annual revenues.

To contrast, this week I’ll present some open-source alternatives to those sometimes pricy products. To be fair, I should also mention that several companies, including AutomatedQA, MKS and Original Software—not to mention Microsoft—also offer solutions in this space.

There were a few ways I could have done this. I decided to visit the one Web site I knew of that contained the most information about the greatest number of open-source tools for software testers, and that's opensourcetesting.org. Using a few simple searches, I compiled lists of some of the more interesting tools for automated testing, manual testing and test management.

I found just one tool project—Salome-TMF—that offers all three capabilities sought in the Forrester report (more on Salome later). The remainder of today’s tools provide the capabilities in various combinations, but for all, the price is right.

Automated Testing
Topping the list was the Software Testing Automation Framework, also known as STAF. This multi-platform, multi-language function testing framework features reusable components such as process invocation, resource management and logging; an execution engine; and a GUI monitoring app. It’s built on Python and XML.

HtmlTestCase is a unit testing tool that consists of classes to simplify automation of HTML function testing. Intended for QA staff members that are slightly technical, tests can be specified using a high-level API.

If you’re versed in Perl and looking to automate IE app testing, you might try Win32::IEAutomation. This object-oriented tool, still in early development, treats all methods as user actions, and it is said to support the testing of complex apps, even those employing dynamic frames and pop-ups.

Automate the Windows GUI with pywinauto, a Python package that’s easy to use and requires no pre-recording or screen capture. It even comes with a screencast to show you how.

Systir stands for System Testing in Ruby. Not to be confused with a tester for Ruby code, this tool was written in Ruby for writing system-level tests in a domain language. If in the product description you have to explain what it’s not, I suggest a different name.

Faban is a project from Sun for developing and running benchmarks. It has two major components: the harness and the driver framework. The harness automates the running of server benchmarks. There’s also a Web interface to launch and queue runs, and functionality to view, compare and graph run outputs.

The Simple Web Automation Toolkit, or SWAT, allows QA engineers to automate Web application testing. IT includes a C# library that provides an interface for different Web browsers and components to integrate with FitNesse.

Manual Testing
Doit: Simple Web Application Testing is a scripting tool and language for testing Web applications that use forms. It can generate random or sequenced form fill-in information, report results (into a database, file or stdout), filter HTML results and compare results to previous results, without having to manually use a Web browser. It uses a console-based Web client tool (like Curl or Wget) to send and receive HTTP requests and responses respectively.

The Eclipse Test and Performance Tools Platform (TPTP) provides performance testing of HTTP applications, JUnit testing and manual testing. TPTP is a mature project with a thriving developer community. The project provides extensible framework and services on which developers can build test and performance tools to precisely suit their needs.

Test Management
If you're using Bugzilla, you might want to try Testopia, a plug-in for Mozilla's bug tracking system. Though it was originally designed for tracking test cases, it's generic enough to track any aspect of the development process. It also allows the integration of bug reporting with test-run results.

"Be-monster not thy feature, wer't my fitnesse." Shakespeare's King Lear was apparently the inspiration behind FitNesse, a wiki, Web server and testing framework that allows teams to collaborate on test specification and execution, and document creation.

Salome-TMF can create and execute manual or automatic tests, track results, manage requirements and defects, and produce HTML documentation. It works with Abbot, Beanshell and JUnit to define automatic tests, and with Bugzilla and Mantis to manage defects. Salome-TMF also is extensible.

Doubtless there were tools I missed simply because their descriptions or keyword tags lacked the terms I used. So if I missed your favorite, e-mail me. Maybe I’ll get another report out of it.


Share this link: http://www.sdtimes.com/link/32583



-----------------------
Continuous integration describes a set of software engineering practices that speed up the delivery of software by decreasing integration times. It emerged in the Extreme Programming (XP) community, and XP advocates Martin Fowler and Kent Beck first wrote about continuous integration near the turn of the millennium.

CI Feature Matrix:There are many Continuous Integration systems available. This page is an attempt to keep an unbiased comparison of as many as possible of them. The goals are:
* Make it easier to choose an appropriate CI tool for your project.
* "Healthy competition aid" for the people involved in the development of these various CI systems.

Which open source CI tool is best suited for your application's environment?
A comparative review of four open source continuous integration tools
- By John Ferguson Smart, JavaWorld.com, 11/01/06
* Cruise Control
* Continuum - The official Maven 2 CI tool
* Luntbuild
* Hudson
Automation for the people: Choosing a Continuous Integration server

Automated Smoke Tests With Selenium, Cargo, TestNG and Maven - See nice 8 step process
- The tools we use for this are:
1. Maven: The build tool we use
2. Selenium Remote Control: A package which lets one programatically control a browser, mimicking the actions of a real user
3. Cargo: A package which lets one programatically control a server
4. TestNG: A test framework
Taking Screenshots with Selenium - see user comments at the end of post



Practical Applications of Agile Web Testing Tools
Notes on Selenium testing


ReviewBoard: open source code review tool