Monday, October 26, 2009

Open Source Tracking Database Tools

No matter how good you think your code is there is always going to be a bug/defect of some kind. Defect numbers may range from 1 to thousands so it’s important to have a very good tracking tool. There are commercial tools out there but I figured I’d focus on the open source tools.

Below are some open source tracking tools that you can use for your projects to track bugs/defects, issues, enhancements and much more.

  • Abuky: Abuky stands for the Aoo BUg tracKing sYstem, while AOO stands for Art Of Open Source. Abuky is a system for tracking bugs and aiding the developer to fix them, written in Java with JSP as web interface. Requirement: Linux, Windows, Solaris

  • Anthill Bug Manager: A tool that aids code development by keeping track of bugs in a multi-project, multi-developer environment. It accomplishes this with a clean, simple, and fast interface that contains all the essential features
    but avoids the enormous complexity associated with most other projects of this type. Requirement: OS Independent

  • BTsys: A lightweight desktop bug tracking system, written using C# and ADO.NET for small developer teams. Easy to install and learn how to use. Requirement:All 32-bit MS Windows (95/98/NT/2000/XP)

  • Bug-A-Boo: A web based bug reporting system for Linux. It runs on any web server providing CGI
    functionality. Bug-A-Boo does not need any database server but brings along its own local tables it accesses with tdbengine.Requirement: Linux

  • BugBye: A web-based bugtracking system developed using ASP.NET technology and C# as scripting language. It offers all the features needed to manage improvement, bugs, and so on. It also provide statistics help-desk management, and further options which allow the user to simplify all the stage of project development and maintainance. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)

  • Bugfree: A lightweight and simple web-based bug tracking system available in Chinese and English. Requirement: Windows

  • Buglog: Allows you to easily record bugs into a database along with description and screenshots of the bugs (file upload feature). The BLT allows tracking of all bugs on a project-wise basis thus making it a bug logging and tracking solution. BuglogV2 incorporates a powerful search feature allowing developers to locate bugs effortlessly. Requirement: All 32-bit MS Windows(95/98/NT/2000/XP)

  • BugNet: A web based bug / issue tracking application programmed in C# and asp.net. The main goals are to keep the codebase simple, well documented, easy to deploy and scalable. Major features include: Generic DAL / XHTML and CSS layout. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)

  • BugRat: Java software that provides a sophisticated, flexible bug reporting and tracking system.

  • BUGS – The Bug Genie: A web-based bug tracking tool, designed with ease of use as it’s foremost goal, but also features a high level of flexibility for developers and administrators. Requirement: PHP4 (or later) and MySQL.

  • Bugs Online: Originally developed in 1997 to serve as the primary bug and issue tracking system to be utilized during a large development oriented project. The Bugs Online system is a very flexible and capable system for bug and issue tracking. Requirement: Windows NT 4.0 SP3+, MS IIS 3 w/ ASP

  • BugtrackWeb based bug tracking system written in Perl/DBI. Supports multiple users, projects, components, versions and email notification. Requirement: Linux, Solaris, Windows

  • BugTracker.NET: Web-based bug or issue tracker written using ASP.NET, C#, and Microsoft SQL Server/MSDE. It is in daily use by hundreds (maybe thousands) of development and support teams around the world. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)

  • Bugzilla: Features include: integrated, product-based granular security schema, inter-bug dependencies and dependency graphing, advanced reporting capabilities, a robust, stable RDBMS back-end, extensive configurability, a very well-understood and well-thought-out natural bug resolution protocol, email, XML, console, and HTTP APIs, available integration with automated software configuration management systems, including Perforce and CVS (through the Bugzilla email interface and checkin/checkout scripts), too many more features to list.

  • CodeTrack: Bug database with a friendly web front end aimed at medium and small development shops. Particularly suited for intranet and extranet environments, CodeTrack includes built-in strong authentication, and allows custom access control to individual projects. No database is required as bug data and developer notes are stored using simple XML text files. Requirement: Apache and PHP

  • Debian bug tracking software: The Debian bug tracking system is a set of scripts which maintain a database of problem reports. Requirement: Unix

  • DITrack (Distributed Issue Tracker): Implemented in Python and runs in UNIX (*BSD, Linux, MacOS X) and Windows environment.

  • Ditz: Open source distributed issue tracker designed to work with distributed version control systems written in Ruby

  • eTraxis: A web-based bug tracking system with unlimited number of custom workflow templates. Some of the main features are: fully custom templates, advanced filters, LDAP support, email notifications, subscriptions, reminders, flexible permissions management, graphical project metrics, etc. Requirement: OS independent

  • Eventum: A user-friendly and flexible issue tracking system that can be used by a support department to track incoming technical support requests, or by a software development team to quickly organize tasks and bugs. Requirement: PHP, MySQL

  • Flyspray: Originally developed for the Psi project, Flyspray is an easy to use bug tracking system for those who do not require all the complexities of something like Bugzilla. It supports multiple users, file attachments, and Jabber notifications. Requirement: Web Environment

  • FOSSology: Started out as an internal development effort at Hewlett Packard Company (HP).

  • Gjallar: Highly customizable issue tracker with advanced email integration and ability to work offline and synchronize with server. Requirement: Windows

  • GNATS: A portable incident/bug report/help request-tracking system which runs on UNIX-like
    operating systems. It easily handles thousands of problem reports, has been in wide use since the early 90s, and can do most of its operations over e-mail. Several front end interfaces exist, including command line, emacs, and Tcl/Tk interfaces. There are also a number of Web (CGI) interfaces written in scripting languages like Perl and Python. Requirement: OS Independent

  • Helis: Iincludes the main features of most bug tracking systems. Requirement:Linux web server (php 4/mysql + cgi)

  • Issue Tracker: by ASP.NET. Issues can be tracked, related issues can be displayed & more. Issue Tracker supports MS Access & MSSQL for storing the data.

  • Issue Tracker Product: A straight forward and user friendly web application built on top of the Zope application server. Requirement: OS Independent, Zope

  • itracker: An issue tracking system designed to support projects with independent user bases. Features include multiple versions and components, detailed issue histories and email notifications. Soon based on Java Spring 2.0 and Hibernate. Requirement: OS Independent

  • JitterBug: A web based bug tracking system. JitterBug operates by receiving bug reports via email or a web form. Authenticated users can then reply to the message, move it between different categories or add notes to it. In some ways JitterBug is like a communal web based email system. This web page is itself a JitterBug page.

  • jTrac: A generic issue-tracking web-application that can be easily customized by adding custom fields and drop-downs. Features include customizable workflow, field level permissions, e-mail integration, file attachments and a detailed history view. Requirement: OS Independent

  • Kwok Information Server: IT web application, providing a centralized application for managing IT assets, software licenses, service contracts, issues, and contacts. Additional modules include portal, RSS, and blogs.

  • Lighthouse Pro: ColdFusion bug tracking application. It lets you easily track bugs and issues for a project. With Lighthouse Pro, you can monitor the complete life cycle of an issue, from creation to QA to closure.

  • Mantis: A php/MySQL/web based bugtracking system. Requirement: Windows, MacOS, OS/2, and a variety of Unix operating systems. Any web browser should be able to function as a clientWindows, MacOS, OS/2

  • Ohloh: Is more than just a tracking database

  • oops-easytrack: A Bug Tracking System, implemented as a LAN-based and a web-based version. Bugs are created, modified, logged or searched. Users, projects,components and releases can be administered. Reports of 4 types may be generated. Requirement: OS Independent

  • Open Track: A problem tracking(PR) system that is table driven and easily
    configurable/customizable for a variety of PR applications. Project defect tracking, help desk tracking, and requirements gathering can be easily handled by OpenTrack.

  • PEST: A bug tracking system written especially for a web enviroment. It supports good testing and bug tracking processes, as well as notification.

  • phpBugTracker: Provides a codebase that is independent of the database and presentation layers. Requirement: webserver with PHP 4.1.0+

  • Project Dune: Mostly a web-based issue tracker with integrated modules: Cocomo II estimates, Scrum tasks, timesheet, simple customer management and a browser-based document writer. Requirement: OS Independent

  • Project Open: Web-based ERP/Project Management software for organizations with 2-2000 users

  • RadTracker: A MYSQL-PHP web based issue tracking system built for healthcare but generic enough for most needs. Designed to ease the support of multiple idiosyncratic information systems needing their own knowledge base and support mechanism.

  • Redmine: Flexible project management web application. Written using Ruby on Rails framework, it is cross-platform and cross-database. Requirement: Ruby and Ruby on Rails must be installed.

  • Request Tracker: RT is an industrial-grade ticketing system. It lets a
    group of people intelligently and efficiently manage requests submitted by a community of users. RT is used by systems administrators, customer support staffs, NOCs, developers and even marketing departments at over a thousand sites around the world. Requirement: Written in object-oriented Perl, RT
    is a high-level, portable, platform independent system

  • Roundup Issue Tracker: Roundup is a simple-to-use and -install issue-tracking system with command-line, web and e-mail interfaces. It is based on the winning design from Ka-Ping Yee in the Software Carpentry “Track” design competition.

  • Scarab: The goal of the Scarab project is to build an Issue / Defect tracking system that has the following features: A full feature set similar to those found in other Issue / Defect tracking systems: data entry, queries, reports, notifications to interested parties, collaborative accumulation of comments, dependency tracking In addition to the standard features, Scarab has fully customizable and unlimited numbers of Modules (your various projects), Issue types (Defect, Enhancement, etc), Attributes (Operating System, Status, Priority, etc), Attribute options (P1, P2, P3) which can all be defined on a per Module basis so that each of your modules is configured for your specific tracking requirements. Built using Java Servlet technology for speed, scalability, maintainability, and ease of installation. Import/Export ability via XML allowing for easy migration from other systems (like Bugzilla). Modular code design that allows manageable modifications of existing and new features over time. Fully customizable through a set of administrative pages. Easily modified UI look and feel. Can be integrated into larger systems by re-implementing key interfaces.

  • Stabilizer: The Stabilizer bugtracking system aims to quickly stabilize buggy GUI applications so that people can get real work done with them. Users collaboratively and quickly stabilize a buggy GUI application simply by using the application normally and reporting any bugs that they encounter. As soon as a few people report the same bug, warnings will be issued to all users whenever they are about to trigger that bug and they will be given the opportunity to abort the input event — thus avoiding the bug altogether and keeping the application stable. Requirement: All POSIX (Linux/BSD/UNIX-like OSes), Linux

  • The Bug Genie: Built with PHP, has multilanguage support & provides an easy to use interface.

  • Trac: An enhanced wiki and issue tracking system for software development projects. Trac allows wiki markup in issue descriptions and commit messages, creating links and seamless references between bugs, tasks, changesets, files and wiki pages. A timeline shows all project events in order, making getting an overview of the project and tracking progress very easy. Requirement:Python, CGI-capable web server

  • Track+: A web based issue tracking and project management application targeted at medium to large commercial development projects. Track+ can be used instantly, yet it is flexible and configurable. Track+ is easy to use, provides user centric as well as project centric views, has configurable workflows, supports e-mail submission of issues, and has a comprehensive time and cost tracking module. Track+ is free to academic institutions, open source projects, and charities, and generally free to anybody up to 10 active users in the database. Track+ comes with the complete source code. Requirement: OS Independent (Written in an interpreted language)

  • TrackIt: A Web-based project tracking tool that incorporates defect tracking functionality. It is designed from the ground up to provide maximum flexibility, customization, and most importantly, usefulness to the developer. It has built-in support for various Extreme Programming constructs, as well as full CVS and Subversion integration. It also supports simple listings via HQL and advanced reporting via SQL. Requirement: JRE 1.5

  • WebIssues is an open source, multi-platform system for issue tracking and team collaboration. The server can be installed on any host with PHP and MySQL, PostgreSQL, Firebird or SQL Server. The client is a native desktop application for both Windows and Linux.

  • WREQ: Designed to be a distributed request/problem tracking system with builtin knowledge database to help systems personnel to stay on top of requests and to prompt knowledge sharing among all local support groups. Requirement: To use wreq, first you must have perl version 5 with GDBM support installed on your web server.

  • zenTrack: Highly configurable bug tracking, project management, and help desk solution. Project focus is on configurability, usability, and clean code. Requirement: OS Independent

Please comment below on your thoughts about any of the tools listed above and/or if you know of any other tools that should be added.


Sunday, October 25, 2009

Open Source Free Functional Testing Tools

It is very important to make sure that your application functions as expected. There may be times that you add one little piece of code and all of a sudden other parts of the application no longer works. You may not have time/capacity to manually go back and regression test all the pieces of your application to make sure they are up to par. Companies use a variety of different testing tools for regression testing. There are lots of tools out there from very expensive to open source.

Below is a list of some open source/free tools that may come in handy.

Please keep in mind that every application is different so the tool you pick from one application may not be the same tool that you pick for another. My advice would be to pick a tool that can meet most of your current and near future needs.

  • Abbot Java GUI Test Framework: The Abbot framework provides automated event generation and validation of Java GUI components, improving upon the very rudimentary functions provided by the java.awt.Robot class (A Better ‘Bot). The framework may be invoked directly from Java code or accessed without programming through the use of scripts. It is suitable for use both by developers for unit tests and QA for functional testing.

  • actiWate: Java-based Web application testing environment from Actimind Inc. Advanced framework for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit etc. but with extended API), and Test Writing Assistant – Web browser plug-in module to assist the test writing process. Freeware.

  • Anteater: A testing framework designed around Ant, from the Apache Jakarta Project. It provides an easy way to write tests for checking the functionality of a Web application or of an XML Web service.

  • Apodora: A framework/tool for automating functional testing of web applications. It provides the user with programmatic control of the web browser allowing them to interact directly with the browser’s user interface. It uses a database backend in order to remember how to find your html elements. This also makes your scripts easier to maintain. Requirement: Windows, IE6

  • Arbiter: Document based acceptance tester. Similar to FIT in goal. Manages requirements documents in Word or RTF format that are created jointly by customer and developer. Requirements are parsed to extract a glossary and test suite.

  • Autonet:Autonet is a GUI network
    test platform, internally it’s based on CLI to communicate with devices. It can help you to arrange test cases, setup commands to devices,run commands to check results and record test results. Requirement: windows, linux and any other platform which support tcl

  • AutoTestFlash: Allows the recording and playback of tests written in Flash and Flex. The tool website provides a live sample. Requirement: Windows / Flash

  • Avignon: An acceptance test system that allows you to write executable tests in a language that you define. It uses XML to define the syntax of the language but, if you choose to extend the language, leaves the semantics of the tests up to you. Avignon includes modules for testing HTML applications (through either IE or FireFox), Swing and .NET WinForm applications. Requirement: Java (MS Windows only for .NET testing)

  • Blerby Test Runner: Ajax test runner for php. Currently supports simpletest and phpunit 3.x. Allows developers to refactor code while being able to receive instant feedback on their changes. Tracks test dependencies and automatically re-runs appropriate tests upon source changes. Requirement: Windows, *nix, apache

  • Canoo WebTest: Used for functional testing of web pages, WebTest is an open source testing framework built on top of HttpUnit. It allows tests to be defined in XML as Ant targets. Requirement: JDK 1.2 and ANT v1.3

  • Celerity: a JRuby wrapper around HtmlUnit – a headless Java browser with JavaScript support. It provides a simple API for programmatic navigation through web applications. Celerity aims at being API compatible with Watir.

  • Concordion: Framework for Java that lets you turn a plain English description of a requirement into an automated test. Concordion specifications are active. Behind the scenes, they are linked to the system under test and therefore do not go out-of-date. If a change is made to the system’s behaviour then the tests associated with the relevant specification will fail and let you know.Requirement: Java 1.5 or above

  • Crosscheck: Framework for verifying your in-browser javascript. It helps you ensure that your code will run in many different browsers such as Internet Explorer and Firefox, but without needing installations of those browsers. The only thing you need is a Java Virtual Machine.Requirement: Java Virtual Machine

  • csvdiff: A Perl script for comparing two files of comma-separated values with each other. In contrast to standard diff, it will show the number of the record where the difference occurs, the column number, and (if provided) the fieldname which is different. The separator can be set to any value you want, not just a comma. It also provides support for multiple column keys, the ability to ignore case and trim leading/tailing spaces, and the ability to ignore selected columns such as timestamps. Requirement: Everywhere you can install perl

  • CubicTest: A graphical Eclipse plug-in for writing Selenium and Watir tests. It makes web tests faster and easier to write, and provides abstractions to make tests more robust and reusable. CubicTest’s test editor is centered around pages/states and transitions between these pages/states. The model is intuitive for both Ajax and traditional web applications and supports most user interaction types. CubicTest features an innovative test recorder and test runner based on Selenium RC which are fully integrated with the graphical test editor. Tests can also run standalone from Maven 2. Requirement: Eclipse plug-in

  • DBFeeder: With DBFeeder you can automatically generate testdata for Oracle Databases which fits primary and foreign keys of tables. A file-based configuration system allows in-depth customization of the generated data.

  • DbFit: Extension to FIT/FitNesse for test-driven database development. Enables developers to manipulate database objects in a relational/ tabular form, making database testing and management much easier then with xUnit-style tools. Requirement: Java/.NET

  • DejaGnu: Framework for testing applications such as Tcl, C, C++, Java and network applications and cross testing of embedded systems. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. Requirement: MacOS, Windows, POSIX

  • Dogtail: A GUI test tool and automation framework written in Python. It uses accessibility technologies to communicate with desktop applications. Dogtail scripts are written in Python and executed like any other Python program. Requirement: Python 2.3 or higher

  • Doit: Simple Web Application Testing: Scripting tool and language for testing web applications that use forms. Doit can generate random or sequenced form fill-in information, report results (into a database, file, or
    stdout), filter HTML results, and compare results to previous results, without having to manually use a web browser. It uses a console-based web client tool (like Curl or Wget) to send and receive HTTP requests and responses respectively. Requirement: You must have Perl 5 or greater and the appropriate Perl modules (detailed in Doit manual) installed on your system before you can use SPL.

  • Eclipse TPTP: The Eclipse test and performance tools platform (TPTP) provides support for three types of testing: Performance testing of HTTP applications, JUnit testing and manual testing. Although each of these areas of testing has its own unique set of tasks and concepts, two sets of topics are common to all three types: creation and use of datapools, and creation of test deployments. Requirement: Eclipse

  • EMOS Framework: A simple yet powerful environment for development of automated WinRunner? tests. Like most frameworks of this sort EMOS Framework separates test data from the test code in order to simplify and speed up test development, increase robustness of the produced solution, and empower non-programmers towards test automation. It is almost completely written in WinRunner’s own scripting language, TSL.Requirement: Mercury WinRunner, All 32-bit MS Windows (95/98/NT/2000/XP)

  • Enterprise Web Test: Allows Java programmers to write re-usable tests for web applications that, unlike HttpUnit, “drive” the actual web browser on the actual platform they intend to support. Tests can be leveraged for functional, stress, reliability. Requirement: Microsoft, OS Independent, Linux

  • Expect: A Unix automation and testing tool, for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, ssh, tip, etc. And by adding Tk, you can also wrap interactive applications in X11 GUIs. Requirement: Windows / UNIX

  • Frankenstein: A Functional Testing tool for Swing applications. Frankenstein’s focus is on readable, simple, fast functional tests that can be shared and run by everyone on a team. Apart from automating your functional tests, you could also use Frankenstein for recording bugs so that they may be easily reproduced later. Requirement: JDK 1.4+

  • FireWatir: Has a similar API to Watir, though accesses the DOM by invoking JavaScript by using the JSSh XPI to telnet into the browser. While Watir works with MSIE, FireWatir is compatible with Firefox 1.5 and above. FireWatir allows Watir scripts written for IE to work with Firefox as well, usually requiring either no change or very small changes to existing scripts. It is planned for FireWatir and Watir to be merged. The wiki includes info on compatibility issues between Watir and Firewatir.

  • Funkload: Web functional testing and load testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF. Functional tests are pure Python scripts using the pyUnit framework.

  • FWPTT: is a web application tester program for load testing web applications which can record normal and Ajax requests

  • GITAK: TIBCO General Interface Test Automation Kit (GITAK) is a test-automation tool for Ajax applications. GITAK extends the Selenium Core test tool for Web applications. It lets developers create automated test cases and run scenarios to validate that an application is performing properly. Once a library of test cases has been built, the Ajax applications and changes to them can be tested with the push of a button. Requirement: IE 6, IE7, Firefox 2, Firefox 1.5.x

  • GNU/Linux Desktop Testing Project: Aimed at producing high quality test automation framework and cutting-edge tools that can be used to test GNU/Linux Desktop and improve it. It uses the “Accessibility” libraries to poke through the application’s user interface. The framework has tools to generate “AppMap” by reading through the user interface components of an application. The framework also has tools to record test-cases based on user-selection on the application. GNU/LDTP can test any GNOME application which are accessibility enabled, Mozilla, OpenOffice.org, any Java application (should have a UI based on swing) and KDE 4.0 applications based on QT 4.0 (based on the press releases by KDE). Requirement: Linux

  • Harness: An open source Java API for creating Java test software

  • HtmlUnit: Java unit testing framework for testing web based applications. (Similar in concept to httpunit but is very different in implementation) HtmlUnit models the returned document so that you can deal with pages, forms and tables.

  • httest: Scriptable HTTP Test Tool for testing and benchmarking web application and HTTP server development. Can act as client (requesting) and server (back-end for reverse proxys). Pattern matching answers (both server(s)and client(s)) to test validity. Has a very simple but powerful syntax. Can execute and stream shell commands into the HTTP stream and vice versa. Requirement: linux, solaris

  • HTTPUnit: Java API for testing web sites without a browser.

  • IdMUnit: Leading xUnit automated testing framework for Identity Management that simplifies and accelerates the functional testing of the solution. Test cases are defined and implemented in spreadsheet format. This product plugs into Eclipse. Requirement: Cross-platform

  • IeUnit: A simple framework to test logical behaviors of web pages, released under IBM’s Common Public License. It helps users to create, organize and execute functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript for the Windows XP platform with Internet Explorer.

  • iMacros for Firefox: Free Firefox add-on to record and automate web interactions. Can use variables inside the macros, and import data from CSV files. Includes user agent switcher, PDF download and Flash, ad and image blocking functions. The recorded macros can be combined and controlled with Javascript, so complex tasks can be scripted. The EXTRACT command enables reading of data from a website and exporting it to CSV files. Full Unicode support and works with all languages including multi-byte languages such as Chinese. STOPWATCH command enables capturing of web page response times

  • Imprimatur: A web application functional testing tool. The tests are described in a simple XML file. Imprimatur handles HTTP methods, authentication and file uploads. The responses can be validated using regular expressions. Requirement: Java

  • ItIN – Infopath testing in .Net: A framework for the testing of InfoPath forms. It is derived from the WatiN testing framework which is used for testing web applications. ItiN needs Visual Studio 2005 and InfoPath 2003 with the .Net programability support installed to work. You may have some trouble with the references, but it should be OK. Requirement: Windows

  • ITP: Lightweight, yet powerful web application test harness. Test scripts written in XML. No programming required and no changes required to your web application. Supports sessions/cookies, POST form data. Command line based for integration into other tools. Also useful for regression and smoke testing.

  • ItsNat, Natural AJAX: A Java AJAX web framework with functional web test built-in. Simulates a Universal Java W3C Browser in the server, the client DOM tree is basically a clone of the server and is updated automatically when the server changes usually as the response of an AJAX event. The server can fire W3C DOM events and send them to the browser simulating user actions. These are received again by the server as in a normal AJAX app. As the test code is in the server too, can check the expected GUI changes (checking the server DOM tree) or the expected business behavior (added/removed/updated data). Requirement: Any supported platform by Java VM 1.4 or upper

  • ivalidator: Regression testing framework written in java but by no means restricted to java testing. Test suites are declared in XML. Especially designed for complex testing scenarios and integration testing. Requirement: JDK 1.3

  • Jacobie: A Java API for use with Internet Explorer. Based on the JACOB project (Java to Com Bridge) and the IE COM Object, it directly controls IE from java. This API can be used as a true end-user web browser test with IE and not a Http-Based test such as HttpUnit. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)

  • Jameleon: A plug-in driven automated testing tool that separates applications into features and allows those features to be tied together independently, creating test cases. Test cases can be data-driven and executed against different environments and test case docs are generated from the test cases. The goal is to create an automated testing tool that can be used for the enterprise. A UI that ties a series of features to a test case, generating both the test script and the test case documentation is in the works. Requirement: OS Independent, JDK 1.4 or higher

  • jDiffChaser: A GUI comparison tool that automates diffs detection between versions. You can record and play scenarios on two different releases of the same Swing application (in sequential or parallel mode); jDiffChaser compares both screens, shows you the differences and list them in a report with images highlighting the diffs. Requirement: Linux, OS X, WinXP

  • Jemmy: A tool allowing you to create automated tests for Java GUI applications. Tests are written on Java, using Jemmy as a regular Java library. No recording (yet), no GUI, no XML, no bells and whistles – all the work being done in Jemmy is dedicated to test stability, so it could be used for big, complicated and “dynamic” Java GUI applications. Requirement: Java 1.2 or above

  • JFunc: JUnit Functional Testing Extension: An extension to the JUnit testing framework to make it easier for use with functional tests. Functional testing (also called integration testing) significantly differs from unit testing in a number of respects. Part of this project is dedicated towards putting together code to address these differences; the other part of this project is putting together methodologies for functional testing. Requirement: JUnit

  • JSystem: An open source framework for writing and running automated system testing. JSystem includes: 1. Services Java API – exposes JSystem services 2. JSystem Drivers- Java modules used to interfaces with the system under test. 3. JRunner – GUI application interface used for creating and running tests scenarios. 4. JSystem Agent – Execution engine used to run scenarios on a distributed setup. 5. JSystem Eclipse plug-in – accelerates the development environment setup and enforces JSystem conventions. JSystem is based on JUnit (tests and steps) and Ant (execution engine).

  • jWebUnit: A Java framework that facilitates creation of acceptance tests for web applications. jWebUnit provides a high-level API for navigating a web application combined with a set of assertions to verify the application’s correctness. This includes navigation via links, form entry and submission, validation of table contents, and other typical business web application features. This code utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit. Requirement: OS Independent

  • Latka: A functional (end-to-end) testing tool. It is implemented in Java, and uses an XML syntax to define a series of HTTP (or HTTPS) requests and a set of validations used to verify that the request was processed correctly. Requirement: JDK 1.3 or better.

  • Linux Test Project: A collection of tools for testing the Linux kernel and related features. Our goal is to improve the Linux kernel by bringing test automation to the kernel testing effort. Requirement: Linux

  • LogiTest: The core application in the LogiTest suite. The LogiTest application provides a simple graphical user interface for creating and playing back tests for testing Internet-based applications. Requirement:JDK 1.2 or higher

  • LReport: Command line tools for comparing csv files and databases (on the level of particular selects). The tools also support test documentation by nice formatting of selects’ results. Requirement: Tested on Win32 but should work on other platforms

  • Mactor: An extensible tool for system integration testing. It can facilitate tests of any XML-based integration regardless of the type of message transfer protocol used (HTTP, SOAP, file-system and IBM MQ series are currently supplied with the tool)

  • Marathon: A general purpose tool for both running and authoring acceptance tests geared at the Java Platform Version 1.3 or later. Included with marathon is a rich suite of components to help you interact with your application at the User Interface Level (GUI). To aid with the regression testing of existing applications, Marathon comes bundled with a recorder to capture events as you use and interact with your application. These events are then converted into a valid Marathon test which can subsequently be played back. Requirement: Java 1.3 or later

  • MaxQ: A free web functional testing tool. It includes an HTTP proxy that records your test script, and a command line utility that can be used to playback tests. The paradigm of MaxQ is similar to commercial web testing tools like Astra QuickTest or Empirix e-Test. These products are quite expensive. MaxQ hopes to provide the essential features: HTTP test recording, scripting, and playback without the huge cost.Requirement: Java 1.2 or later

  • Mechanize: Ruby library for automating interaction with websites; automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Also keeps track of the sites visited. NOTE: does not handle javascript.

  • Mockito: Java mocking is dominated by expect-run-verify libraries like EasyMock or jMock. Mockito offers simpler and more intuitive approach: you ask questions about interactions after execution. Using mockito, you can verify what you want. Using expect-run-verify libraries you are often forced to look after irrelevant interactions. Mockito has very slim API, almost no time is needed to start mocking. There is only one kind of mock, there is only one way of creating mocks. Just remember that stubbing goes before execution, verifications of interactions go afterwards. Requirement: Java

  • MozUnit: Develop test-first style or just test against regressions: MozUnit provides framework, test runner, source browser, and API hooks for personalized reports. MozUnit is part of MozLab, a suite of tools and libraries for developers of AJAX and Mozilla applications, packaged as a Firefox extension.Requirement: Firefox

  • OLVER – Open Linux VERification: A test suite for automated conformance and functional testing of various Linux distributions against LSB standard requirements on base system interfaces behavior. The tests are being developed at the Linux Verification Center of Russia. Requirement:Linux

  • org.tigris.mbt: An implementation of Model-based testing built in Java. It allows you to generate test sequences from a finite-state machine (graph). The test sequences can be created statically, or run dynamically.Requirement: Any platform that runs Java 1.4.2

  • Ottomate: Suite of six Mac OS X Automator Actions that contains everything needed to graphically configure automated, repeatable user-acceptance tests for web-based applications. Requirement: Safari

  • PAMIE: ‘Python Automated Module For Internet Explorer’ Allows control of an instance of MSIE and access to it’s methods though OLE automation . Utilizes Collections, Methods, Events and Properties exposed by the DHTML Object Mode Requirement:Windows NT/2000

  • Pounder: A utility for testing Java GUIs. It allows developers to dynamically load components, record scripts, and then use those scripts in JUnit. It supports custom components, drag and drop, and the examination of test runs in source. This project is no longer being actively developed. For similar tools under active development, the Pounder team recommend considering Abbot, Marathon, jfcunit and others. Requirement: OS Independent

  • PureTest: From Minq Software AB, includes an HTTP Recorder and Web Crawler. Create scenarios using the point and click interface. Includes a scenario debugger including single step, break points and response introspection. Supports HTTPS/SSL, dynamic Web applications, data driven scenarios, and parsing of response codes or parsing page content for expected or unexpected strings. Includes a Task API for building custom test tasks. The Web Crawler is useful for verifying consistency of a static web structure, reporting various metrics, broken links and the structure of the crawled web. Multi-platform – written in Java.

  • pywinauto: A python package that allows you to automate the windows GUI. Very easy to get started, and quite powerful. Requirement:Windows 2000, XP, +

  • QAT (Quality Assurance Tests): Developed to ease the issues encountered by having to perform Quality Assurance tests across a variety of hardware and software combinations. The QAT tool can be divided into two main sections, the Agent, responsible for actually running each test or group of tests, and the Harness, which is responsible for test selection, management, result and agent co-ordination. Requirement:Java 2

  • QMTest: CodeSourcery’s QMTest provides a cost-effective general purpose testing solution that allows an organization to implement a robust, easy-to-use testing program tailored to its needs. QMTest’s extensible architecture allows it to handle a wide range of application domains: everything from compilers to graphical user interfaces to web-based applications. Requirement: QMTest works with most varieties of UNIX, including GNU/Linux, and with Microsoft Windows.

  • Rasta: A keyword-driven test framework using spreadsheets to drive testing. It’s loosely based on FIT, where data tables define parameters and expected results. The spreadsheet can then be parsed using your test fixtures.Requirement:Windows, Ruby

  • Robot Framework: Robot Framework is a Python-based keyword-driven test automation framework for acceptance level testing and acceptance test-driven development (ATDD). It has an easy-to-use tabular syntax for creating test cases and its testing capabilities can be extended by test libraries implemented either with Python or Java. Users can also create new keywords from existing ones using the same simple syntax that is used for creating test cases.

  • safariwatir: The original Watir (Web Application Testing in Ruby) project supports only IE on Windows. This project aims at adding Watir support for Safari on the Mac. Requirement: OS X running Safari

  • Sahi: An automation and testing tool for web applications, with the facility to record and playback scripts. Developed in Java and JavaScript, it uses simple JavaScript to execute events on the browser. Features include in-browser controls, text based scripts, Ant support for playback of suites of tests, and multi-threaded playback. It supports HTTP and HTTPS. Sahi runs as a proxy server and the browser needs to use the Sahi server as its proxy. Sahi then injects JavaScript so that it can access elements in the webpage. This makes the tool independant of the website/ web application. Requirement:Needs Java 1.4+

  • Samie: S.A.M. for I.E. is a Perl module (SAM.pm) that allows a user to run automated tests for their browser applications. Requirement: Windows NT/2000

  • Scalable Test Platform: STP is a system for automating the QA testing process for the Linux Kernel, as well as automating benchmarking and regression testing on diverse hardware systems. Requirement: Linux

  • Siege: http regression testing and benchmarking utility. It was designed to let web developers measure the performance of their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It allows the user hit a web server with a configurable number of concurrent simulated users. Those users place the webserver “under siege.”

  • Selenium: Testing tool for browser-based testing of web applications. It can be used both for functional, compatability (it has extensive cross-browser support) and regression testing Requirement: Windows, Linux or Mac

  • Selenium Grid: An open source web functional testing tool that can transparently distribute your tests on multiple machines to enable running tests in parallel, cutting down the time required for running in-browser test suites. This enables speed-up of in-browser web testing. Selenium tests interact with a ‘Selenium Hub’ instead of Selenium Remote Control. The Hub allocates Selenium Remote Controls to each test. The Hub is also in charge of routing the Selenium requests from the tests to the appropriate Remote Control as well as keeping track of testing sessions. Requires Java 5+ JDK, Ant 1.7.x

  • SharpRobo: A Functional Testing and Recording tool for WinForm applications written in C#. It supports all the standard WinForm controls. SharpRobo records the tests in FIT format which can be played back using Fit (File or Directory Runner). Requirement:Windows NT/2000/XP

  • SimpleTest: Unit testing framework which aims to be a complete PHP developer test solution. Includes all of the typical functions that would be expected from JUnit and the PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This includes web page navigation, cookie testing and form submission.

  • soapui: A java-swing based desktop application for inspecting, invoking and functional testing of webservices over HTTP. It is mainly aimed at developers/testers providing and/or consuming webservices (java, .net, etc). Functional testing can be done interactively in soapui or within a CI-process using the soapui maven plugin.Requirement: Java 1.5

  • Software Automation Framework Support (SAFS): Provides for the implementation of compatible keyword-driven test automation frameworks. Currently, developing independent, multi-platform, Java-based Driver. Will be followed by independent, multi-platform Engines. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)

  • Software Testing Automation Framework (STAF): An open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring). STAF removes the tedium of building an automation infrastructure, thus enabling you to focus on building your automation solution. STAX is an execution engine which can help you thoroughly automate the distribution, execution, and results analysis of your testcases. STAX builds on top of three existing technologies, STAF, XML, and Python, to place great automation power in the hands of testers. STAX also provides a powerful GUI monitoring application which allows you to interact with and monitor the progress of your jobs. Requirement:Windows, Linux, Solaris, AS/400, AIX, HP-UX, Irix

  • Solex: This project is a set of Eclipse plugins providing non regression and stress tests of Web application servers. Test scripts are recorded from internet browser thanks to a built in web proxy. Requirement: Eclipse 2.1 or above

  • SWAT (Simple Web Automation Toolkit): A library written in C# designed to provide an interface to interact with several different web browsers. SWAT also includes components to integrate with Fitnesse allowing Q/A engineers to automate web application testing. Requirement: Windows (IE and FireFox)

  • SWTBot: A functional testing tool for SWT and Eclipse applications. The focus of SWTBot is to provide a simple, readable and fast way to write tests. The API is simple which means that everyone on a team can use SWTBot to write functional tests. It is also very flexible when it comes to extensibility. Requirement: SWT/Eclipse

  • Systin: Systin stands for System Testing in .Net and allows you to write system-level tests in a “domain language”. This is a port of the popular Systir program. Systin will allow for an abstraction of Test Case specification and Test Case automation execution. Requirement: .Net Windows

  • tclwebtest: A tool for writing automated tests on web applications in tcl. It implements some basic html parsing functionality to provide comfortable commands for operations on the html elements (most importantly forms) of the result pages.

  • TestGen4Web: A capture-replay tool which can record user actions on Firefox, saving the recording to an XML file, and replaying the saved recording. The output of the recorder can also be translated into automatic testing scripts such as httpunit, selenium, simple-test, etc. Requirement: Firefox 1.5 alpha1 +

  • TextTest: An application-independent tool for text-based functional testing. This means running a batch-mode binary in lots of different ways, and using the text output produced as a means of controlling the behaviour of that application.Requirement: Most UNIX flavours + Windows XP (not Windows 9x)

  • Tomato: (the Automation Tool Abstraction Project) An abstraction layer for automation engines. Its design allows automation scripts or tests to be written in one language, against one library, and remain portable across different architectures, OS platforms, and even widely different automation engines (e.g. HP Mercury Interactive WinRunner or the Linux Desktop Test Project). Requirement: Windows/Linux

  • Toster – The Object-oriented Sofware Testing Environment: A system for sharing a set of tools that allow you to implement methods for object-oriented testing. Any method based on UML diagrams and on the software source code can easily be implemented as a TOSTER module. The environment itself makes a number of mechanisms available, such as information transfer from UML diagrams, mapping this information to source code, introducing modifications to the source code, launching the tested application, or presenting the results.

  • Watij: (pronounced wattage) stands for Web Application Testing in Java. Based on the simplicity of Watir and enhanced by the power of Java, Watij automates funtional testing of web applications through the real browser. There is a Google group at http://groups.google.com/group/watij Requirement:Windows

  • WatiN: WatiN stands for Web Application Testing in dotNet. Inspired by Watir, WatiN enables web application testing, through Internet Explorer on a Windows platform, expressed in any .Net language.Requirement: Windows

  • Watir: Watir (Web Application Testing in Ruby) is a functional testing tool for web applications. It supports tests executed at the web browser layer by driving a web browser and interacting with objects on a web page. It uses the Ruby scripting language. Requirement: Windows (currently only supports Internet Explorer)

  • WebCorder: Free GUI web testing tool from Crimson Solutions, developed in VB. Designed for end users who are doing web based software testing, as a simple tool to record test scenarios, and play them back and generate log files. The user may also check for text or images on the screen or save screenshots.

  • Web Form Flooder: A Java console utility that will analyze a Web page, complete any forms present on the page with reasonable data, and submit the data. The utility will also crawl links within the site in order to identify and flood additional forms that may be present.

  • WebDriver: A developer focused tool for automated testing of webapps: WebDriver has a simple API designed to be easy to work with and can drive both real browsers, for testing javascript heavy applications, and a pure “in memory” solution for faster testing of simpler applications. Requirement: Any java-compatible platform

  • WebInject: A free tool for automated testing of web applications and services. It can be used to test any individual system component with an HTTP interface, and as a test harness to create a suite of automated functional and regression tests. Requirement: Windows, OS Independent, Linux

  • Webrat: Ruby-based utility to enable quick development of web app acceptance tests. Open source by Bryan Helmkamp. Leverages the DOM to run tests similarly to in-browser test tools like Watir or Selenium without the associated performance hit and browser dependency. Best for web apps that do NOT utilize Javascript; apps using Javascript in-browser tools may be more appropriate.

  • WebTst: AWeb development test infrastructure. It aims to simplify testing by implementing a capture engine: a Web proxy which records a tester’s actions using a real browser, and then replays them during testing. It comes with support for digital certificates, and a number of simple tests, such as cookie setting, pattern matching, response status, and many others. It features an extensible plug-in system. Requirement: POSIX, Linux

  • WET: An opensource web automation testing tool which uses Watir as the library to drive web pages. You don’t have to download / install Watir separately or know anything about Watir. WET drives an IE Browser directly and so the automated testing done using WET is equivalent to how a user would drive the web pages. WET allows you to perform various checks as a part of the testing process by using Checkpoints. Requirement: Windows 98/ME/2000 SP3/XP SP2/Server 2003

  • Win32::IEAutomation: A Perl module which automates functional testing of web applications. It can be used to automate any complex web application including dynamic frames and popup windows. It is an object oriented module and all methods are like user actions on web browser. Requirement: Windows (only Internet Explorer is supported)

  • XML Test Suite: Provides a powerful way to test web applications. Writing tests requires only a knowledge of HTML and XML. We want XmlTestSuite to be adopted by testers, business analysts, and web developers who don’t have a java background. Requirement: Windows 95/98/2000, Windows NT/2000, Linux, SunOS/Solaris

  • Yawet: Visual web test tool from InforMatrix GmbH enables graphical creation of web app tests. Create, run and debug functional and regression tests for web applications. Can verify HTML, XML, and PDF’ ability to do report generation, reusable step libraires and parameterization. Freeware; download jar file and start by double-click or with command javaw -jar yawet.jar

Please comment below on your thoughts/experience on any of the tools listed above and/or if you know of any other tools that should be added.



Thursday, October 22, 2009

Open Source Test Management tools

Test Management tools are very important to any test team. Test teams use these tools to help capture requirements, design test cases, map test cases to requirements, test execution reports and much more. Companies may use one to many tools for this, which range from very expensive to open source. My advice would be to pick a tool that can meet most of your current and near future needs.

Below is a list of some open source/free tools that may come in handy.

  • Bugzilla Testopia: A test case management extension for Bugzilla. It is designed to be a generic tool for tracking test cases, allowing for testing organizations to integrate bug reporting with their test case run results. Though it is designed with software testing in mind, it can be used to track testing on virtually anything in the engineering process. Requirement: Bugzilla

  • Data Generator: The Data Generator is a free, open source script written in JavaScript, PHP and MySQL that lets you quickly generate large volumes of custom data in a variety of formats for use in testing software, populating databases and more.

  • Fitnesse: A collaborative testing and documentation tool. It provides a very simple way for teams to collaboratively create documents, specify tests, and run those tests. Requirement: Mac, Windows, POSIX

  • Kineo Open Source: Advises on the best open source tools tailored for a specific group of users

  • Mantis: Can be used for bug tracking as well.

  • MTS: Multi-TeSTer: A simple domain-specific language for maintaining tests that supports two desirable properties: – single source: the command, input, and expected output are all in one file. – controlled experimentation: multiple slightly different tests can be generated from the same source with different expected outputs. Requirement: Unix command line tool; written in Perl

  • qaManager: A web based application used for keeping track of engineering and customer releases of Software Projects, Resource allocations and related information. It helps QA Managers to keep track of releases
    effectively.Powered by OpenXava and Java.

  • QATraq: Covers everything from defining test plans to writing test cases and recording results. Requirement: Windows, BSD, Linux, SunOS/Solaris

  • Radi: Radi-testdir is a lightweight test director. Radi supports test director features like configuring the test plan, updating (create/edit) the test results for the test image/build stores in the image results set. Requirement: All POSIX (Linux/BSD/UNIX-like OSes)

  • rth: web-based tool designed to manage requirements, tests, test results, and defects throughout the application life cycle. The tool provides a structured approach to software testing and increases the visibility of the testing process by creating a common repository for all test assets including requirements, test cases, test plans, and test results. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP), All POSIX (Linux/BSD/UNIX-like OSes), IBM AIX

  • RTH-Turbo: Optimized version of RTH, a testing management tool, that allows: requirement management; test case management; defect tracking; create test plans; analyze your test results. This project was originally created from RTH version 1.2, and this new branch and version is more powerful and optimized. Requirement: PHP, MySql and Apache

  • Salome-TMF: An independent Test Management Tool, which helps you to manage your entire testing process by creating tests, executing manual or automatic tests, tracking results, managing requirements and defects and producing HTML documentation. Salome-TMF is compatible with Junit, Abbot and Beanshell to define your automatic tests, and with Bugzilla and Mantis to manage your defects. Salome-TMF can also be extended by plug-in according to your requirements. Requirement: Java

  • TCW: Test Case Web (TCW) is an online TCM system built with PHP and a SQL backend. It provides an efficient means for generation, organization, and execution reporting of test cases among projects and by multiple testers and versions. It provides various at-a-glance views of the test suite for easy status determination and test suite navigation. TCW also provides basic reporting capabilities and per-project access control. Requirement:Any (PHP/SQL/Apache)

  • Tesly: Tesly is a Web application written in PHP that helps you create, execute, and report on test plans. QA leaders can track the progress of testing as testers use the interface to report completion of test cases. Requirement: OS Independent

  • Test Environment Toolkit: Open source version of TETware, a proprietary multi-platform test framework for test suite management (administration, reporting and sequencing of tests). This open-source version is for Linux and UNIX system users only. The differences between the open source version and the other version for which a commercial support offering is available is in platform support (WIN32 and Java) rather than functionality. Requirement: Linux or Unix

  • Testitool: A Web-based application for QA test planning. It creates a test plan and populates it with test cases, maps test cases to functional requirements, instantiates a test plan, begins executing test cases and marks them as successful or failed, generates reports on your test plans, copies test plans and test cases, and tailors test plan instances by adding and removing test cases from them.Requirement: Apache, PHP, MySQL

  • TestLink: Web-based test management and test execution system allowing QA teams to create, manage, execute and track test cases and organize them into test plans. Requirement: Apache, MySQL, PHP

  • TestMaster: A testcase management, logging, reporting and test automation tool, similar to the commercial product Test Director. Features: Progress stats, reports, test case import from CSV,doc,web or SQL, STAF plugin. Requirement: Linux, Apache, PostgreSQL

  • WebTst: Tool aimed at creating and managing user-centric testing

  • XQual Studio (XStudio): A free 100% graphical and modular in design test management application that handles the complete life-cycle of your QA/testing projects from end to end: users, requirements, specifications, development projects (scrum oriented), SUTs, tests, tesplans, test reports and test campaigns

Please comment below on your thoughts/experience on any of the tools listed above and/or if you know of any other tools that should be added.