Continuous Integration and Testing

Configuring Bamboo, Hudson, and TestMaker

Operate PushToTest TestMaker tests from Continuous Integration environments. PushToTest checks TestMaker compatibility with Hudson and Bamboo.

• Product Version Details
• Setup Build to run tests in Hudson (Windows - localhost)
• Setup Build to run tests in Hudson (Linux - localhost)
• Setup Build to run tests in Bamboo (Windows - localhost)
• Setup Build to run tests in Bamboo (Linux - localhost)
• Setup Build to run tests in Bamboo (Atlassian/JIRA Hosted)
• Configure/execute ANT based targets (Atlassian/JIRA Hosted)
  

1. Product Version Details

TestMaker Version : 5.5 RC3
Bamboo Version : 2.6.2 (Standalone)
Hudson Version : 1.374
Java Verison : 1.6 (1.6.18 or later)
Tomcat Verison : 6.0.26 (To run Hudson)

2. Setup Build to run tests in Hudson (Windows - localhost)

• Setup a build in Hudson

• Pre-requisite to execute the build

• Execute the build in Hudson

• Configure the build/test output to detect failures in Junit output

Setup a build in Hudson:

Hudson comes with a web archive file (hudson.war). You need a web server (tomcat/jboss/jetty) to run Hudson.

Tomcat Console (runs in 8090 http port)

Hudson Home URL: http://localhost:8090/hudson/

Create New Job link to create a new Job

Click OK to proceed

You can select Source Code Management option to checkout the code from (CVS/Subversion). All the source/tests will checkout in the build workspace directory.

Select Execute Windows build Command from the Build

Type the following command

%TESTMAKER_HOME%\TestMaker.cmd -t example_agents\AjaxRIASeleniumTest_Tutorial\Calendar_Functional_Test.scenario -log-junit testreport\junitreport.xml

Note: The scenario file path (example_agents\AjaxRIASeleniumTest_Tutorial\Calendar_Functional_Test.scenario) and reports path (testreport\junitreport.xml

) is relative to your build workspace.

Click Save.

Pre-requisite to execute the build:

Create Environment variables:

Create the below two environment variables in Hudson Console (Hudson -> Manage Hudson -> Configure Your System -> Global Properties) as shown below:

JAVA_HOME

TESTMAKER_HOME

Or you can create these two system environment variables (My computer -> Advanced -> Environment Variables)

Start TestNode:

If required, start the Testmaker testnode. The current example is using the local testnode which runs in 8080 http port.

Testnode console

Execute the build in Hudson:

Click on Build Now link available in the left side to start the build

Due to some reason, the first build failed, so I ran the build for the second time.

You can watch the logs by clicking the build link

Click on the Console output

In the meantime, you can watch the testnode logs:

Configure the build/test output to detect failures in Junit output:

Check the Publish JUnit test result report option from the Post-build Actions

Note: You may not have the report file for the first time build. So, do not select this option until unless you have the report file (junitreport.xml) available in the specified directory.

Click on the Test Result

3. Setup Build to run tests in Hudson (Linux - localhost)

• Setup a build in Hudson

• Pre-requisite to execute the build

• Execute the build in Hudson

• Configure the build/test output to detect failures in Junit output

Setup a build in Hudson:

Same as windows - Please refer windows section

You can select Source Code Management option to checkout the code from (CVS/Subversion). All the source/tests will checkout in the build workspace directory.

Select Execute Shell from the Build

Type the following command

$TESTMAKER_HOME/TestMaker.sh -t example_agents/AjaxRIASeleniumTest_Tutorial/Calendar_Functional_Test.scenario -log-junit testreport/junitreport.xml

Note: The scenario file path (example_agents/AjaxRIASeleniumTest_Tutorial/Calendar_Functional_Test.scenario) and reports path (testreport/junitreport.xml

) is relative to your build workspace.

Make sure the / , $, TestMaker.sh

Click Save.

Pre-requisite to execute the build:

Create Environment variables:

--Same as Windows

But only the Path is going to change. It has to be valid unix path.

Start the TestNode:

--Same as Windows

Execute the build in Hudson:

--Same as Windows

Configure the build/test output to detect failures in Junit output:

--Same as Windows

4. Setup Build to run tests in Bamboo (Windows - localhost)

• Setup a build in Bamboo

• Pre-requisite to execute the build

• Execute the build in Bamboo

• Configure the build/test output to detect failures in Junit output

Setup build in Bamboo

Start Bamboo Server

Launch Bamboo console in browser and login with your admin user.

(By default it runs in 8085 port. So the URL would be http://localhost:8085/ )

Click on Create Plan link (key shortcut: Alt-Shift-C) to start creating a new plan:

Start filling all the details:

Source Repository Details (It supports CVS, Subversion & Perforce. But JIRA hosted bamboo only supports Subversion)

Select the Ant / Command Builder type

Ant Builder type

If Command Builder type is not available, click on the Add New Builder to add one.

Either we can provide full path of the command or append the command path in PATH System environment variable and call the command only.

Command Builder type

Requirements:

Artifacts: (No artifacts setup for this)

Notification: (No notification setup for this)

Post Action: (No Post Actions setup for this)

Select the appropriate permissions:

Click Save to create the plan, which will create the first build immediately.

Pre-requisite to execute the build:

Create Environment variables:

Create these two system environment variables (My computer -> Advanced -> Environment Variables)

JAVA_HOME

TESTMAKER_HOME

Create one Global Variable in Bamboo administrator Console (Home -> Administration -> Global Variable)

Note : Why we need to create the TESTMAKER_HOME variable in two places?

Ans: As we can not access the system enviorment variables inside bamboo. To execute the TestMaker.cmd command, we need to tell Bamboo where to lookup the command. This we can achive by setting in PATH varibale also as PATH=%PATH%;%TESTMAKER_HOME%

Start TestNode:

If required, start the Testmaker testnode. The current example is using the local testnode which runs in 8080 http port.

Testnode console

Execute the Build plan in Bamboo

When you save the build for first time, it automatically triggers the build.

Or select the build and click Run Build under Plan Actions

Monitor the result in the running log.

Configure the build/test output to detect failures in Junit output

Upon completion of build, you can see your report in the following section:

Click Test

5. Setup Build to run tests in Bamboo (Linux - localhost)

• Setup a build in Bamboo

• Pre-requisite to execute the build

• Execute the build in Bamboo

• Configure the build/test output to detect failures in Junit output

Setup build in Bamboo

Start Bamboo Server

Launch Bamboo console in browser and login with your admin user.

-- Same as Windows

Start filling all the details:

Make sure the path is linux based.

Select the Script Builder type

Script Builder type

For default installation click next-> next and click Save/Finish to create the plan, which will create the first build immediately.

Pre-requisite to execute the build:

Create Environment variables:

Create these two system environment variables in user’s .profile file

JAVA_HOME

TESTMAKER_HOME

Command to modify .profile file (if you do not have .profile file, please create a file with the name .profile) as shown below:

vi ~/.profile

# add the following lines at the end of the file

JAVA_HOME=<<JAVA_INSTALL_PATH>>

TESTMAKER_HOME=<<Testmaker_INSTALL_PATH>>

export JAVA_HOME TESTMAKER_HOME

Create one Global Variable in Bamboo administrator Console (Home -> Administration -> Global Variable)

Note : Why we need to create the TESTMAKER_HOME variable in two places?

Ans: As we can not access the system enviorment variables inside bamboo. To execute the TestMaker.cmd command, we need to tell Bamboo where to lookup the command.

Start TestNode:

-- Same as Windows

Execute the Build plan in Bamboo

-- Same as Windows

Configure the build/test output to detect failures in Junit output

-- Same as Windows

6. Setup Build to run tests in Bamboo (Atlassian/JIRA Hosted)

Login to JIRA System (http://pushtotest.jira.com/secure/Dashboard.jspa)

Click on the

It will list down all the projects with their build plans.

To create a New Build plan click

Select project name, Build Plan Name and prove a unique Build plan Key. Click Next to enter source repository details:

Click Save to create the plan.

7. Configure/execute ANT based targets (Atlassian/JIRA Hosted)

The community version of TestMaker can have 3 different versions according to the OS.

Click on any of these build link will take you to the build plan screen.

To run a new build click Run Build from Plan Actions. To Edit the plan, click Edit Plan option.

Note: The Bamboo remote agent is configured in Amazon EC2. So, whenever we run a build, it checks for the agent. If not started, click on the following “Run Elasitc Instance” as shown below. We can start the elastic instance using other mechanism as well.

Click on “State Elastic Instance” link

Select # of instances you want to run and then click Submit

The remote instance and then the remote bamboo agent will start automatically and the queued Build will start automatically.

Click the Build link will show you the live logs of your build.

If the build successful, you can see the following screen.

If it failed, you will get the following screen

Monitor the changes:

Click on diffs will give you a clear indication of change:

Do not forget to shutdown the elastic instance which was started manually by us to run this build.

Goto (Administration -> Builds -> Elastic Bamboo -> Instances) and click Shutdown or Shutdown all instances link.

(http://pushtotest.jira.com/builds/admin/agent/configureAgents!default.action)

Click Confirm to shutdown.

Shutdown will take 30-40 secs. So, make sure it shutdown properly.

2. To build enterprise TestMaker product (Windows/Mac/Linux)

Select Project Enterprise TestMaker and select the build as shown below:

Repeat the same steps as described to build the Community Product.

3. To build standalone PTTMonitor (Windows (32 and 64 bit)/Mac/Linux)

- TBD -

Repeat the same steps as described to build the Community Product.

4. To build standalone Testnode (Windows/Mac/Linux)

- TBD -

Repeat the same steps as described for to build the Community Product.

5. To Run Tests (Example Agents)

Repeat the same steps as described for to build the Community Product.

6. To deploy installers in Amazon S3

- TBD -

7. To Compiling/building EC2 AMI file

- TBD -

8. Start cloud testnode

Goto (Administration -> Builds -> Elastic Bamboo -> Instances) and click Start New Elastic Instances

(http://pushtotest.jira.com/builds/admin/elastic/manageElasticInstances.action)

Additional documentation, product downloads and updates are at www.PushToTest.com. While the PushToTest testMaker software is distributed under an open-source license, the documenation remains (c) 2011 PushToTest. All rights reserved. PushToTest is a trademark of the PushToTest company.