TestNG Annotations

In this article, we will come to know about all the annotations supported by TestNG.
TestNG supports many annotations and the philosophy behind using annotations is to bring the Test methods closer to any normal java method.

1. The test methods doesn’t need to follow any naming convention
2. They should be allowed to accept parameters like any other java method
3. The annotations allows further configuration using its attributes
4. Test classes doesn’t need to extend any base test class

Below is the list of TestNG annotations:

1. @BeforeSuite will be executed before any tests declared inside a TestNG suite.
2. @BeforeTest will be executed before each test section declared inside a TestNG suite.
3. @BeforeClass will be executed before any of the test methods of a test class.
4. @BeforeMethod will be executed before the execution of each test method.
5. @BeforeGroups will run before any of the test method of the specified group is executed. groups attribute must contain the list of groups this method belongs to.
6. @AfterSuite will be executed after any tests declared inside a TestNG suite.
7. @AfterTest will be executed after each test section declared inside a TestNG suite.
8. @AfterClass will be executed after the last test method of a test class.
9. @AfterMethod will be executed after the execution of each test method.
10. @AfterGroups will run after the last test method of the specified group is executed. groups attribute must contain the list of groups this method belongs to.
11. @DataProvider method provides the data for a test method and must return a two dimensional object array (Object[ ][ ]) as data.
12. @Factory method returns an array of class objects (Object[ ]). This is used to run a set of test cases with different values provided to the test class during its instantiation.
13. @Test marks a class or a method as a test method. If used at class level, all the public methods of a class will be considered as a test method.
14. @Listeners is defined at class level to specify an array of test listeners classes extending org.testng.ITestNGListener.
15. @Parameters is used to pass parameters to a test method. These parameter values are provided in the testng.xml configuration file at runtime.

Configuration Annotations

Configuration annotations start with @Before and @After annotations. Each of these annotations run at a specific event in the TestNG lifecycle. To know those events it is important that we understand the testng.xml structure. It consists of:

1. suite which is the first element of testng.xml.
2. A suite contains one or more test elements.
3. A test contains one more test classes
4. A class consists of one or more @test methods

A test is made of one or more classes and a class is made or one or more methods.

 

Let’s first run a simple testNg.xml which contains just a test class.

simpleTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="UnitTesting">
		<classes>
			<class name="com.javarticles.testng.TestA" />
		</classes>
	</test>
</suite>

As you see our testNg.xml is very simple, it just includes one test class TestA. Few things to note about:

1. You can always provide multiple methods of the same annotation type. For example, in the below class, I have two @BeforeSuite methods.
2. The annotation methods are allowed to accept some special typed parameters like Method and ITestContext. TestNG takes the responsibility to feed in the runtime object. In our example, beforeTest test method accepts ITestContext, this helps us to know the TestNG’s test section name.
3. We all the configuration annotation methods in our test class TestA. This will give us an idea of the sequence in which the methods are invoked.

TestA:

package com.javarticles.testng;

import org.testng.ITestContext;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestA {
    @BeforeSuite
    public void beforeSuite1() {
        System.out.println("TestA: beforeSuite1");
    }
    
    @BeforeSuite
    public void beforeSuite2() {
        System.out.println("TestA: beforeSuite2");
    }
    
    @BeforeTest
    public void beforeTest(ITestContext testContext) {
        System.out.println("TestA: beforeTest Test<" + testContext.getName() + ">");
    }
    
    @BeforeClass
    public void beforeClass() {
        System.out.println("TestA: beforeClass");
    }    
    
    @BeforeMethod
    public void beforeMethod() {
        System.out.println("TestA: beforeMethod");
    }
    
    @Test
    public void unitTest1A() {
        System.out.println("TestA: unitTest1A");
    }
    
    @Test(groups="feature1")
    public void unitTest2A() {
        System.out.println("TestA: unitTest2A, feature1 test");
    }    
    
    @Test
    public void unitTest3A() {
        System.out.println("TestA: unitTest3A");
    }
    
    @Test(groups="feature2")
    public void unitTest4A() {
        System.out.println("TestA: unitTest4A, feature2 test");
    }    
    
    @AfterMethod
    public void afterMethod() {
        System.out.println("TestA: afterMethod");
    }
    
    @AfterClass
    public void afterClass() {
        System.out.println("TestA: afterClass");
    }
    
    @AfterTest
    public void afterTest(ITestContext testContext) {
        System.out.println("TestA: afterTest Test<" + testContext.getName() + ">");
    }
    
    @AfterSuite
    public void afterSuite() {
        System.out.println("TestA: afterSuite");
    }
}

Right click on the testNg.xml file and then click on ‘Run-as->TestNG Suite’.

As you can see from the output, the sequence in which the configurations methods run are:

1. @BeforeSuite methods
2. @BeforeTest methods
3. @BeforeClass methods
4. @BeforeMethod methods
5. @Test methods
6. @AfterMethod methods
7. @AfterClass methods
8. @AfterTest methods
9. @AfterSuite methods

@BeforeMethod, @Test and @AfterMethod repeat for each test method.

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\simpleTestng.xml

TestA: beforeSuite1
TestA: beforeSuite2
TestA: beforeTest Test
TestA: beforeClass
TestA: beforeMethod
TestA: unitTest1A
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest2A, feature1 test
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest3A
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest4A, feature2 test
TestA: afterMethod
TestA: afterClass
TestA: afterTest Test
TestA: afterSuite

===============================================
Suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================

Multiple tests

You can also run multiple tests together. For example, I can group my tests into ‘Unit Tests’, ‘Functional Tests’ and ‘UI Tests’.

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="UnitTesting">
		<classes>
			<class name="com.javarticles.testng.TestA" />
		</classes>
	</test>
	<test name="FunctionalTesting">
		<classes>
			<class name="com.javarticles.testng.TestA" />
			<class name="com.javarticles.testng.TestB" />
		</classes>
	</test>
	<test name="UITesting">
		<classes>
			<class name="com.javarticles.testng.TestC" />
		</classes>
	</test>
</suite>

We have added two new test classes TestB and TestC.

TestB:

package com.javarticles.testng;

import org.testng.ITestContext;
import org.testng.annotations.AfterGroups;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestB {
    @BeforeSuite
    public void beforeSuite() {
        System.out.println("TestB: beforeSuite");
    }
    
    @BeforeTest
    public void beforeTest(ITestContext testContext) {
        System.out.println("TestB: beforeTest Test<" + testContext.getName() + ">");
    }
    
    @BeforeMethod
    public void beforeMethod() {
        System.out.println("TestB: beforeMethod");
    }
    
    @Test
    public void scenario1B() {
        System.out.println("TestB: Scenario1B");
    }
    
    @Test(groups="feature1")
    public void scenario2B() {
        System.out.println("TestB: Scenario2B, feature1 test");
    }    
    
    @AfterMethod
    public void afterMethod() {
        System.out.println("TestB: afterMethod");
    }
    
    @AfterTest
    public void afterTest(ITestContext testContext) {
        System.out.println("TestB: afterTest Test<" + testContext.getName() + ">");
    }
    
    @AfterSuite
    public void afterSuite() {
        System.out.println("TestB: afterSuite");
    }
    
    @BeforeGroups(groups="feature1")
    public void beforeGroup() {
        System.out.println("TestB: beforeGroup for feature1");
    }
    
    @AfterGroups(groups="feature1")
    public void afterGroup() {
        System.out.println("TestB: afterGroup for feature1");
    }
}

TestC:

package com.javarticles.testng;

import org.testng.ITestContext;
import org.testng.annotations.AfterGroups;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class TestC {
    @BeforeSuite
    public void beforeSuite() {
        System.out.println("TestC: beforeSuite");
    }
    
    @BeforeTest(alwaysRun=true)
    public void beforeTest(ITestContext testContext) {
        System.out.println("TestC: beforeTest Test<" + testContext.getName() + ">");
    }
    
    @BeforeMethod
    public void beforeMethod() {
        System.out.println("TestC: beforeMethod");
    }
    
    @Test
    public void uiSceario1C() {
        System.out.println("TestC: uiSceario1C");
    }    
    
    @Test(groups="feature1")
    public void uiScenario2C() {
        System.out.println("TestC: uiScenario2C, feature1 test");
    }
    
    @Test(groups="feature2")
    public void uiScenario3C() {
        System.out.println("TestC: uiScenario3C, feature2 test");
    }
    
    @AfterMethod(groups="feature1")
    public void afterMethod() {
        System.out.println("TestC: afterMethod");
    }
    
    @AfterTest(alwaysRun=true)
    public void afterTest(ITestContext testContext) {
        System.out.println("TestC: afterTest Test<" + testContext.getName() + ">");
    }
    
    @AfterSuite
    public void afterSuite() {
        System.out.println("TestC: afterSuite");
    }
    
    @BeforeGroups(groups="feature2")
    public void beforeGroup() {
        System.out.println("TestC: beforeGroup for feature2");
    }
    
    @AfterGroups(groups="feature2")
    public void afterGroup() {
        System.out.println("TestC: afterGroup for feature2");
    }
}

That’s lot of output! If you note the sequence of execution remains same. We now see couple of new annotation methods @BeforeGroup and @AfterGroup methods. They fire whenever a test method is associated to a group is invoked.

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\multiTestsTestng.xml

TestA: beforeSuite1
TestA: beforeSuite2
TestB: beforeSuite
TestC: beforeSuite
TestA: beforeTest Test
TestA: beforeClass
TestA: beforeMethod
TestA: unitTest1A
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest2A, feature1 test
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest3A
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest4A, feature2 test
TestA: afterMethod
TestA: afterClass
TestA: afterTest Test
TestA: beforeTest Test
TestB: beforeTest Test
TestA: beforeClass
TestA: beforeMethod
TestA: unitTest1A
TestA: afterMethod
TestB: beforeGroup for feature1
TestA: beforeMethod
TestA: unitTest2A, feature1 test
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest3A
TestA: afterMethod
TestA: beforeMethod
TestA: unitTest4A, feature2 test
TestA: afterMethod
TestA: afterClass
TestB: beforeMethod
TestB: Scenario1B
TestB: afterMethod
TestB: beforeMethod
TestB: Scenario2B, feature1 test
TestB: afterMethod
TestB: afterGroup for feature1
TestA: afterTest Test
TestB: afterTest Test
TestC: beforeTest Test
TestC: beforeMethod
TestC: uiSceario1C
TestC: afterMethod
TestC: beforeMethod
TestC: uiScenario2C, feature1 test
TestC: afterMethod
TestC: beforeGroup for feature2
TestC: beforeMethod
TestC: uiScenario3C, feature2 test
TestC: afterMethod
TestC: afterGroup for feature2
TestC: afterTest Test
TestA: afterSuite
TestB: afterSuite
TestC: afterSuite

===============================================
Suite
Total tests run: 13, Failures: 0, Skips: 0
===============================================

Run groups

You can also group your tests based on your feature using <groups> element. For example,

groupTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="Feature1">
		<groups>
			<run>
				<include name="feature1" />
			</run>
		</groups>
		<classes>
			<class name="com.javarticles.testng.TestA" />
			<class name="com.javarticles.testng.TestB" />
			<class name="com.javarticles.testng.TestC" />
		</classes>
	</test>
	<test name="Feature2">
		<groups>
			<run>
				<include name="feature2" />
			</run>
		</groups>
		<classes>
			<class name="com.javarticles.testng.TestA" />
			<class name="com.javarticles.testng.TestC" />
		</classes>
	</test>
</suite>

One thing to note when you are running the groups:
The configuration annotated methods won’t run unless you specify the group in the groups attribute or set alwaysRun to true. For example, in our TestC, we have included both the cases.

    @AfterMethod(groups="feature1")
    public void afterMethod() {
        System.out.println("TestC: afterMethod");
    }
    
    @AfterTest(alwaysRun=true)
    public void afterTest(ITestContext testContext) {
        System.out.println("TestC: afterTest Test<" + testContext.getName() + ">");
    }

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\groupTestng.xml

TestC: beforeTest Test
TestB: beforeGroup for feature1
TestA: unitTest2A, feature1 test
TestB: Scenario2B, feature1 test
TestC: uiScenario2C, feature1 test
TestC: afterMethod
TestB: afterGroup for feature1
TestC: afterTest Test
TestC: beforeTest Test
TestC: beforeGroup for feature2
TestA: unitTest4A, feature2 test
TestC: uiScenario3C, feature2 test
TestC: afterGroup for feature2
TestC: afterTest Test

===============================================
Suite
Total tests run: 5, Failures: 0, Skips: 0
===============================================

Example of @Parameters

One of the important features of TestNG is data-driven testing.
One can pass parameter values to test methods as arguments following the below methods:

1. Using testng.xml configuration file.
2. Through DataProviders

Parameterization is about defining the parameters and its value in testng.xml and then referring to the parameters using @Parameters annotation at method level. The attributes section of @Parameters will have the array of parameters used.
The parameters are defined in testng.xml using the parameter element. Its name and value are specified using the attributes name and value.

paramTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<parameter name="param1" value="One"></parameter>
	<parameter name="param2" value="1"></parameter>
	<test name="ParamTesting">
		<classes>
			<class name="com.javarticles.testng.TestParam" />
		</classes>
	</test>
</suite>

Now we use these parameters in the test method.

TestParam:

package com.javarticles.testng;

import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class TestParam {

    @Parameters({"param1", "param2"})
    @Test
    public void t(String p1, String p2) {
        System.out.println("t(" + p1 + ", " + p2 + ")" );
    }        
}

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\paramTestng.xml

t(One, 1)

===============================================
Suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

For more example on @Parameters, click here

Example of @DataProvider

One can provide data using @DataProvider annotated method.The method must return two dimensional object[][] array. The first array represents data set thus takes care of the number of time the tests has to run. The second array represents number of parameters. You can even provide a name to it else by default the method name becomes its name.

If a test is referring to a DataProvider for its parameter values then it has to specify the dataProvider attribute in its Test annotation.
A data-driven test will run once for each set of data passed-in and the data will be provided by the method annotated with @DataProvider.

In the below example, dp() is my data provider method. It returns two sets of data. Each set contains two parameter values. First set is ‘A’ and ‘a’. The second set is ‘B’ and ‘b’.
This is use to inject values to test method alphabets. Since the DataProvider has no name, we refer to it using the method name dp.

TestDataProvider:

package com.javarticles.testng;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class TestDataProvider {
    @Test(dataProvider="dp")
    public void alphabets(String upperCase, String lowerCase) {
        System.out.println("alphabets(" + upperCase + ", " + lowerCase + ")");
    }
     
    @DataProvider
    public Object[][] dp() {
        return new Object[][]{{"A", "a"}, {"B", "b"}};
    }
}

dataProviderTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="DataProviderTesting">
		<classes>
			<class name="com.javarticles.testng.TestDataProvider" />
		</classes>
	</test>
</suite>

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\dataProviderTestng.xml

alphabets(A, a)
alphabets(B, b)

===============================================
Suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

For more examples on @DataProvider, click here

Example of @Factory

We can use TestNG @Factory to create multiple test instances. The test class may either have the default no-arg constructor or constructor with one or more arguments.

In the below example, create is our @Factory method. It creates three instances of TestFactory with different set of data.

TestFactory:

package com.javarticles.testng;

import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class TestFactory {
    private String param;
    
    public TestFactory(String param) {
        this.param = param;
    }
 
    @Test
    public void t() {
        System.out.println("TestFactory.t:" + param);
    }
     
    @Factory
    public Object[] create() {
        return new Object[] { new TestFactory("one"), new TestFactory("two"), new TestFactory("three")};
    }
}

factoryTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="FactoryTesting">
		<classes>
			<class name="com.javarticles.testng.TestFactory" />
		</classes>
	</test>
</suite>

When we run the factoryTestng.xml, we end up running the test case thrice, once for each test instance.

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\factoryTestng.xml

TestFactory.t:two
TestFactory.t:three
TestFactory.t:one

===============================================
Suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================

For more examples on @Factory, click here.

Example of @Listeners

In TestNG, all listeners extend the marker interface ITestNGListener. You can register listeners using @Listeners at class level. We specify here an array of test listeners class that extend org.testng.ITestNGListener.
Implement an ISuiteListener.

SuiteListener:

package com.javarticles.testng;

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class SuiteListener implements ISuiteListener {

    @Override
    public void onStart(ISuite suite) {
        System.out.println("Start of suite " + suite.getName());
    }

    @Override
    public void onFinish(ISuite suite) {
        System.out.println("Finish of suite " + suite.getName());
    }

}

Using Listeners annotation specify the above listener.

TestListener:

package com.javarticles.testng;

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners({SuiteListener.class})
public class TestListener {

    @Test
    public void t() {
        System.out.println("test t()" );
    }        
}

listenerTestng.xml:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite">
	<test name="TestingListener">
		<classes>
			<class name="com.javarticles.testng.TestListener" />
		</classes>
	</test>
</suite>

Output:

[TestNG] Running:
  C:\javarticles_ws\testNGAnnotations\listenerTestng.xml

Start of suite Suite
test t()
Finish of suite Suite

===============================================
Suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

One particular pitfall TestNG may cause is, that TestNG does not guarantee that i.e. a @BeforeTest method is directly run before the Test. It only guarantees that it was executed at some point in time before starting the test. (Based on bentolor comments in Reddit).

If you want to run a specific method right before or after the invocation of a test method then implement a IInvokedMethodListener. You need to implement beforeInvocation and afterInvocation, compare the method name to make sure that the test method being invoked (or already invoked) is the one you are interested in.

For example, in the below InvokedTestMethodListenerExample, we want to run a method right before t2() and after t2(). We will implement IInvokedMethodListener and add it to our @Listeners annotation.

InvokedTestMethodListenerExample:

package com.javarticles.testng;

import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(InvokedTestMethodListener.class)
public class InvokedTestMethodListenerExample {

    @BeforeTest
    public void beforeTest1() {
        System.out.println("beforeTest1");
    }

    @BeforeTest
    public void beforeTest2() {
        System.out.println("beforeTest2");
    }

    @Test
    public void t1() {
        System.out.println("t1");
    }

    @Test
    public void t2() {
        System.out.println("t2");
    }

    @Test
    public void t3() {
        System.out.println("t3");
    }

    @Test
    public void t4() {
        System.out.println("t4");
    }

    @AfterTest
    public void afterTest1() {
        System.out.println("afterTest1");
    }

    @AfterTest
    public void afterTest2() {
        System.out.println("afterTest2");
    }
}

InvokedTestMethodListener:

package com.javarticles.testng;

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class InvokedTestMethodListener implements IInvokedMethodListener {

    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
        if ("t2".equals(method.getTestMethod().getMethodName())) {
            System.out.println("Invoked just before t2()");
        }
    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
        if ("t2".equals(method.getTestMethod().getMethodName())) {
            System.out.println("Invoked just after t2()");
        }
    }
}

Output:

[TestNG] Running:
  C:\Users\mokkara\AppData\Local\Temp\testng-eclipse-1301133492\testng-customsuite.xml

beforeTest1
beforeTest2
t1
Invoked just before t2()
t2
Invoked just after t2()
t3
t4
afterTest1
afterTest2
PASSED: t1
PASSED: t2
PASSED: t3
PASSED: t4

For more examples on TestNG listeners, click here

Download source code

In this article, I showed you examples of TestNG annotations. You can download the source code here: testNGAnnotations.zip