TestNG supports many annotations and the philosophy behind using annotations is to bring the Test methods closer to any normal java method.
- The test methods doesn’t need to follow any naming convention
- They should be allowed to accept parameters like any other java method
- The annotations allows further configuration using its attributes
- Test classes doesn’t need to extend any base test class
Below is the list of TestNG annotations:
@BeforeSuite
will be executed before any tests declared inside a TestNG suite.@BeforeTest
will be executed before each test section declared inside a TestNG suite.@BeforeClass
will be executed before any of the test methods of a test class.@BeforeMethod
will be executed before the execution of each test method.@BeforeGroups
will run before any of the test method of the specified group is executed.groups
attribute must contain the list of groups this method belongs to.@AfterSuite
will be executed after any tests declared inside a TestNG suite.@AfterTest
will be executed after each test section declared inside a TestNG suite.@AfterClass
will be executed after the last test method of a test class.@AfterMethod
will be executed after the execution of each test method.@AfterGroups
will run after the last test method of the specified group is executed.groups
attribute must contain the list of groups this method belongs to.@DataProvider
method provides the data for a test method and must return a two dimensional object array (Object[ ][ ]) as data.@Factory
method returns an array of class objects (Object[ ]). This is used to run a set of test cases with different values provided to the test class during its instantiation.@Test
marks a class or a method as a test method. If used at class level, all the public methods of a class will be considered as a test method.@Listeners
is defined at class level to specify an array of test listeners classes extendingorg.testng.ITestNGListener
.@Parameters
is used to pass parameters to a test method. These parameter values are provided in thetestng.xml
configuration file at runtime.
Configuration Annotations
Configuration annotations start with @Before
and @After
annotations. Each of these annotations run at a specific event in the TestNG lifecycle. To know those events it is important that we understand the testng.xml
structure. It consists of:
suite
which is the first element oftestng.xml
.- A
suite
contains one or moretest
elements. - A
test
contains one more test classes - A class consists of one or more
@test
methods
A test
is made of one or more classes
and a class
is made or one or more methods.
Let’s first run a simple testNg.xml
which contains just a test class.
simpleTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="UnitTesting"> <classes> <class name="com.javarticles.testng.TestA" /> </classes> </test> </suite>
As you see our testNg.xml
is very simple, it just includes one test class TestA
. Few things to note about:
- You can always provide multiple methods of the same annotation type. For example, in the below class, I have two
@BeforeSuite
methods. - The annotation methods are allowed to accept some special typed parameters like
Method
andITestContext
. TestNG takes the responsibility to feed in the runtime object. In our example,beforeTest
test method acceptsITestContext
, this helps us to know the TestNG’s test section name. - We all the configuration annotation methods in our test class
TestA
. This will give us an idea of the sequence in which the methods are invoked.
TestA:
package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterClass; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestA { @BeforeSuite public void beforeSuite1() { System.out.println("TestA: beforeSuite1"); } @BeforeSuite public void beforeSuite2() { System.out.println("TestA: beforeSuite2"); } @BeforeTest public void beforeTest(ITestContext testContext) { System.out.println("TestA: beforeTest Test<" + testContext.getName() + ">"); } @BeforeClass public void beforeClass() { System.out.println("TestA: beforeClass"); } @BeforeMethod public void beforeMethod() { System.out.println("TestA: beforeMethod"); } @Test public void unitTest1A() { System.out.println("TestA: unitTest1A"); } @Test(groups="feature1") public void unitTest2A() { System.out.println("TestA: unitTest2A, feature1 test"); } @Test public void unitTest3A() { System.out.println("TestA: unitTest3A"); } @Test(groups="feature2") public void unitTest4A() { System.out.println("TestA: unitTest4A, feature2 test"); } @AfterMethod public void afterMethod() { System.out.println("TestA: afterMethod"); } @AfterClass public void afterClass() { System.out.println("TestA: afterClass"); } @AfterTest public void afterTest(ITestContext testContext) { System.out.println("TestA: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestA: afterSuite"); } }
Right click on the testNg.xml
file and then click on ‘Run-as->TestNG Suite’.
As you can see from the output, the sequence in which the configurations methods run are:
@BeforeSuite
methods@BeforeTest
methods@BeforeClass
methods@BeforeMethod
methods@Test
methods@AfterMethod
methods@AfterClass
methods@AfterTest
methods@AfterSuite
methods
@BeforeMethod
, @Test
and @AfterMethod
repeat for each test method.
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\simpleTestng.xml TestA: beforeSuite1 TestA: beforeSuite2 TestA: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestA: afterTest Test TestA: afterSuite =============================================== Suite Total tests run: 4, Failures: 0, Skips: 0 ===============================================
Multiple tests
You can also run multiple tests together. For example, I can group my tests into ‘Unit Tests’, ‘Functional Tests’ and ‘UI Tests’.
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="UnitTesting"> <classes> <class name="com.javarticles.testng.TestA" /> </classes> </test> <test name="FunctionalTesting"> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestB" /> </classes> </test> <test name="UITesting"> <classes> <class name="com.javarticles.testng.TestC" /> </classes> </test> </suite>
We have added two new test classes TestB
and TestC
.
TestB:
package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterGroups; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeGroups; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestB { @BeforeSuite public void beforeSuite() { System.out.println("TestB: beforeSuite"); } @BeforeTest public void beforeTest(ITestContext testContext) { System.out.println("TestB: beforeTest Test<" + testContext.getName() + ">"); } @BeforeMethod public void beforeMethod() { System.out.println("TestB: beforeMethod"); } @Test public void scenario1B() { System.out.println("TestB: Scenario1B"); } @Test(groups="feature1") public void scenario2B() { System.out.println("TestB: Scenario2B, feature1 test"); } @AfterMethod public void afterMethod() { System.out.println("TestB: afterMethod"); } @AfterTest public void afterTest(ITestContext testContext) { System.out.println("TestB: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestB: afterSuite"); } @BeforeGroups(groups="feature1") public void beforeGroup() { System.out.println("TestB: beforeGroup for feature1"); } @AfterGroups(groups="feature1") public void afterGroup() { System.out.println("TestB: afterGroup for feature1"); } }
TestC:
package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterGroups; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeGroups; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestC { @BeforeSuite public void beforeSuite() { System.out.println("TestC: beforeSuite"); } @BeforeTest(alwaysRun=true) public void beforeTest(ITestContext testContext) { System.out.println("TestC: beforeTest Test<" + testContext.getName() + ">"); } @BeforeMethod public void beforeMethod() { System.out.println("TestC: beforeMethod"); } @Test public void uiSceario1C() { System.out.println("TestC: uiSceario1C"); } @Test(groups="feature1") public void uiScenario2C() { System.out.println("TestC: uiScenario2C, feature1 test"); } @Test(groups="feature2") public void uiScenario3C() { System.out.println("TestC: uiScenario3C, feature2 test"); } @AfterMethod(groups="feature1") public void afterMethod() { System.out.println("TestC: afterMethod"); } @AfterTest(alwaysRun=true) public void afterTest(ITestContext testContext) { System.out.println("TestC: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestC: afterSuite"); } @BeforeGroups(groups="feature2") public void beforeGroup() { System.out.println("TestC: beforeGroup for feature2"); } @AfterGroups(groups="feature2") public void afterGroup() { System.out.println("TestC: afterGroup for feature2"); } }
That’s lot of output! If you note the sequence of execution remains same. We now see couple of new annotation methods @BeforeGroup
and @AfterGroup
methods. They fire whenever a test method is associated to a group is invoked.
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\multiTestsTestng.xml TestA: beforeSuite1 TestA: beforeSuite2 TestB: beforeSuite TestC: beforeSuite TestA: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestA: afterTest Test TestA: beforeTest Test TestB: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestB: beforeGroup for feature1 TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestB: beforeMethod TestB: Scenario1B TestB: afterMethod TestB: beforeMethod TestB: Scenario2B, feature1 test TestB: afterMethod TestB: afterGroup for feature1 TestA: afterTest Test TestB: afterTest Test TestC: beforeTest Test TestC: beforeMethod TestC: uiSceario1C TestC: afterMethod TestC: beforeMethod TestC: uiScenario2C, feature1 test TestC: afterMethod TestC: beforeGroup for feature2 TestC: beforeMethod TestC: uiScenario3C, feature2 test TestC: afterMethod TestC: afterGroup for feature2 TestC: afterTest Test TestA: afterSuite TestB: afterSuite TestC: afterSuite =============================================== Suite Total tests run: 13, Failures: 0, Skips: 0 ===============================================
Run groups
You can also group your tests based on your feature using <groups>
element. For example,
groupTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="Feature1"> <groups> <run> <include name="feature1" /> </run> </groups> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestB" /> <class name="com.javarticles.testng.TestC" /> </classes> </test> <test name="Feature2"> <groups> <run> <include name="feature2" /> </run> </groups> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestC" /> </classes> </test> </suite>
One thing to note when you are running the groups:
The configuration annotated methods won’t run unless you specify the group in the groups
attribute or set alwaysRun
to true. For example, in our TestC
, we have included both the cases.
@AfterMethod(groups="feature1") public void afterMethod() { System.out.println("TestC: afterMethod"); } @AfterTest(alwaysRun=true) public void afterTest(ITestContext testContext) { System.out.println("TestC: afterTest Test<" + testContext.getName() + ">"); }
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\groupTestng.xml TestC: beforeTest Test TestB: beforeGroup for feature1 TestA: unitTest2A, feature1 test TestB: Scenario2B, feature1 test TestC: uiScenario2C, feature1 test TestC: afterMethod TestB: afterGroup for feature1 TestC: afterTest Test TestC: beforeTest Test TestC: beforeGroup for feature2 TestA: unitTest4A, feature2 test TestC: uiScenario3C, feature2 test TestC: afterGroup for feature2 TestC: afterTest Test =============================================== Suite Total tests run: 5, Failures: 0, Skips: 0 ===============================================
Example of @Parameters
One of the important features of TestNG is data-driven testing.
One can pass parameter values to test methods as arguments following the below methods:
- Using
testng.xml
configuration file. - Through DataProviders
Parameterization is about defining the parameters and its value in testng.xml
and then referring to the parameters using @Parameters
annotation at method level. The attributes section of @Parameters
will have the array of parameters used.
The parameters are defined in testng.xml
using the parameter
element. Its name and value are specified using the attributes name
and value
.
paramTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <parameter name="param1" value="One"></parameter> <parameter name="param2" value="1"></parameter> <test name="ParamTesting"> <classes> <class name="com.javarticles.testng.TestParam" /> </classes> </test> </suite>
Now we use these parameters in the test method.
TestParam:
package com.javarticles.testng; import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestParam { @Parameters({"param1", "param2"}) @Test public void t(String p1, String p2) { System.out.println("t(" + p1 + ", " + p2 + ")" ); } }
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\paramTestng.xml t(One, 1) =============================================== Suite Total tests run: 1, Failures: 0, Skips: 0 ===============================================
For more example on @Parameters
, click here
Example of @DataProvider
One can provide data using @DataProvider
annotated method.The method must return two dimensional object[][]
array. The first array represents data set thus takes care of the number of time the tests has to run. The second array represents number of parameters. You can even provide a name to it else by default the method name becomes its name.
If a test is referring to a DataProvider for its parameter values then it has to specify the dataProvider
attribute in its Test
annotation.
A data-driven test will run once for each set of data passed-in and the data will be provided by the method annotated with @DataProvider
.
In the below example, dp()
is my data provider method. It returns two sets of data. Each set contains two parameter values. First set is ‘A’ and ‘a’. The second set is ‘B’ and ‘b’.
This is use to inject values to test method alphabets
. Since the DataProvider
has no name, we refer to it using the method name dp
.
TestDataProvider:
package com.javarticles.testng; import org.testng.annotations.DataProvider; import org.testng.annotations.Test; public class TestDataProvider { @Test(dataProvider="dp") public void alphabets(String upperCase, String lowerCase) { System.out.println("alphabets(" + upperCase + ", " + lowerCase + ")"); } @DataProvider public Object[][] dp() { return new Object[][]{{"A", "a"}, {"B", "b"}}; } }
dataProviderTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="DataProviderTesting"> <classes> <class name="com.javarticles.testng.TestDataProvider" /> </classes> </test> </suite>
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\dataProviderTestng.xml alphabets(A, a) alphabets(B, b) =============================================== Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================
For more examples on @DataProvider
, click here
Example of @Factory
We can use TestNG @Factory
to create multiple test instances. The test class may either have the default no-arg constructor or constructor with one or more arguments.
In the below example, create
is our @Factory
method. It creates three instances of TestFactory
with different set of data.
TestFactory:
package com.javarticles.testng; import org.testng.annotations.Factory; import org.testng.annotations.Test; public class TestFactory { private String param; public TestFactory(String param) { this.param = param; } @Test public void t() { System.out.println("TestFactory.t:" + param); } @Factory public Object[] create() { return new Object[] { new TestFactory("one"), new TestFactory("two"), new TestFactory("three")}; } }
factoryTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="FactoryTesting"> <classes> <class name="com.javarticles.testng.TestFactory" /> </classes> </test> </suite>
When we run the factoryTestng.xml
, we end up running the test case thrice, once for each test instance.
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\factoryTestng.xml TestFactory.t:two TestFactory.t:three TestFactory.t:one =============================================== Suite Total tests run: 3, Failures: 0, Skips: 0 ===============================================
For more examples on @Factory
, click here.
Example of @Listeners
In TestNG, all listeners extend the marker interface ITestNGListener
. You can register listeners using @Listeners
at class level. We specify here an array of test listeners class that extend org.testng.ITestNGListener
.
Implement an ISuiteListener
.
SuiteListener:
package com.javarticles.testng; import org.testng.ISuite; import org.testng.ISuiteListener; public class SuiteListener implements ISuiteListener { @Override public void onStart(ISuite suite) { System.out.println("Start of suite " + suite.getName()); } @Override public void onFinish(ISuite suite) { System.out.println("Finish of suite " + suite.getName()); } }
Using Listeners
annotation specify the above listener.
TestListener:
package com.javarticles.testng; import org.testng.annotations.Listeners; import org.testng.annotations.Test; @Listeners({SuiteListener.class}) public class TestListener { @Test public void t() { System.out.println("test t()" ); } }
listenerTestng.xml:
<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="TestingListener"> <classes> <class name="com.javarticles.testng.TestListener" /> </classes> </test> </suite>
Output:
[TestNG] Running: C:\javarticles_ws\testNGAnnotations\listenerTestng.xml Start of suite Suite test t() Finish of suite Suite =============================================== Suite Total tests run: 1, Failures: 0, Skips: 0 ===============================================
One particular pitfall TestNG may cause is, that TestNG does not guarantee that i.e. a @BeforeTest
method is directly run before the Test. It only guarantees that it was executed at some point in time before starting the test. (Based on bentolor comments in Reddit).
If you want to run a specific method right before or after the invocation of a test method then implement a IInvokedMethodListener
. You need to implement beforeInvocation
and afterInvocation
, compare the method name to make sure that the test method being invoked (or already invoked) is the one you are interested in.
For example, in the below InvokedTestMethodListenerExample
, we want to run a method right before t2()
and after t2()
. We will implement IInvokedMethodListener
and add it to our @Listeners
annotation.
InvokedTestMethodListenerExample:
package com.javarticles.testng; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeTest; import org.testng.annotations.Listeners; import org.testng.annotations.Test; @Listeners(InvokedTestMethodListener.class) public class InvokedTestMethodListenerExample { @BeforeTest public void beforeTest1() { System.out.println("beforeTest1"); } @BeforeTest public void beforeTest2() { System.out.println("beforeTest2"); } @Test public void t1() { System.out.println("t1"); } @Test public void t2() { System.out.println("t2"); } @Test public void t3() { System.out.println("t3"); } @Test public void t4() { System.out.println("t4"); } @AfterTest public void afterTest1() { System.out.println("afterTest1"); } @AfterTest public void afterTest2() { System.out.println("afterTest2"); } }
InvokedTestMethodListener:
package com.javarticles.testng; import org.testng.IInvokedMethod; import org.testng.IInvokedMethodListener; import org.testng.ITestResult; public class InvokedTestMethodListener implements IInvokedMethodListener { @Override public void beforeInvocation(IInvokedMethod method, ITestResult testResult) { if ("t2".equals(method.getTestMethod().getMethodName())) { System.out.println("Invoked just before t2()"); } } @Override public void afterInvocation(IInvokedMethod method, ITestResult testResult) { if ("t2".equals(method.getTestMethod().getMethodName())) { System.out.println("Invoked just after t2()"); } } }
Output:
[TestNG] Running: C:\Users\mokkara\AppData\Local\Temp\testng-eclipse-1301133492\testng-customsuite.xml beforeTest1 beforeTest2 t1 Invoked just before t2() t2 Invoked just after t2() t3 t4 afterTest1 afterTest2 PASSED: t1 PASSED: t2 PASSED: t3 PASSED: t4
For more examples on TestNG listeners, click here
Download source code “testng annotations”
In this article, I showed you examples of TestNG annotations. You can download the source code here: testNGAnnotations.zip