Test Driven Development Sample – Template Engine

0

What is Test Driven Development?

Driving development with automated tests is called Test Driven Development. The main difference between writing unit tests for an existing code and test-driven-development is that, in TDD, you will use unit tests to drive your design. You will write a test case first and then the code to make it pass. So how do you decide the use case? Well…pick up a use case which is simple enough to test. Most of the times, the secret is to simplify the use case into smaller use cases.
  1. Red—write a little test that doesn’t work, perhaps doesn’t even compile at first
  2. Green—make the test work quickly, committing whatever sins necessary in the process
  3. Re-factor—eliminate all the duplication created in just getting the test to work

Test Driven Development Sample Problem

Template Engine

In the current post, I demonstrate one of the TDD problems ‘Template Engine’.

Template Engine

Template engine helps us to transform template strings, “Hello {$name}” into “instanced” strings.
To do that a variable->value mapping must be provided.
For example, if name=”Cenk” and the template string is “Hello {$name}” the result would be “Hello Cenk”.

– Should evaluate template single variable expression:

mapOfVariables.put(“name”,”Cenk”);
templateEngine.evaluate(“Hello {$name}”, mapOfVariables)
=> should evaluate to “Hello Cenk”

– Should evaluate template with multiple expressions:

mapOfVariables.put(“firstName”,”Cenk”);
mapOfVariables.put(“lastName”,”Civici”);
templateEngine.evaluate(“Hello {$firstName} ${lastName}”, mapOfVariables);
=> should evaluate to “Hello Cenk Civici”

– Should give error if template variable does not exist in the map:

map empty
templateEngine.evaluate(“Hello {$firstName} “, mapOfVariables);
=> should throw missingvalueexception

– Should evaluate complex cases:

mapOfVariables.put(“name”,”Cenk”);
templateEngine.evaluate(“Hello ${$name}}”, mapOfVariables);
=> should evaluate to “Hello ${Cenk}”

Approach

The broader approach is to think in terms of APIs but only after decoupling ourselves from the subject in context.
First step would be to think of the simplest scenario possible.
I came up with the following test case.

Test case: Parse a simple text. It should simply return the text as it is.

    public void testParseSimpleText() {
        assertEquals("Test", TemplateEngine.parse("Test"));
    }

TemplateEngine class doesn’t exist so it has to be created as well as the parse method. When the method is not implemented, the test will fail.
Implement the method to pass the test.

    public static String parse(String test) {
        return test;
    }

 Test case: Parse a string which contains simple text and a parameter. It should return us two tokens. First token should be the text and the next one the parameter name.
    public void testParseParameterText() {
        String[] tokens = TemplateEngine.parse("Test me {$name}");
        assertEquals(2, tokens.length);
        assertEquals("Test me ", tokens[0]);
        assertEquals("name", tokens[1]);
    }

There will be a compilation error as parse(string) returns String and not String[]. We need to fix the compilation error and implement just enough code to pass the test.

I re-factored parse method to return String[].

    public static String[] parse(String test) {
        String[] tokens = new String[2];
        int paramIndex = test.indexOf("{$");
        tokens[0]= test.substring(0, paramIndex);
        tokens[1]= test.substring(paramIndex + 2);
        return tokens;
    }

When I ran the test suite, both test cases failed.

java.lang.StringIndexOutOfBoundsException: String index out of range: -1
 at java.lang.String.substring(String.java:1937)
 at tdd.TemplateEngine.parse(TemplateEngine.java:14)
 at TemplateEngineTests.testParseSimpleText(TemplateEngineTests.java:16)

java.lang.StringIndexOutOfBoundsException: String index out of range: -1
 at java.lang.String.substring(String.java:1937)
 at tdd.TemplateEngine.parse(TemplateEngine.java:14)
 at TemplateEngineTests.testParseParameterText(TemplateEngineTests.java:20)

The tests are in red and it is important that we get back to green as fast as possible.
I realized the first one was failing as the string used in testing is a simple text and has no “{$”.

int paramIndex = test.indexOf("{$");
tokens[0]= test.substring(0, paramIndex);

I modified the parse implementation.

    public static String[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new String[]{test};
        }
        String[] tokens = new String[2];
        tokens[0] = test.substring(0, paramIndex);
        tokens[1]= test.substring(paramIndex + 2);
        return tokens;
    }

Now the first unit test passes but second one still fails.

junit.framework.ComparisonFailure: expected: but was:
 at TemplateEngineTests.testParseParameterText(TemplateEngineTests.java:25)

The issue, seems to me, is with substring toIndex as it included an extra char ‘}’. I fixed that by adding toIndex=test.indexOf(‘}’).

    public static String[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new String[]{test};
        }
        String[] tokens = new String[2];
        tokens[0] = test.substring(0, paramIndex);
        tokens[1]= test.substring(paramIndex + 2, test.indexOf('}'));
        return tokens;
    }

Now the tests are passing and we are ready for the next test case.

Test case: This is just an extension of the second test case. We add an extra space after the parameter.

    public void testParseMultiParameterText() {
        String[] tokens = TemplateEngine.parse("Test me {$name} ");
        assertEquals(3, tokens.length);
    }

The test fails at token length assertion, clearly the hard-coded array length has to be removed.

 

junit.framework.AssertionFailedError: expected: but was:
 at TemplateEngineTests.testParseMultiParameterText(TemplateEngineTests.java:30)

For the test string, “Test me {$name} “, we expect 3 tokens – “Test me “, “name” and ” “.
When we traverse through the characters, once we hit the ‘{‘ and ‘$’, the string parsed before becomes the first token. We hit ‘}’ when we are done parsing the parameter, so the parameter becomes the next token. The string after ‘}’ becomes the third token.

The tokens are added as we parse.

    public static String[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new String[]{test};
        }
        List tokens = new ArrayList();
        char[] chars = test.toCharArray();
        int startIndex = 0;
        for (int i = 0; i < chars.length; i++) {
            if (chars[i]== '{' && chars[i + 1]== '$' ) {
                tokens.add(test.substring(startIndex, i));
                startIndex = i;
            } else if (chars[i]== '}') {
                tokens.add(test.substring(startIndex + 2, i));
                startIndex = i + 1;
            } else if (i == chars.length - 1) {
                tokens.add(test.substring(startIndex));
            }
        }
        return tokens.toArray(new String[tokens.size()]);
    }

I ran the test suite and all tests passed.

Test Case: Further improved the test case by adding more components and asserts on tokens

    public void testParseMultiParameterText() {
        String[] tokens = TemplateEngine.parse("Test me {$name} ");
        assertEquals(3, tokens.length);
        tokens = TemplateEngine.parse("Test me {$firstname} {$lastname}!");
        assertEquals(5, tokens.length);
        assertEquals("Test me ", tokens[0]);
        assertEquals("firstname", tokens[1]);
        assertEquals(" ", tokens[2]);
        assertEquals("lastname", tokens[3]);
        assertEquals("!", tokens[4]);
    }

I ran the test suite and all tests passed.

When I add the below statement to the test case, the test fails.

tokens = TemplateEngine.parse("{$name}");

junit.framework.AssertionFailedError: expected: but was:
 at tdd1.TemplateTests.testParseMultiParameterText(TemplateTests.java:31)

When we hit ‘{$’ we add the already parsed string to the tokens. This we need to do only when we are sure that we have parsed at least one character before we hist ‘{‘.

    public static String[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new String[]{test};
        }
        List tokens = new ArrayList();
        char[] chars = test.toCharArray();
        int startIndex = 0;
        for (int i = 0; i < chars.length; i++) {             if (chars[i]== '{' && chars[i + 1]== '$' ) {                 ==>if (i > 0) {
                    tokens.add(test.substring(startIndex, i));
                }
                startIndex = i;
            } else if (chars[i]== '}') {
                tokens.add(test.substring(startIndex + 2, i));
                startIndex = i + 1;
            } else if (i == chars.length - 1) {
                tokens.add(test.substring(startIndex));
            }
        }
        return tokens.toArray(new String[tokens.size()]);
    }



Test case: Now my next goal would be to know from the token whether it is a parameter or a simple text.

    public void testParseParameter() {
        Token[] tokens = TemplateEngine.parse("{$name}");
        assertEquals(1, tokens.length);
        assertEquals("name", tokens[0].getValue());
        assertEquals("{$name}", tokens[0].toString());
        assertTrue(tokens[0].isParameter());
    }

Since Token doesn’t exist, we create the new class and modify the implementation.

    public static Token[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new Token[]{new Token(test)};
        }
        List tokens = new ArrayList();
        char[] chars = test.toCharArray();
        int startIndex = 0;
        for (int i = 0; i < chars.length; i++) {
          if (chars[i]== '{' && chars[i + 1]== '$' ) {
                if (i > 0) {
                    tokens.add(new Token(test.substring(startIndex, i)));
                }
                startIndex = i;
            } else if (chars[i]== '}') {
                tokens.add(new Token(test.substring(startIndex + 2, i), true));
                startIndex = i + 1;
            } else if (i == chars.length - 1) {
                tokens.add(new Token(test.substring(startIndex)));
            }
        }
        return tokens.toArray(new Token[tokens.size()]);
    }

Since parse() now returns Token[], I had to re-factor the old tests. Token class is a simple bean which has value and a boolean to know whether the token is a parameter.

public class Token {
    private String value;
    private boolean isParam;
    public Token(String v) {
        value = v;
    }
    public Token(String v, boolean p) {
        this(v);
        isParam = p;
    }
    public String getValue() {
        return value;
    }
    public boolean isParameter() {
        return isParam;
    }
    public String toString() {
        return isParam ? "{$" + value + "}" : value;
    }
}



Test case: Let’s now try to evaluate the parsed string for given a set of values

    public void testEvaluateParsedString() {
        Token[] tokens = TemplateEngine.parse("{$name}");
        Map paramValues = new HashMap();
        paramValues.put("name", "Ram");
        assertEquals("Ram", TemplateEngine.evaluate(tokens, paramValues));
    }

evaluate() doesn’t exist so we will create it. It should get the parameter value from paramValues.

    public static String evaluate(Token[] tokens, Map paramValues) {
        StringBuilder sb = new StringBuilder();
        for (Token token :  tokens) {
            sb.append(token.getValue(paramValues));
        }
        return sb.toString();
    }

If the value is not found, I append it as parameter. One can throw an exception instead.

    public void testEvaluateParamValueNotInMap() {
        Token[] tokens = TemplateEngine.parse("{$name}");
        Map paramValues = new HashMap();
        assertEquals("{$name}", TemplateEngine.evaluate(tokens, paramValues));
    }

Token.getValue() should return value if token is a simple text else should fetch the param value from the given param values.

    public String getValue(Map paramValues) {
        String value = paramValues.get(getValue());
        return value == null ? toString() : value;
    }

I ran the test suite and all the test cases pass.



Test case: Let’s now try to evaluate embedded parameters. A normal parameter {$name} has one level, a parameter within another parameter has two levels {${$name}}.

    public void testEmbeddedTokensLevel() {
        Token[] tokens = TemplateEngine.parse("{${${$name}}}");
        assertEquals(1, tokens.length);
        assertTrue(tokens[0].isParameter());
        assertEquals(3, tokens[0].getLevel());
        assertEquals("name", tokens[0].getValue());
    }

To pass the above unit test, I add level attribute to Token class and improve parse() method to increment the levels.

    public static Token[] parse(String test) {
        int paramIndex = test.indexOf("{$");
        if (paramIndex == -1) {
            return new Token[]{new Token(test)};
        }
        List tokens = new ArrayList();
        char[] chars = test.toCharArray();
        int startIndex = 0;
        int level = 0;
        for (int i = 0; i < chars.length; i++) {             if (chars[i]== '{' && chars[i + 1]== '$' ) {                 if (i > 0 && level == 0) {
                    tokens.add(new Token(test.substring(startIndex, i)));
                }
                if (level == 0) {
                    startIndex = i;
                }
                level++;
            } else if (chars[i]== '}') {
                tokens.add(new Token(test.substring(startIndex + 2 * level, i), true, level));
                startIndex = i + level;
                i = i + level -1;
                level = 0;
            } else if (i == chars.length - 1) {
                tokens.add(new Token(test.substring(startIndex)));
            }
        }
        return tokens.toArray(new Token[tokens.size()]);
    }



Test case: Evaluate an embedded parameter.

    public void testEmbeddedTokensLevel() {
        Token[] tokens = TemplateEngine.parse("{${${$name}}}");
        assertEquals(1, tokens.length);
        assertTrue(tokens[0].isParameter());
        assertEquals(3, tokens[0].getLevel());
        assertEquals("name", tokens[0].getValue());

        Map values = new HashMap();
        values.put("name", "company");
        values.put("company", "navis");
        values.put("navis", "Navis India");
        assertEquals("Navis India", TemplateEngine.evaluate(tokens, values));

        tokens = TemplateEngine.parse("Hello {${${$name}}}");
        assertEquals("Hello Navis India", TemplateEngine.evaluate(tokens, values));
    }

The test will fail as Token.getValue() doesn’t consider level while evaluating the parameter. To pass the test I introduce level to this method.

    public String getValue(Map paramValues) {
        String value = getValue();
        for (int i = 0; i < paramLevel; i++) {
            value = paramValues.get(value);
            if (value == null) {
                return toString();
            }
        }
        return value;
    }

We are done with all the unit tests.

More re-factoring

  1. If we notice the evaluation is done in two steps. The first one is parsing and the next step is evaluation of the tokens. The issue here is the user has to store the parsed tokens somewhere since later it has to be passed to the evaluation method. I guess this is an unnecessary step as the user is forced to remember this extra step. To fix this we need to assign the parsed tokens to an instance variable and reuse it in evaluation. This is possible when TemplateEngine becomes a pure Object, right now it acts as a utility class. It would be appropriate if we rename the class to indicate that it contains the parsed tokens and it can evaluate the parsed tokens into a string based on the input value map. I rename the class to ParsedString.
  2. Token class has isParam variable to indicate whether it is string token or parameter token. If a token is of parameter type, we also need to know the levels. The instance variables isParam and level, doesn’t make sense if the token is a string based one. Instead of using one generic class Token, we can have SimpleToken and ParamToken to distinguish.

Final Solution

 Class Diagram
TDD Samples - Template Engine
TDD Samples – Template Engine

 

Unit tests

TemplateTests:

import junit.framework.TestCase;

import java.util.HashMap;
import java.util.Map;


public class TemplateTests extends TestCase {

    public void testParseSimpleText() {
        Token[] tokens = new ParsedString("Test").getTokens();
        assertEquals(1, tokens.length);
        assertEquals("Test", tokens[0].getValue());
    }

    public void testParseParameterText() {
        Token[] tokens = new ParsedString("Test me {$name}").getTokens();
        assertEquals(2, tokens.length);
        assertEquals("Test me ", tokens[0].getValue());
        assertEquals("name", tokens[1].getValue());
    }

    public void testParseMultiParameterText() {
        Token[] tokens = new ParsedString("Test me {$name} ").getTokens();
        assertEquals(3, tokens.length);
        tokens = new ParsedString("{$name}").getTokens();
        assertEquals(1, tokens.length);
        tokens = new ParsedString("Test me {$firstname} {$lastname}!").getTokens();
        assertEquals(5, tokens.length);
        assertEquals("Test me ", tokens[0].getValue());
        assertEquals("firstname", tokens[1].getValue());
        assertEquals(" ", tokens[2].getValue());
        assertEquals("lastname", tokens[3].getValue());
        assertEquals("!", tokens[4].getValue());
    }

    public void testParseParameter() {
        Token[] tokens = new ParsedString("{$name}").getTokens();
        assertEquals(1, tokens.length);
        assertEquals("name", tokens[0].getValue());
        assertEquals("{$name}", tokens[0].toString());
        assertTrue(tokens[0].isParameter());
    }

    public void testEvaluateParsedString() {
        ParsedString parsedString = new ParsedString("{$name}");
        Map paramValues = new HashMap();
        paramValues.put("name", "Ram");
        assertEquals("Ram", parsedString.evaluate(paramValues));
    }

    public void testEvaluateParamValueNotInMap() {
        ParsedString parsedString = new ParsedString("{$name}");
        Map paramValues = new HashMap();
        assertEquals("{$name}", parsedString.evaluate(paramValues));
    }

    public void testEmbeddedTokensLevel() {
        Token[] tokens = new ParsedString("{${${$name}}}").getTokens();
        assertEquals(1, tokens.length);
        assertTrue(tokens[0].isParameter());
        assertEquals(3, ((ParamToken)tokens[0]).getLevel());
        assertEquals("name", tokens[0].getValue());

        Map values = new HashMap();
        values.put("name", "company");
        values.put("company", "navis");
        values.put("navis", "Navis India");
        assertEquals("Navis India", new ParsedString("{${${$name}}}").evaluate(values));

        ParsedString parsedString = new ParsedString("Hello {${${$name}}}");
        assertEquals("Hello Navis India", parsedString.evaluate(values));
    }

    public void testSimpleExpression() {
        Map values = new HashMap();
        values.put("firstName", "Cenk");
        assertEquals("Hello Cenk", new ParsedString("Hello {$firstName}").evaluate(values));
    }

    public void testEmbeddedExpressionWithStringToken() {
        Map values = new HashMap();
        values.put("firstName", "Cenk");
        assertEquals("Hello {$Cenk}", new ParsedString("Hello {${$firstName}}").evaluate(values));
    }

    public void testMultipleExpression() {
        Map values = new HashMap();
        values.put("ev", "ev");
        values.put("er", "ery");
        ParsedString parsedString = new ParsedString("Hello {$ev}{$er} one");
        assertEquals("Hello every one", parsedString.evaluate(values));
    }

    public void testExpressionWithoutParamValue() {
        Map values = new HashMap();
        values.put("fa", "Cenk");
        values.put("te", "test");
        ParsedString parsedString = new ParsedString("Hello{}$fa}{$te}h");
        assertEquals("Hello{}$fa}testh", parsedString.evaluate(values));
    }

    public void testExpressionWith$() {
        Map values = new HashMap();
        values.put("factName", "Cenk");
        ParsedString parsedString = new ParsedString("Hello ${{$factName}}");
        assertEquals("Hello ${Cenk}", parsedString.evaluate(values));
    }

    public void testExpressionWithParamLikePattern() {
        Map values = new HashMap();
        values.put("factName", "Cenk");
        ParsedString parsedString = new ParsedString("Hello $}{$factName");
        assertEquals("Hello $}{$factName", parsedString.evaluate(values));
    }

    public void testMultiExpressionWithStringToken() {
        Map values = new HashMap();
        values.put("factName", "Cenk");
        ParsedString parsedString = new ParsedString("{$factName} Hello {$factName}");
        assertEquals("Cenk Hello Cenk", parsedString.evaluate(values));
    }

    public void testExpressionWithoutParams() {
        Map values = new HashMap();
        values.put("fa", "Cenk");
        values.put("te", "test");
        ParsedString parsedString = new ParsedString("TEST");
        assertEquals("TEST", parsedString.evaluate(values));
    }

    public void testExpressionWithEmptyOrNullMap() {
        ParsedString parsedString = new ParsedString("${fe}");
        assertEquals("${fe}", parsedString.evaluate(null));
    }
}

Source Code

ParsedString:

import java.util.ArrayList;
import java.util.List;
import java.util.Map;


public class ParsedString {
    private final Token[] _tokens;

    public ParsedString(String inTemplate) {
        _tokens = parse(inTemplate);
    }

    private Token[] parse(String inTemplate) {
        int paramIndex = inTemplate.indexOf("{$");
        if (paramIndex == -1) {
            return new Token[]{new Token(inTemplate)};
        }
        List tokens = new ArrayList();
        char[] chars = inTemplate.toCharArray();
        int startIndex = 0;
        int level = 0;
        for (int i = 0; i < chars.length; i++) {
            if (chars[i]== '{' && chars[i + 1]== '$' ) {
                if (i > 0 && level == 0) {
                    tokens.add(new Token(inTemplate.substring(startIndex, i)));
                }
                if (level == 0) {
                    startIndex = i;
                }
                level++;
            } else if (chars[i]== '}' && level >= 1) {
                tokens.add(new ParamToken(inTemplate.substring(startIndex + 2 * level, i), level));
                startIndex = i + level;
                i = i + level -1;
                level = 0;
            } else if (i == chars.length - 1) {
                tokens.add(new Token(inTemplate.substring(startIndex)));
            }
        }
        return tokens.toArray(new Token[tokens.size()]);
    }

    public String evaluate(Map paramValues) {
        StringBuilder sb = new StringBuilder();
        for (Token token :  _tokens) {
            sb.append(token.getValue(paramValues));
        }
        return sb.toString();
    }

    public Token[] getTokens() {
        return _tokens;
    }

}

Token:

import java.util.Map;

public class Token {
    private String value;

    public Token(String v) {
        value = v;
    }

    public String getValue() {
        return value;
    }

    public String toString() {
        return value;
    }

    public String getValue(Map paramValues) {
        return getValue();
    }

    public boolean isParameter() {
        return false;
    }
}

ParamToken:

import java.util.Map;

public class ParamToken extends Token {
    private int paramLevel;
    public ParamToken(String v, int level) {
        super(v);
        paramLevel = level;
    }

    public String toString() {
        return "{$" + super.getValue() + "}";
    }

    public String getValue(Map paramValues) {
        String value = super.getValue();
        for (int i = 0; i < paramLevel; i++) {
            String v = paramValues.get(value);
            if (v == null) {
                return "{$" + value + "}";
            } else {
                value = v;
            }
        }
        return value;
    }

    public int getLevel() {
        return paramLevel;
    }

    public boolean isParameter() {
        return true;
    }
}
Share.

Leave A Reply