KBDD

Kotlin BDD Framework based on Corounit test engine.

Framework guiding examples

Example can be found at jfix-kbdd-example module. Allure test report can be used as a documentation for KBDD:
allure-report/index.html#behaviors

Suspending scenarios

KBDD based on suspendable methods.
This allows for big amount of test suites to start and work simultaneously and dramatically reduces time required for behave tests to complete. Test suite of thousands test can take time no more that the longest test within the suite.

In order to achieve good speed, test designer should take into consideration next practices:

All test and step methods are suspendable, do not block current thread - suspend it.
Use dedicated thread pools for blocking operations like network calls.
If you sent command to the tested system and waiting fixed time for results, do active polling. Instead of:

MySystem.`Send command to start heavy process`()
delay(15_000)
MySystem.`Get result of process`()
bodyJson()["status"].isEquals("activated")

Use active polling:

MySystem.`Send command to start heavy process`()
repeatUntilSuccess{
    MySystem.`Get result of process`()
    bodyJson()["status"].isEquals("activated")
}

HTTP client and assertions

KBDD provides basic functionality of HTTP client and response assertions. Detailed examples can be found here:
allure-report/index.html#behaviors

Real time Allure report update

Gradle Allure Plugin allows to start a daemon, that will monitor allure-results. Daemon will rebuild Allure html report after each test run. This option is handy when you are writing new test and want to immediately see updates of the Allure report in real time.

gradle allureReport -t

How to integrate in to your project

It is often useful to run KBDD tests as separate step in your build pipeline. This allows not to run integration kbdd scenarios and unit test within same gradle test task.

my-project
 ┕ my-a-module
 ┕ my-b-module
 ┕ my-c-module
 ┕ kbdd-test
   ┕ build.kts

Disable test task in kbdd-test module and rename it to kbdd

tasks.test {
    onlyIf { false }
}

tasks.register("kbdd", org.gradle.api.tasks.testing.Test::class) {
    description = "Start tests in KBDD module"
    dependsOn( "<setup mock servers and testing environment>" )
    outputs.upToDateWhen { false }
}

Now you can use use * gradle build to build and test your project as usual without involving kbb-test module * gradle kbdd to start kbdd tests in your pipeline

To enable junit tag support:

tasks {
    withType<Test> {
        useJUnitPlatform() {
            val includeTags = "includeTags"
            val excludeTags = "excludeTags"

            if (project.hasProperty(includeTags)) {
                includeTags(project.properties[includeTags] as String)
            }

            if (project.hasProperty(excludeTags)) {
                includeTags(project.properties[excludeTags] as String)
            }
        }
    }
}

Example: gradle kbdd -PincludeTags={tag1,tag2,tag3} gradle kbdd -PexcludeTags=tag4

Best practices

How to manage test data

Let’s consider a development process where you test application in dedicated environments called stands. The local stand is environment on your laptop. You launch docker-compose, setups database images required for your application, start application and run test suite. Any time you can drop database and start all over again wtih clean database that does not contain results of previously running tests. The dynamic stand is a similar environment that is created automatically by your build servers. Dynamic stands created with clean databases and destroyed at the end of a build process. The permanent stands are QA stand, Stage or Pre-Production stands that lives forever. If you run your automated tests over them, the results of the tests will stay in stands database and will. Also permanent stands used to test integration with different external to your team services. So this permanent stands contain data created with automated tests and data manually entered in order to manually test integration scenarios with external services. Data within permanent stands should be treated carefully since it is very easy to make stand database to become a big ball of mud, where you see tons of unreadable and understandable data and does not know can you change something and why particular entities configured in this way.
Next several principles can be very handy in resolving this problem.

Provide clean explanatory names and description to entities

account: {
  name: "test1"
  amount: 0
}

account: {
  name: "Auto TestCase-4534 User debet account without money which does not accrue interest on the balance"
  amount: 0
}

Entity names or description should contain prefix, that allows easily separate data that was created by automated tests, data that was entered manually and data that is created and used by application itself. In given example team decided to use three prefixes:
- Auto for data that is created by automated tests
- Int for manually created data for integration tests with other teams
- Manual for manually created data that is used by manually perfomed test cases.

accounts :[
{name: "Auto TestCase-4233 User debet..."},
{name: "Auto TestCase-3243 User debet..."},
{name: "Int User debet for SQX service used for bonus program (TestCase-3249,TestCase-3255)")
]

Do not share test data among different automated tests. Suppose that we are testing payment system. In order to process payment request system requires Contractor and Contract entities to be configured appropriately. It is bad idea to be lazy and simply reuse data that was configured by another test written before you. Test by itself serves as a documentation, so if application allows to run different payment scenarios based on different contract and contractor configuration - different tests should use different contract and contractors.

// DO NOT DO THAT
// Contractors.AL_BANK, Contracts.BAR_K is used by other tests
// That leads to data coupling
@Test
suspend fun `Success registrly re-upload from Bank to ABX after failed upload due to invalid config`() {
    ...
    abx.`Prepare contractor`(Contractors.AL_BANK)
    abx.`Prepare contact`(Contracts.BAR_K)
    ...
}

// Test prepare it's own data used only in one place.
// No coupling with other tests.
@Test
suspend fun `Success registrly re-upload from Bank to ABX after failed upload due to invalid config`() {
    ...
    abx.`Prepare contractor`(Contract22or(name="Auto T3234 registry uploading contractor", ...)
    abx.`Prepare contact`(Contracts(nane="Auto T3234 registry uploading contract for single product merchant",...)))
    ...
}

Add reference to documentation in @Description

Scenario description have a reference to project wiki or documentation with detailed description of tested cases.

@Description("""
    User makes a simple purchase in the site
    http://documentation.acme.com/purchase/details
    """)
class PurchaseTest(){
    //...
}

String steps describe hi level scenario

Use string steps to describe business process in clear way that all members of your team, including non-tech people, easily understand. This will lead to a clear readable Allure report.

suspend fun `make a purchase in the shop`(){
    "Ensure that user account with amount of 100 exist"{
        //...
    }
    "User adds item of price 45 into the basket"{
        //...
    }
    "User creates a purchase order"{
        //...
    }
    "User select shipment condition"{
        //...
    }
    "User agrees for money withdraw from use account"{
        //...
    }
    "User account balance became 55"{
        //...
    }
}

Each test prepare it’s own data in self-recoverable way

Keep in mind that all tests are running in parallel. Our task is to make tests independent on each other. Best way to do that is through tested system configuration. E.g. we can use unique account id for each test case. Since test can broke on eny step we should take into consideration that should be able to restart the test. So our test should be able to reset test conditions and system state that was corrupted due to previous failed test run.

suspend fun `make a purchase in the shop`(){
    val userAccount = 9473234983L
    "Ensure that user account with amount of 100 exist"{
        //create account 9473234983L with amount 100 if such account does not exist yet
        //if account exist, then set account amount to 100
        //...
    }
    "User adds item of price 45 into the basket"{
        //...
    }
    //...
}