Agile Testing – A Must Do, Anyone Can Do!
cPrime’s 1st Hackathon
This past Friday, cPrime held our very first Hackathon. The goal: debug www.cprime.com.
I am sure you are curious, no, we did not bring on a QA team or developers; we were them for the day. Team members from our entire back office, recruiters, sales, finance, marketing and more, teamed up and learned to use Git and Ruby to write tests in Cucumber and run them with Selenium, with the guidance and patience of our CTO, Joel Brass. Source code was stored in Stash, pushed to Jenkins and Automated tests were run with Saucelabs . There was a lot of sweat, blood, tears, grunts and bottles of wine, but all in the name of good fun and better understanding our customers. Not only that, but by learn by doing, our team truly understood the importance of proper testing, but why the right tools, process and team matter!
That being said, below is the juicy info you actually want to know about testing, as noted by cPrime coach, Brian Dsouza.
Here are the reasons to focus on a proper functional testing approach for your agile teams.
Functional testing is more important than unit testing in an Agile context – deliver working software, satisfy the customer
- Unit testing, while valuable to ensure engineering quality, is costly.
- Unit testing typically tests much more than the current scope for a user
- Unit testing cannot identify defects found in functional testing.
- Functional testing cannot guarantee engineering quality but guarantees product functional quality from a users perspective
- From an end user’s perspective, functional testing is more important as, at a minimum, it ensures a working solution is delivered
Note: Unit testing is necessary to achieve engineering quality. This approach does not eliminate the need for the same
Agile teams have adopted automation, including BDD and ATDD, to overcome the challenges of manual functional testing. This was supposed to alleviate the challenges but it did not
- Traditional automation is focused on achieving a regression test suite – does not solve the need for in sprint automation testing
- ATDD, BDD/Gherkin, Cucumber move the work to developers – developers do not like to test and this burdens them further
- Use of scripting languages to automate ( selenium , QTP, UFT etc) is slow and requires the test writer to be skilled in using the technology
- The first round of testing is typically manual which is a waste of time, if automation is the intent
- Even if the automation tests get completed in the Sprint defects are identified after the developer checks in their code
- Developers resort to extensive unit testing as functional testing becomes available too late in the development cycle
For in-sprint functional testing to be valuable we need to adopt an Automated Functional Test First development approach
Automated functional tests must be available while a developer is coding (before they check in their code)
- Automated tests must be created rapidly: 10 – 15 minutes for small tests
- Tests are immediately automated – no manual testing
- Anyone with minimal training must be able to create a test ( QA, PO, Dev): no coding, scripting or script tool knowledge required
- Person writing the test must not have to deal with device issues. Tests must be able to work on a browser, phone or desktop
Developers check in their code only if unit AND functional tests pass
- Developer executes automated functional tests as part of their code testing
- Developer fixes defects identified by automated functional tests
- Developer checks in code AND also ‘checks in’ passing functional tests
Functional regression testing must occur all the time (CI)
- Functional tests become part of the CI run
- Any broken functionality becomes quickly obvious to all
I was introduced to Opkey a year ago while sourcing tools to support this approach. The client selected Opkey from a list of approximately 30 QA tools. I have successfully stood up multiple in-sprint automation test teams using this tool and hence my recommendation. Whereas I am unaware of the tool’s past capabilities/failing, I inquired from Crestech about their Syniverse relationship and they responded that the tool continues to be used effectively at Mach but is currently not being used at Syniverse, Would help to understand the reasons – tool issue or incorrect approach?
Please find attached a document that describes the basic set up and training for an Automated Functional Test First Development approach using Opkey.
One suggestion is to have Crestech demonstrate this by quickly standing up an automated test environment to test the common UI components being currently developed.
