Acceptance Testing in DevOps

19 %
81 %
Information about Acceptance Testing in DevOps

Published on March 9, 2014

Author: LaurentPY



To support efficiently ATDD or BDD at the speed and scale required in Agile and DevOps projects, your acceptance criteria need to:
- Be understandable by business experts, Product owners, developers (defining the business terminology)
- Facilitates test authoring as well as maintenance (based on a semantic and not just text)
- Can be transformed automatically into code for test execution

Laurent PY CEO, Smartesting @py_laurent Acceptance testing in DevOps

Our long and painful journey towards DevOps

Overview of our development process 2004/06  Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing) – Waterfall process – 1 release every 6 months – Few tests and no TDD – 1 month (x5 people) to do release testing before deployment – Poor quality impacting customers’ feedback and adoption 3

Tasting agility end of 2006  Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing) – First XP and then Scrum process – We do TDD – Continuous integration (code & unit test) – 1 customer release every 3 months (1 operation release every 3 weeks) – 1 man/month to do release testing before deployment – Good feedback from customers! 4

Now we are experiencing DevOps - 2012  Product: Zest, testing platform in the cloud – Still Scrum process – We do TDD and… – …Acceptance Testing Driven Development (ATDD), 100% automated – Acceptance tests are part of the CI process – We do several deployments a day (≈10) 5

What we’ve learnt  Shorter iterations (1 to 4 weeks) lead to massive test automation completed by exploratory testing  Acceptance tests become the specification  Testing starts earlier in the development process: Shift left! Req Management & Definition Test Planning Execution Defect management

What we’ve learnt  To achieve this level of speed (DevOps context), acceptance tests should be: – Understandable to both developers and business experts to strengthen alignment and enable the ‘shift left’ – Maintainable to manage efficiently changes in requirements and keep pace with continuous deployment – Automated to enable rapid execution and feed-back  ATDD with Business Domain Language and refactoring capabilities 7

ATDD & Refactoring My view as product owner & tester: Acceptance tests need to be continually reviewed and refactored just like code!!!

ATDD & Refactoring My view as product owner & tester: Acceptance tests need to be continually reviewed and refactored just like code!!! Martin Fowler

Acceptance testing driven development

Shift left Daily meeting Product Backlog Sprint Backlog Product at the end of the iteration Sprint 1 to 4 weeks Acceptance testing Shift left Acceptance testing Scrum team PO developers testers Scrum master

Acceptance Testing Driven Development (ATDD)  Acceptance test is a powerful artifact to improve communication  Test as the definition of ‘STOP’  Written prior to development by tester  Based on a business DSL (domain specific Language)  Confirmed with stakeholders  Mostly automated Test in natural language Test fixture Code

Acceptance Testing Driven Development (ATDD)  Benefits: – Improve communication and collaboration between project stakeholders – Shared understanding of what a successful implementation means – Better coverage of business expectations – Faster feed back  Challenges: – New methodology that requires rigor and discipline – Find the right balance between people/process/tool

Continuous acceptance testing

Zest: create acceptance tests with DSL  Key features: – – Suggestion to promote the reuse of Action Word and avoid duplication – Refactoring: when an Action Word is modified, impacts are performed automatically across the tests – Optimization: inspection features enable to continuously optimize the acceptance tests –  Define progressively your Action Words and your business domain language for test authoring Create scripts and accelerate test automation (Ruby, Java, XML…) Integrations with: Watir, Appium… 15

Collaboration through acceptance tests & DSL Tester Create acceptance Tests Product Owner Validate acceptance tests Developer Automate acceptance tests • DSL • Refactoring capabilities 16

Define new business entities… Define progressively your business terminology with Action Words. Enable collaboration based on acceptance tests.

…or define business entities right from the tests Promote test steps into Action Words (what developers call extract function). This is refactoring!

Evils of duplication

A fundamental principle

Reuse, reuse and reuse Action Words! Suggestions Create and maintain consistent scenarios for your project

Analyse and optimize continuously your tests When duplications are identified, refactoring options are suggested!

Add, remove, modify Action Words and Scenarios Add parameters to Action Word Propagate automatically to the scenarios Test refactoring enables to perform automatically impact analysis and test maintenance tasks

Generate scripts Use of Action Words significantly decrease the cost of automation and accelerate the overall testing cycle

So your acceptance tests are… Understandable Definition of business domain tests enable better alignment of the team • DSL • Refactoring capabilities Can be automated Good design based on Action Words streamlines the test automation phase Maintainable Refactoring and optimization features dramatically accelerate maintenance

And remember Acceptance tests need to be continually reviewed and refactored just like code!!!

Give a try Laurent PY CEO, Smartesting @py_laurent

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Dave Farley on Agile, DevOps, Acceptance Testing and More

4. Both continuous delivery and DevOps put very strong emphasis on non-functional requirements, let’s say, but in many organizations business ...
Read more

Testing In Devops | LinkedIn

View 423 Testing In Devops posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn.
Read more

Importance of Testing From Developer's Point of View ...

Importance of Testing From Developer's Point ... The DevOps Zone is brought to you by DZone in partnership with Hewlett Packard ... Acceptance Testing.
Read more

7: Testing in the Software Lifecycle

Testing for Continuous Delivery with Visual Studio 2012 7: Testing in the Software Lifecycle
Read more

Better Automated Acceptance Tests With Serenity Screenplay ...

Better Automated Acceptance Tests With Serenity Screenplay ... (Serenity automated acceptance testing project using ... The DevOps Zone is brought to you ...
Read more

DevOps Testing Services | Software Testing Services ...

DevOps integrates development & operations in the same cycle with a large emphasis on automation of build, deployment and testing, devOps automation, agile ...
Read more

Automated acceptance testing - Practical DevOps [Book]

Automated acceptance testing Automated acceptance testing is a method of ensuring that your testing is valid from the end user's point of view. Cucumber is ...
Read more

Understanding DevOps – Part 4: Continuous Testing and ...

It needs to have the necessary scripts and test data for acceptance and ... Continuous Testing and ... Testing Mindset in DevOps ...
Read more

Automated Security Testing in a Continuous Delivery Pipeline

Automated unit, integration and acceptance tests are essential quality controls in running a reliable continuous integration or continuous delivery pipeline.
Read more