Published on March 9, 2014
2014 Workshop process improvement Testing BI/ DWH AGENDA Personal introduction & expectations 15m Niels Bor and Marcus Drost (workshop leaders) Participants workshop Contents Part 1 Short presentation: Waterfall versus Agile/ Scrum (Marcus Drost) 15m Part 2 Problem awareness game (insight) test problems BI/ DWH systems (Niels Bor) 20m Part 3 Short presentation: More about testing (Marcus Drost) 15m (then break 10m) Part 4 Example from practice: Agile regression test tool DREAM (Marcus Drost) 30m Part 5 Problem awareness game (from cause to action) (Niels Bor) 20m Finish 10m Highlights workshop (Niels Bor and Marcus Drost) as well as feedback participants (15m extra time) V2.3
Introduction and expectations Goal workshop “Getting insight in the problems of waterfall and agile test processes, and identifying solutions. Especially for data intensive environments like Business Intelligence, Data Warehousing and Database Applications.”
Part 1 Short presentation: Waterfall versus Agile/ Scrum (Marcus Drost)
The Agile Manifesto introduced the term in 2001. Since then, the Agile Movement, with all its values, principles, methods and cultures, has significantly changed the landscape of the modern software engineering. The manifesto's values: 1.Individuals and interactions over Processes and tools 2.Working software over Comprehensive documentation 3.Customer collaboration over Contract negotiation 4.Responding to change over Following a plan Source: Wikipedia
The Agile Manifesto is based on twelve principles: 1.Customer satisfaction by rapid delivery of useful software 2.Welcome changing requirements, even late in development 3.Working software is delivered frequently (weeks rather than months) 4.Working software is the principal measure of progress 5.Sustainable development, able to maintain a constant pace 6.Close, daily cooperation between business people and developers 7.Face-to-face conversation is the best form of communication (co-location) 8.Projects are built around motivated individuals, who should be trusted 9.Continuous attention to technical excellence and good design 10.Simplicity—the art of maximizing the amount of work not done—is essential 11.Self-organizing teams 12.Regular adaptation to changing circumstances Source: Wikipedia
Iterative vs. Agile First of all, Agile is iterative already but it is way more than just iterative. Here are a number of differences between Agile and “just” Iterative development: Mini-waterfall is still waterfall Iterative is still waterfall, just on a smaller scale. A series of mini-waterfalls is certainly better and less risky than one big waterfall but mini-waterfall still is fundamentally waterfall and comes with all its known problems such as difficulty to adapt to change (“Nice idea but sorry, the requirements have been signed off months ago”), cascading delays (“Oops, we need to shorten the testing phase”) and low quality (“We don’t have time to fix those bugs. We’ll fix them in a later phase/iteration”). Source: Sandy Mamoli
Comparing Agile and Traditional approaches Source: Scott Ambler
A look back at waterfall
Currently in AGILE/ SCRUM
Change frequency LOW HIGH
Regression test frequency LOW HIGH
Regression in AGILE/ SCRUME/ SCRUM
AGILE/ SCRUM test bottleneck
Frequent application changing
Part 2 Problem awareness game (insight) based on a BI/DWH system (Niels Bor)
Problem awareness game (insight) The participants of the workshop discuss potential problems which they do encounter in practice during testing of data intensive systems. They identify testing problems and describe where and when the problems occur regularly. As auxiliary means, the participants could use the diagram of the data warehouse system and the V-model on the next page. Describe the problems in detail in order to come to know the root cause.
Typical problems (examples) 1. Data from production and test systems is functional not stable. 2. Test data delivered automatically or manually is not available in time. 3. Setting up test environments and preparation of tests is very time consuming. 4. Run time of tests is often very long due to bad performance. 5. Test cases are not clear and without connection to physical test cases; i.e. test cases are made ad hoc during the test phase; test coverage is random 6. No description of output of test cases; high workload during the test phase 7. Often double work and overlap between UT, FAT and GAT. 8. Test cases are not re-usable; no connection between functionality (use cases) and functional and physical test cases. 9. No standard process for adaption and extension of test cases; often ad hoc 10. The pressure on testers during test phases is very high.
Part 3 Short presentation: More about testing and agile testing (Marcus Drost)
Testing levels (for information) Tests are frequently grouped by the level of specificity of the test. Unit Test (UT): A method by which individual units of source code is tested to determine if they are fit for use. Integration Test (IT): Integration testing is any type of software testing that seeks to verify the interfaces between components against a software design. System Test (ST): System testing, or end-to-end testing, tests a completely integrated system to verify that it meets its requirements. Regression Test (RT): Regression testing focuses on finding defects after a major code change has occurred. Functional Acceptance Test (FAT): Functional testing refers to activities that verify a specific action or function of the code. User Acceptance Test (UAT): Acceptance testing performed by the customer, often in their lab environment on their own hardware. Operational Acceptance Test (OAT): Also known as operational readiness testing, this refers to the checking done to a system to ensure that processes and procedures are in place to allow the system to be used and maintained. Source: Wikipedia
Methods for regression testing Random Sample: Controlling just a few results in order to generalize them Advantage: Fast, and no automation necessary Disadvantage: Inexact and incomplete Proof Total: By totalizing some figures, the total result will be forecasted. Advantage: Can be automated relatively fast Disadvantage: Incomplete; root cause of differences are difficult to explain; strong dependency on application logic Data Model: As the output data is the fingerprint of applications' functionality, a data model approach can be used to find every differences. Advantage: complete coverage of 100%; independent of application logic; fast analysis of root cause Disadvantage: Test data management is necessary Source: Marcus Drost
Spoken about regression...
More details… U.K. financial regulators are investigating an IT failure at British bank RBS (The Royal Bank of Scotland) that left millions of users without access to their accounts last summer. The problem, which stemmed from an error during a software update, has already cost RBS £175 million ($270 million) in interest waivers, reversed charges and compensation payments, InformationWeek reported. Additional penalties could be forthcoming based on the determination made by the Financial Conduct Authority (FCA), a new regulatory agency in charge of policing the U.K. financial services market. The error last summer occurred when RBS, which owns British chains NatWest and Ulster Bank, performed an update to the CA-7 software that controls the bank’s batch processing systems. The program is used to automate large sequences of work involved in handling daily transactions from ATMs, bank-to-bank payments and elsewhere. Source: http://www.klocwork.com
Agile Development Team Testing Strategies Agile development teams generally follow a whole team strategy where people with testing skills are effectively embedded into the development team and the team is responsible for the majority of the testing. This strategy works well for the majority of situations but when your environment is more complex you'll find that you also need an independent test team working in parallel to the development and potentially performing end-of-lifecycle testing as well. Regardless of the situation, agile development teams will adopt practices such as continuous integration (CI) which enables them to do continuous regression testing, either with a testdriven development (TDD) or test-immediately after approach. Source: Scott Ambler
TOP 5 actions for DWH test automation Automatic regression test: Fast and frequent changing as well as code refactoring requires continuous regression testing. Simulation of source systems: Source systems do not deliver accurate test data; delivery is often not in time; the generation of synthetic test data makes the DWH software development independent of source systems. Automatic output control: Automatic output control allows to control the output directly after the test run; thereby the duration of test cycles will be reduced Process automation: In practice many processes like scheduling are not automated in test and development environments. This will become expensive in the long run as processes in agile development occur more frequently. Deployment automation: Deployment processes in test and production environments take much time. As they are very frequent in agile processes, the lack of automation can lead to a slowdown of processes. Source: Marcus Drost
Part 4 Example from practice Agile Test Tool DREAM (Marcus Drost)
DEMO TEST TOOL DREAM (www.testtooldream.com) World’s first tool for continuous database and data warehouse regression testing. Shouldn’t we be doing better? (Scott W. Ambler)Mission-critical business functionality is implemented in RDBMSs. In the survey, 63.7% of respondents indicated that their organizations did this, but of those only 46% had regression tests in place to validate the logic. Shouldn’t we be doing better? Author: Scott W.Ambler
Part 5 Problem awareness game Agile Test Tool DREAM (Marcus Drost)
Problem awareness game (from problem to action) The participants of the workshop talk with each other about the problems which they have found in the first part of the game. Now, the problems will be grouped together. Then, the participants try to learn more about the problems by evaluating them in a dimensional problem analysis. Use the net chart on the next page for the analysis. The result is a good start in order to think about possible solutions and actions. Try to find for every group of problems suitable actions!
Example dimensional problem analysis Source: Esther Derby (agile retrospectives )
Possible solutions 1.Regression testing for all subsystems in a chain. 2.Simulation of test data and generation of syntethic test data. 3.Automation of deployment processes. 4.Improving performance by usage of faster machines in the cloud. 5.Define test cases for every user story and functional specification. 6.Define expected output in advance, and formally for automatic output control. 7.Arrange when to test what; try to shift all tests to the left in the SD process 8.Establish test data management by splitting test data to the level of user stories 9.Make test data management flexible, continuous and re-useable. 10. Automatic regression testing and output controll to prevent manual work.
Finish Recapitulation of the workshop Feedback participants
Luxoft Agile Software Development Engineering Business Agility. Since 2004, Luxoft has embraced the Agile software development methodology to become ...
Agile software development is a set of software ... Agile testing; ... first proposed as a long research paper in the APSO workshop at the ...
... http://www.scrumhub.com/ What is Agile Scrum? Learn Scrum in under 10 minutes. Thi ... Agile Scrum Testing ... English Country: ...
... the leading framework for Agile software development ... Our certifications provide assurance that team members can immediately impact projects.
SNS Bank; Education: ... English. Professional working proficiency. Dutch. ... Agile Testing; Exploratory Testing; Risk Based Testing; Test Execution;
Amazon Web Services offers reliable, ... English. Explore AWS solutions and products. Deutsch; English; ... Amazon SNS. Push Notification ...
... English SQS Group. Select Country ... Agile Approach; Our ... The iqnite conferences are Europe's number one meeting place for experts in software ...