Published on November 10, 2014
1. Case Study: How Siemens Energy Saved Time and Money by Managing and Automating Testing for Its Recent Upgrade Project Produced by Wellesley Information Services, LLC, publisher of SAPinsider. © 2014 Wellesley Information Services. All rights reserved. Nathan Sharp Siemens Energy
2. 1 Case Study Project • Primary goal of the project Technical upgrade of SAP release 4.6C to ECC6.0 Support for SAP and Oracle versions was ending Required for new functionality after the project • Secondary goal of the project Implement and utilize automated testing software Aim was automation of 70% of manual business tests Upgrade was the pilot project
3. 2 Introduced a Governance Model • Attended StarEast Conference (2012) and heard the strong recommendation for a governance model • We wanted to avoid a “Wild West” scenario in automation tool • Governance model covers Granting access to the tool – but only after training Defining the development process Documenting and enforcing naming conventions Setting up and maintaining security in the tool
4. 3 Started Development of a “Product Bible” • The Product Bible includes Roles and responsibilities New user information Folder structure Training material Issue tracking Test design Best practices
5. 4 Started Development of a “Product Roadmap” • The Product Roadmap includes When we install and evaluate the next version Completion date for the Product Bible Which automation projects will be supported When additional modules will be acquired and/or implemented
6. 5 Appointed Automation Product Lead • Responsible internally for the service • Conduit for the vendor • Organize training • Prepare and manage budget • Build and maintain the Product Roadmap
7. 6 Automated Testing Tool Training • Diverse group of attendees – both business and technical • 3 classes, 40+ attendees trained in total • Each class 4.5 days in duration • Used SurveyMonkey to get feedback from attendees
8. 7 Building an Example to Impress • We chose a large end-to-end order-to-cash process • Collaborative effort across multiple groups • 30 reusable sub-processes and 391 steps • No manual intervention • Would take 2 or 3 days to accomplish manually • Runs in 12 minutes • Example has been used in various meetings and demos to help understanding of test automation and to get buy-in
9. 8 Built a SharePoint System to Support Automation 4 SharePoint linked lists make up the SharePoint testing database
10. 9 Documented Test Script Execution and Issues SharePoint used to track testing progress and allows for instant status reporting
11. 10 Progress at End of Round 1 Testing • Manual or Automation tests Executed 96% of tests Passed 94% of tests • However, less than 10% automation achieved
12. 11 Progress at End of Round 1 Testing (cont.) • Some root causes for low amount of automation Core team was not trained early and first Not enough time between tool purchase and use The business group initially driving for the tool developed new priorities The automation tool was mastered by only a few due to the functionality restriction caused by both the automation tool version and SAP version No powerful and interested sponsor
13. 12 Second Approach • Round 2 of upgrade testing completed with 100% executed and successful • Three business analysts trained on process capture module (now usable, as an SAP ECC 6.0 environment was available) • Those business analysts used the period following completion of upgrade testing and the end of project to work on further automation
14. 13 Second Approach (cont.) • Planned script automation fell below expectations Automation tasks more complicated and took longer than the individuals themselves estimated Business analysts not fully dedicated to the task In one case, it was just not interesting work for the individual
15. 14 Third Approach • Tests captured locally by business analysts (and later extended to end users) • Fully dedicated off-shore developer completed automation • 69% of planned automation achieved with this approach
16. 15 Third Approach (cont.) • Renewed management commitment due to the capture/off-shore approach which works for us • Developed a product roadmap – including new versions and modules • Automation extended to other projects • “Lights-out” testing was developed. This is the scheduling of test scripts to execute while unattended
17. 16 Lights-Out Test Example • The script is a total of 2,347 steps and runs in approximately 16 minutes • The script tests the following 7 closing programs: Reclass PC equity Reclass 482/483 Z85/Z86 entries Income statement closeout Class 6 to 7 allocation Personnel cost reclass Calculate and post third-party inventories
18. 17 Lights-Out Test Example (cont.) • Additionally, the script: Creates 4 unique multi-line journal entries in two different company codes using both actual and statistical accounts (for unique data creation) Changes status on WBS elements and internal orders Changes sets and custom CL+ tables Runs the profit center balancing program Automatically determines current date/period/fiscal year Validates the results of the 7 closing programs against expected results from the JEs created
19. 18 Lights-Out Test Example (cont.) • Benefits: This script detected an issue before the changes planned by a new project were put live Prevented a major failure in production saving both IT and business time
20. 19 Expectations from Automation • Efficiency and labor savings Hundreds of manual testers freed up • Shorter project timelines • With 70% test automation, just one upgrade project on one of our SAP boxes would eliminate testing costs of $270,000
21. 20 Expectations from Automation (cont.) • Improved quality Fewer production defects Catch defects before they impact the business • Supports our direction to move from Waterfall to Agile development frameworks Testing is required in each development iteration and an automation tool can allow this to happen more efficiently
22. 21 Internal Conference Held • Two days of sessions related specifically to the automation tool • Attended by key players from various Siemens companies • Shared plans, issues, hints, and tips
23. 22 7 Key Points to Take Home • Sponsorship is essential for ensuring success of any project • Test automation is a project to itself – don’t try to combine with another initiative • Appointing a Product Lead helps ensure there is someone driving progress • Creating a Product Roadmap will give direction to an automation initiative • Avoid the “Wild West” in the automation tool with governance • Ensure there is a mechanism for tracking manual and automatic test development and execution • A great way to generate excitement is to create an “Example to Impress” automated test
Speaker: Matt Stine Developing for the Cloud Track Marc Andressen has famou...
This presentation explains how to develop a Web API in Java using (JAX-RS or Restl...
How to bring innovation to your organization by streamlining the deployment proces...
Cisco Call-control solutions can handle voice, video and data
Nathan Sharp of Siemens Energy recently spoke at the SAP Project Management in Atlanta and shared 7 important elements for the successful adoption of ...
It has a component that can simplify the configuration of Siemens PCS 7 automation ... this tips and tricks session ... of embedded energy that is ...
... configuration and maintenance of the system but no success. ... enescobedo at: 5/7/2009 9 ... It is a Siemens Power Generation division (energy ...
Siemens SIMATIC Step 7 ... i am having a training on automation at present particularly in siemens s7 theory,where ... If any one need success in ...
The Green Building Monitor provides you with ... And there is more: energy saving tips ... even visitors can see the success of your company's ...
Siemens is the world’s automation technology leader. ... Your Global Leader for Automation Technology. From A to Z, Siemens is the ... Siemens Automation ...
SIRIUS Media. Tools. ... About Siemens; Automation Technology; Drive Technology; Services; ... Energy efficiency tool SinaSave 6.7.