The benefits of using an APM solution while performance testing

50 %
50 %
Information about The benefits of using an APM solution while performance testing
Technology

Published on March 6, 2014

Author: devopsguys

Source: slideshare.net

Description

The benefits of using an APM solution while performance testing or "why load testing without APM is like Corona without the lime...".

The deck covers a brief overview of APM, the market & major players, and 4 key benefits from using APM tools during your performance testing cycle.

DevOpsGuys The benefits of using an APM solution while performance testing Why Perf testing without APM is like Corona without the Lime… @DevOpsGuys DevOpsGuys www.devopsguys.com 1

http://www.flickr.com/photos/lendog64/5781379311/ @DevOpsGuys DevOpsGuys www.devopsguys.com 2

Perf Testing & APM in 30mins • • • • • Why Performance Test? What is APM anyway? Benefits of APM when Perf Testing With some case studies along the way! Q&A @DevOpsGuys DevOpsGuys www.devopsguys.com 3

Why Performance Test? 1. Ensure that we meet our customer’s expectations of website response time – So that they convert and we make £££ 2. Ensure that we have sufficient computing capacity to meet #1 under load (load testing) 3. Understand how the system reacts when things go wrong (stress testing) @DevOpsGuys DevOpsGuys www.devopsguys.com 4

@DevOpsGuys DevOpsGuys www.devopsguys.com 5

APM Defined “In the fields of information technology and systems management, application performance management (APM), is the monitoring and managing of performance and availability of software applications. APM strives to detect and diagnose application performance problems to maintain an expected level of service. APM is the translation of IT metrics into business meaning (i.e. value).” http://en.wikipedia.org/wiki/Application_Performance_Management @DevOpsGuys DevOpsGuys www.devopsguys.com 6

“5 Dimensions of APM*” 1. End user experience monitoring - (Active and passive) 2. Application runtime architecture discovery and modeling 3. User-defined transaction profiling (also called business transaction management) 4. Application component monitoring 5. Reporting & Application data analytics *According to Gartner  @DevOpsGuys DevOpsGuys www.devopsguys.com 7

Gartner APM Magic Quadrant 2013 2012 2011 IBM Riverbed CA Compuware Oracle Quest Opnet Microsoft BM Dell SL C New Relic HP AppDynamics OpTier Inetco AppNeta NetScout Precise Progress Managed Engine eG Innovations Network Instruments Idera Visual networks ASG Correlsense Nastel Coradient Aternity Knoa Info vista OpTier Arctures Technologies @DevOpsGuys DevOpsGuys www.devopsguys.com

Never leave home without it… BENEFITS OF LOAD TESTING WITH APM TOOLS @DevOpsGuys DevOpsGuys www.devopsguys.com 9

#1 See the Big Picture (aka Systems Thinking) @DevOpsGuys DevOpsGuys www.devopsguys.com 10

Systems Thinking 101 “Systems thinking is a framework for seeing interrelationships rather than things, for seeing patterns rather than static snapshots. It is a set of general principles spanning fields as diverse as physical and social sciences, engineering and management.” Peter Senge @DevOpsGuys DevOpsGuys www.devopsguys.com 11

Command & Control vs Systems Thinking @DevOpsGuys DevOpsGuys http://www.systemsthinking.co.uk/2-1.asp www.devopsguys.com 12

“any improvement not made at the constraint is an illusion” CASE STUDY #1 2 WEEKS WASTED… @DevOpsGuys DevOpsGuys www.devopsguys.com 13

The Scenario E-commerce customer New website High data-start times on page load Poor load testing results Initial conversation with client 2 week delay while hosting provider scaled out the hardware… • Problem still not fixed… • • • • • • @DevOpsGuys DevOpsGuys www.devopsguys.com 14

Waterfall Diagram of page load times What’s the root cause of the ~1sec Data Start? @DevOpsGuys DevOpsGuys www.devopsguys.com 15

@DevOpsGuys DevOpsGuys www.devopsguys.com 16

#2 Drill down to the Detail (Code-level) @DevOpsGuys DevOpsGuys www.devopsguys.com 17

Find the slow code! @DevOpsGuys DevOpsGuys www.devopsguys.com 18

#3 Faster Iteration = Better Value @DevOpsGuys DevOpsGuys www.devopsguys.com 19

Maximise Testing Value Deploy Fix @DevOpsGuys Test Analyse DevOpsGuys www.devopsguys.com 20

9am to 9pm. 12hrs. 56x faster! CASE STUDY #1 56X FASTER IN A DAY… @DevOpsGuys DevOpsGuys www.devopsguys.com 21

The Scenario • • • • • New website pre-launch testing Customer Portal (SaaS solution) Start Load testing 9am 4 “Test & Fix” cycles in the day Final testing completed at 9pm @DevOpsGuys DevOpsGuys www.devopsguys.com 22

The End Results = 56x Throughput (Req/sec) 16/min 59/min 56x faster Initial Testing Final Result Average Response 3024ms 54ms 3.5x throughput Before 3024ms @DevOpsGuys DevOpsGuys After 54ms www.devopsguys.com 23

Nice flat line! Before After @DevOpsGuys DevOpsGuys www.devopsguys.com 24

#4 Team Work “The Enemy is Performance, not each other…” @DevOpsGuys DevOpsGuys www.devopsguys.com 25

The Blame Game… Ops @DevOpsGuys QA DevOpsGuys Dev www.devopsguys.com 26

The Fix-It Game… Ops Dev QA @DevOpsGuys DevOpsGuys www.devopsguys.com 27

@DevOpsGuys DevOpsGuys www.devopsguys.com 28

Summary 1. 2. 3. 4. See the Big Picture (systems thinking) Drill down to the details Faster Iteration = Better Value Stop the “Blame Game” – aka “There’s your Problem!” @DevOpsGuys DevOpsGuys www.devopsguys.com 29

DevOpsGuys Thank You Workshop Session @ 1345 End-to-end testing & APM demo operations | availability | performance | scalability | migration | strategy @DevOpsGuys DevOpsGuys www.devopsguys.com 30

What We Do DevOpsGuys provide development & operations support (DevOps) services for your online applications. You run your business. We run your website. It’s as simple as that. @DevOpsGuys DevOpsGuys www.devopsguys.com 31

Our service portfolio • 24 x 7 Application-Centric web operations • DevOps Transformation & Training • Continuous Delivery implementation • Infrastructure Automation & “Infrastructure as Code” • Application Performance Management • AppDynamics Implementation & Consultancy @DevOpsGuys • Web Performance consulting • Web monitoring • Load Testing • Test Automation and scripting • JetBrains Product Implementation & Consultancy • Octopus Deploy implementation DevOpsGuys www.devopsguys.com 32

The Leadership Team James Smith (@TheDevMgr) Steve Thair (@TheOpsMgr) • 16 years of software delivery experience. • Proven track record • 25 years of IT Operations experience. • Proven track record – – – – – – – Pizza Hut KFC Dell RAC Aviva Fujitsu Siemens Whitbread @DevOpsGuys – – – – – – – DevOpsGuys Vodafone Credit Suisse Group IBM BNP Paribas Zurich Financial Services Totaljobs Group TSL Education www.devopsguys.com 33

Contact Us Visit us at www.devopsguys.com Call our team on +44 (0)20 8798 3079 Email our engineers at team@devopsguys.com @DevOpsGuys DevOpsGuys www.devopsguys.com 34

Add a comment

Related presentations

Related pages

Performance Testing and APM - NCC Group

The benefits of using an APM solution while performance testing ... •Benefits of APM when Perf Testing •Q&A 3 @DevOpsGuys DevOpsGuys www.devopsguys.com
Read more

The benefits of using an APM solution while performance ...

Website Load Testing ; Performance Analyser ; Software Testing. Overview ; Software Testing Services ; Test Centre ; Specialist Software Testing ;
Read more

The benefits of using an #APM solution while performance ...

DevOpsGuys presented at an NCC Group customer event yesterday on “The benefits of using an APM solution while performance testing”. We gave a 30min ...
Read more

Keynote: The benefits of using an APM solution when ...

... The benefits of using an APM solution when performance testing ... Live performance tests using an APM solution Steve Thair ...
Read more

APM | DevOpsGuys

DevOpsGuys presented at an NCC Group customer event yesterday on “The benefits of using an APM solution while performance testing”. We gave a 30min ...
Read more

The fastest, most automated performance testing solution ...

NeoLoad load and performance testing solution helps you ... design time while helping you automate testing with your ... integrated Neotys ...
Read more

10 Bottom-Line Business Benefits of APM | APMdigest ...

But the reality is that no company should be deploying Application Performance ... by using APM solutions ... benefits that an APM solution ...
Read more

Leveraging CA APM in Pre-Production Environments | June ...

Leveraging CA APM in Pre-Production Environments ... By using CA APM for performance testing and ... While the focus was on quantifiable benefits, ...
Read more