The benefits of using an APM solution while performance testing

20 %
80 %
Information about The benefits of using an APM solution while performance testing

Published on March 6, 2014

Author: devopsguys



The benefits of using an APM solution while performance testing or "why load testing without APM is like Corona without the lime...".

The deck covers a brief overview of APM, the market & major players, and 4 key benefits from using APM tools during your performance testing cycle.

DevOpsGuys The benefits of using an APM solution while performance testing Why Perf testing without APM is like Corona without the Lime… @DevOpsGuys DevOpsGuys 1 @DevOpsGuys DevOpsGuys 2

Perf Testing & APM in 30mins • • • • • Why Performance Test? What is APM anyway? Benefits of APM when Perf Testing With some case studies along the way! Q&A @DevOpsGuys DevOpsGuys 3

Why Performance Test? 1. Ensure that we meet our customer’s expectations of website response time – So that they convert and we make £££ 2. Ensure that we have sufficient computing capacity to meet #1 under load (load testing) 3. Understand how the system reacts when things go wrong (stress testing) @DevOpsGuys DevOpsGuys 4

@DevOpsGuys DevOpsGuys 5

APM Defined “In the fields of information technology and systems management, application performance management (APM), is the monitoring and managing of performance and availability of software applications. APM strives to detect and diagnose application performance problems to maintain an expected level of service. APM is the translation of IT metrics into business meaning (i.e. value).” @DevOpsGuys DevOpsGuys 6

“5 Dimensions of APM*” 1. End user experience monitoring - (Active and passive) 2. Application runtime architecture discovery and modeling 3. User-defined transaction profiling (also called business transaction management) 4. Application component monitoring 5. Reporting & Application data analytics *According to Gartner  @DevOpsGuys DevOpsGuys 7

Gartner APM Magic Quadrant 2013 2012 2011 IBM Riverbed CA Compuware Oracle Quest Opnet Microsoft BM Dell SL C New Relic HP AppDynamics OpTier Inetco AppNeta NetScout Precise Progress Managed Engine eG Innovations Network Instruments Idera Visual networks ASG Correlsense Nastel Coradient Aternity Knoa Info vista OpTier Arctures Technologies @DevOpsGuys DevOpsGuys

Never leave home without it… BENEFITS OF LOAD TESTING WITH APM TOOLS @DevOpsGuys DevOpsGuys 9

#1 See the Big Picture (aka Systems Thinking) @DevOpsGuys DevOpsGuys 10

Systems Thinking 101 “Systems thinking is a framework for seeing interrelationships rather than things, for seeing patterns rather than static snapshots. It is a set of general principles spanning fields as diverse as physical and social sciences, engineering and management.” Peter Senge @DevOpsGuys DevOpsGuys 11

Command & Control vs Systems Thinking @DevOpsGuys DevOpsGuys 12

“any improvement not made at the constraint is an illusion” CASE STUDY #1 2 WEEKS WASTED… @DevOpsGuys DevOpsGuys 13

The Scenario E-commerce customer New website High data-start times on page load Poor load testing results Initial conversation with client 2 week delay while hosting provider scaled out the hardware… • Problem still not fixed… • • • • • • @DevOpsGuys DevOpsGuys 14

Waterfall Diagram of page load times What’s the root cause of the ~1sec Data Start? @DevOpsGuys DevOpsGuys 15

@DevOpsGuys DevOpsGuys 16

#2 Drill down to the Detail (Code-level) @DevOpsGuys DevOpsGuys 17

Find the slow code! @DevOpsGuys DevOpsGuys 18

#3 Faster Iteration = Better Value @DevOpsGuys DevOpsGuys 19

Maximise Testing Value Deploy Fix @DevOpsGuys Test Analyse DevOpsGuys 20

9am to 9pm. 12hrs. 56x faster! CASE STUDY #1 56X FASTER IN A DAY… @DevOpsGuys DevOpsGuys 21

The Scenario • • • • • New website pre-launch testing Customer Portal (SaaS solution) Start Load testing 9am 4 “Test & Fix” cycles in the day Final testing completed at 9pm @DevOpsGuys DevOpsGuys 22

The End Results = 56x Throughput (Req/sec) 16/min 59/min 56x faster Initial Testing Final Result Average Response 3024ms 54ms 3.5x throughput Before 3024ms @DevOpsGuys DevOpsGuys After 54ms 23

Nice flat line! Before After @DevOpsGuys DevOpsGuys 24

#4 Team Work “The Enemy is Performance, not each other…” @DevOpsGuys DevOpsGuys 25

The Blame Game… Ops @DevOpsGuys QA DevOpsGuys Dev 26

The Fix-It Game… Ops Dev QA @DevOpsGuys DevOpsGuys 27

@DevOpsGuys DevOpsGuys 28

Summary 1. 2. 3. 4. See the Big Picture (systems thinking) Drill down to the details Faster Iteration = Better Value Stop the “Blame Game” – aka “There’s your Problem!” @DevOpsGuys DevOpsGuys 29

DevOpsGuys Thank You Workshop Session @ 1345 End-to-end testing & APM demo operations | availability | performance | scalability | migration | strategy @DevOpsGuys DevOpsGuys 30

What We Do DevOpsGuys provide development & operations support (DevOps) services for your online applications. You run your business. We run your website. It’s as simple as that. @DevOpsGuys DevOpsGuys 31

Our service portfolio • 24 x 7 Application-Centric web operations • DevOps Transformation & Training • Continuous Delivery implementation • Infrastructure Automation & “Infrastructure as Code” • Application Performance Management • AppDynamics Implementation & Consultancy @DevOpsGuys • Web Performance consulting • Web monitoring • Load Testing • Test Automation and scripting • JetBrains Product Implementation & Consultancy • Octopus Deploy implementation DevOpsGuys 32

The Leadership Team James Smith (@TheDevMgr) Steve Thair (@TheOpsMgr) • 16 years of software delivery experience. • Proven track record • 25 years of IT Operations experience. • Proven track record – – – – – – – Pizza Hut KFC Dell RAC Aviva Fujitsu Siemens Whitbread @DevOpsGuys – – – – – – – DevOpsGuys Vodafone Credit Suisse Group IBM BNP Paribas Zurich Financial Services Totaljobs Group TSL Education 33

Contact Us Visit us at Call our team on +44 (0)20 8798 3079 Email our engineers at @DevOpsGuys DevOpsGuys 34

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Performance Testing and APM - NCC Group

The benefits of using an APM solution while performance testing ... •Benefits of APM when Perf Testing •Q&A 3 @DevOpsGuys DevOpsGuys
Read more

The benefits of using an APM solution while performance ...

Website Load Testing ; Performance Analyser ; Software Testing. Overview ; Software Testing Services ; Test Centre ; Specialist Software Testing ;
Read more

The benefits of using an #APM solution while performance ...

DevOpsGuys presented at an NCC Group customer event yesterday on “The benefits of using an APM solution while performance testing”. We gave a 30min ...
Read more

Keynote: The benefits of using an APM solution when ...

... The benefits of using an APM solution when performance testing ... Live performance tests using an APM solution Steve Thair ...
Read more

APM | DevOpsGuys

DevOpsGuys presented at an NCC Group customer event yesterday on “The benefits of using an APM solution while performance testing”. We gave a 30min ...
Read more

The fastest, most automated performance testing solution ...

NeoLoad load and performance testing solution helps you ... design time while helping you automate testing with your ... integrated Neotys ...
Read more

10 Bottom-Line Business Benefits of APM | APMdigest ...

But the reality is that no company should be deploying Application Performance ... by using APM solutions ... benefits that an APM solution ...
Read more

Leveraging CA APM in Pre-Production Environments | June ...

Leveraging CA APM in Pre-Production Environments ... By using CA APM for performance testing and ... While the focus was on quantifiable benefits, ...
Read more