Performance Testing from the Cloud

50 %
50 %
Information about Performance Testing from the Cloud

Published on February 10, 2014

Author: TomKleingarn



Presented by Tom Kleingarn at HP Discover 2012

Your test system has met all performance requirements, but now your executive team wants external metrics on production hardware before you go-live. You could spend tens of thousands of dollars to work with a third-party vendor for a few hours of external load tests, or run equivalent tests from the public cloud with the money you have in your budget right now.

Learn how to respond to calls for high-capacity external load tests quickly and effectively. The presenter, Tom Kleingarn, performance engineer with Digital River will cover the reasons cloud-based testing is attractive and the common challenges involved. Then he will move on to practical examples of implementing a cloud-based testing infrastructure on Amazon’s Elastic Compute Cloud (EC2) platform, with examples for the configuration of Windows and Linux virtual machines.

The session will end with a demonstration of how extensible and cost effective a cloud testing infrastructure can be. You’ll come away knowing when external tests are applicable, and how you can avoid common pitfalls and confidently test from the cloud.

Performance Testing from the Cloud Presented at HP Discover 2012 Tom Kleingarn Technical Manager, Performance Engineering Digital River

Agenda > Digital River > Case Study > The Cloud > Challenges > Capacity > Accuracy > Connectivity > Optimization > Summary

> Global Cloud Commerce Solution Provider > Founded in 1994 > Over $3 billion in annual online sales > Daily Stats > > > > 20 million page views 30 million emails 175,000 orders 5 terabytes of digital content > 8 global data centers > Industries > > > > Software Consumer Electronics Games Payment Processing

Case Study > Dec, 2011 – Digital River launched high-capacity order capture solution > Capable of capturing 6,000 orders per second > Per hour, that’s… > 2 million orders > 400 stadiums > The state of Nevada > 3 day timeline for external performance validation

Challenges > Load Generators > Capacity > Tuning > Space > Supporting teams > License cost > Internal network infrastructure > Time!

Solutions > Add internal generators > Cost, space, infrastructure, installation, time > Not external > External vendor > Cost, connectivity, unfamiliar with app > External generators in public cloud > Connectivity

The Cloud > Virtual machine (VM) impermanence > Capacity and accuracy > Connectivity

The Cloud

The Cloud Impermanence

The Cloud Impermanence

The Cloud Capacity and accuracy > > Dedicated CPU, memory, local storage > Shared network, disk subsystem > Larger instances guarantee less variability in shared resources

Cost >

The Cloud Connectivity 54345 Controller Generator 50500 Controller <PORT> System Under Test (SUT) 443 MI Listener Generator System Under Test (SUT)

The Cloud Connectivity 54345 Controller <PORT> Generator System Under Test (SUT) RDP (3389) - SSH (22)

Installation > Launch an Instance

Associate an IP Address > Each generator needs a static IP

Installation - Windows > Windows Server 2008 > Limited to c1.xlarge > 8 Cores, 7GB RAM > I/O performance: High 1. RDP to running instance 2. Download LoadRunner 3. Mount image 4. Install 5. Open port 54345

Installation - Linux > A little more complicated ... > Red Hat Enterprise Linux 6.2 > Limited to c1.xlarge > 8 Cores, 7GB RAM > I/O performance: High 1. Connect via SSH and Keyfile > Windows - Use PuttyGen to generate .ppk file from .pem file 2. Log in as root user

Installation - Linux 4. Set environment variables > To set automatically on startup, create a .profile file in the “/root” directory, add this to the file: export M_LROOT=/opt/HP/HP_LoadGenerator export LD_LIBRARY_PATH=${M_LROOT}/bin export PATH=${M_LROOT}/bin:${PATH} export DISPLAY=localhost0.0 5. Copy LoadRunner installation files to the instance > Use WINSCP to connect and copy via SFTP

Installation - Linux 6. Unzip the archive > unzip <filename> 7. Mount the image Create the directory i.e. mount point: > mkdir -p /mnt/disk Use mount command as follows to mount iso file called disk1.iso: > mount -o loop LG_Linux_Install.iso /mnt/disk Change directory to list files stored inside an ISO image: > cd /mnt/disk > ls -l

Installation - Linux 8. Run the installer: > /mnt/disk/Linux/ 9. Check files in path of LoadRunner installation: > /opt/HP/HP_LoadGenerator 10. Install additional required packages > yum install compat-lib* > yum install glibc.i686 > yum install 11. Install daemon process /opt/HP/HP_LoadGenerator/bin/m_daemon_setup -install

Installation - Linux 12. Create a wrapper script the starts the m_daemon_setup process in /usr/local/bin (to follow common practices) #!/bin/bash # LoadRunner load generator daemon startup script echo Initializing load generator daemon process … echo This process is required to connect to the generator from the controller machine echo without using RSH # Installation command is: # /opt/HP/HP_LoadGenerator/bin/m_daemon_setup -install # If the daemon is already installed, just start it with the following command m_daemon_setup start $M_LROOT/bin # Other args include: -kill, -remove, -start, -stop, -stat

Installation - Linux 13. Add this to your .profile file to run the script on startup > source /usr/local/bin/init_loadgenerator_daemon > Considerations, additional security > MI Listener configuration > RSH vs. daemon process > Host Security Manager (since LR 9.5)

Connect Controller to Generators > Windows, no extra steps > Linux, specify UNIX OS > Log in via SSH, verify connectivity and daemon process > Generator Details > Controller > Unix Environment > Check “Don’t use RSH”

Generator - Unix Environment Settings

Optimize for High Throughput > Load generators create thousands of connections > Limit of 65,534 connections per network interface > Avoid port starvation, increase the port range used, and allow reuse of sockets in TIME_WAIT > Windows > HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServi cesTcpipParameters > Set value to 65534 > Linux > sysctl -w net.ipv4.ip_local_port_range="1024 65535“ > sysctl -w net.ipv4.tcp_tw_recycle=1 > Linux has a throughput advantage here!

Use Multiple Data Sources Controller Cloud Generator Physical Generator System Under Test (SUT) External Monitor

Case Study Results > Cloud generators yielded reliable results > Alignment across internal generators, external generators, and independent client validation > Results from single server yielded 2,000+ orders per second

Case Study Results > Linear scale confirmed Orders Captured per Second by Server Count 35,000 30,000 Orders per Second 25,000 20,000 15,000 10,000 5,000 0 1 2 3 4 5 6 7 8 Server Count 9 10 11 12 13 14 15

Summary > The cloud is cost effective but unpredictable > Capacity > Use the largest footprint possible > Accuracy > Compare results across data sources > Monitor physical and virtual utilization (if possible) > Connectivity > 54345 or MI Listener > Linux - LR Daemon or RSH > Configure optimal port range, remember Linux advantage

Benefits > Cost Savings > Hardware > $4000 for 8 core, 8GB server > $400 monthly maintenance > External vendor - $20K > Time > No dependence on infrastructure team for provisioning/support/maintenance > On demand flexibility


Add a comment

Related presentations

Related pages

Performance and Load Testing from the Cloud - Blitz by Spirent

Load testing from the cloud. Performance testing on websites, web apps and REST APIs with Blitz
Read more

Load and Performance Testing from the Cloud

Provision Load Generators Fast and Securely. Load generators spin up in seconds; All cloud load generators are preconfigured; IPs can be reserved for ...
Read more

Performance Testing From the Cloud | TIM Review

"Anything you build on a large scale or with intense passion invites chaos." Francis Ford Coppola. Mr. Coppola was, of course, talking about the making of ...
Read more

Load test in the cloud | Visual Studio Team Services

... Load test your app in the cloud. ... //
Read more

Cloud testing - Wikipedia, the free encyclopedia

Overview. Cloud Testing uses cloud infrastructure for software testing. Organizations pursuing testing in general and load, performance testing and ...
Read more

Performance Testing Tools Software | SOASTA

... scalable performance testing tool for understanding and optimizing web performance ... Webinar Performance Warrior Tales: Cloud Testing the Retail Giants
Read more

Performance testing in the cloud - Load Testing & Website ...

Use WebLOAD to run performance testing from the cloud by generating load from Amazon EC2 within minutes.
Read more

Blitz: Performance and Load Testing from the Cloud ...

... Performance and Load Testing from the Cloud by Douglas Karr on ... automated performance testing can ensure that no code push can cause a ...
Read more

Performance testing in the cloud - Software Quality ...

Looking for something else? Automated tools essential when deploying apps to the cloud; Hosting in cloud computing: Using Software as a Service for source ...
Read more

How to improve performance testing from the Azure Cloud ...

Recently in the SOASTA 55.20 update we added support for Microsoft Azure Resource Manager (ARM), a management feature of Microsoft Azure. After moving to ...
Read more