Published on March 1, 2014
Synthetic performance monitoring with Selenium Andriy Samilyak OpsWay CEO
Who is talking? Working in IT since 2005 ● Since 2008 - Smile Ukraine CTO ● Since 2013 – OpsWay CEO ● Coaching with SmartMe.com.ua ●
You know this... “Metrics show that a 0.1 second delay in page rendering can translate into a 1% drop in customer activity” Amazon research papers
Projects are changed daily ● Application code ● Product database ● Marketing tags ● Server configuration ● External services ● etc … and no QA engineers at all...
User checkout cycle under control Add to cart Select product Success page! Login ~30-50 sec Billing/Shipping options Payment
Is it still running smoothly? ● Passive checks – analyzing end-user activity – – Webserver log analysis – ● NewRelic Web analytics (real user monitoring) Active checks – run test and record results – Gomez GPN – Uptrends – Pingdom
NewRelic Cool, sexy, not so cheap and not really what we need
GoogleAnalytics with UserTiming https://github.com/opsway/magento-mod-rum … but we need immediate alerts!
Server-side log analysis ● Complex models: – 2 add to cart requests timing – 1 login step timing – 5 checkout steps timings ● not comparable side by side ● not representative when user activity is low
Server-side logs: false-positive 3 mins to checkout? Are you serious?
Pingdom/Uptrends ● Transaction monitoring via scripting ● Fancy scripting language ● ● Robust, popular but not so cheap for multiple projects Fine for public sites, but not for intranet
We liked Compuware Gomez!
Compuware Gomez is cool
But we have already Selenium! Selenium Splunk 1. Load product page Send timestamp via UDP 2. Add to cart Send timestamp via UDP 3. Shopping cart Send timestamp via UDP 4. Address 5. Payment 6. Success page ...
It looks like this in code...
Splunk combines to transaction...
We can chart transactions
… and alert on abnormal behavior Slow transaction Failed Transaction!
Why did transaction fail? ● Selenium reported timeout – not always sufficient ● Rarely reproducible (many servers) ● We needed screenshots!
Capturing a screenshot
Saving screenshot with Jenkins
Jenkins build with errors We got it!
Why it took so long? ● ● We don't know always what http request blocked transaction Inspired by Gomez we integrated HAR recording of test sessions
Inspired by Gomez ...
HAR is not perfect, but ...
HAR proxy ● Based on BrowserMob proxy server http://bmp.lightbody.net/ HAR file Launching WebDriver Selenium Grid Proxy on port 64001 Request passes through proxy www.mysite.com Selenium node (Firefox)
Jenkins build with errors Here we have HAR
Issues we have ● Supporting test scripts takes a lot of time ● Support test infrastructure is tedious ● False-positives ● Still not enough debug information
Andriy Samilyak firstname.lastname@example.org skype: samilyaka http://opsway.com
Load and Performance Testing with Selenium. ... the performance of the web application ... for doing load and performance testing of a web application ...
Performance testing using Selenium ... even if you improve performance elsewhere the users of the system will not see an improvement in the ...
Synthetic performance ... Selenium WebDriver Testing Selenium Jobs ... Selenium to automate and test web ...
... it is for automating web applications for testing purposes, ... Selenium WebDriver ... Selenium Remote Control; Selenium WebDriver;
See why Catchpoint’s Synthetic Monitoring Solution is the ... level by testing the performance of a ... testing is supported on both our Web and ...
How to perform load testing using Selenium WebDriver? ... The API or the backend's performance testing for ... How to perform feasible web smoke test with ...
Our Selenium WebDriver load testing ... The same skills you've honed using Selenium IDE or writing WebDriver ... When you call Web Performance you ...
TestMaker runs Selenium tests for browser compatibility testing of your Web ... Record a Selenium test in Selenium ... load and performance testing, ...