Q2.12: Benchmarking Techniques

36 %
64 %
Information about Q2.12: Benchmarking Techniques

Published on March 20, 2014

Author: linaroorg

Source: slideshare.net


Resource: Q2.12
Name: Benchmarking Techniques
Date: 30-05-2012
Speaker: Michael Hope

Benchmarking Techniques Michael Hope <michael.hope@linaro.org> bzr branch lp:~michaelh1/+junk/benchmarking-techniques r6

2 Why benchmark?

3 Issues are Relevance Accuracy Repeatability

4 Picking relevant benchmarks: Profile Workload Features We use SPEC 2000 and EEMBC We'd like shareable benchmarks

5 Test Platform Build / test / benchmark via Linux / web / commodity hardwar

6 Measuring

7 Realtime timers See 'man clock_gettime'

8 <comparison of different timers> See https://wiki.linaro.org/WorkingGroups/ToolChain/Benchmarks/TimerAc



11 External influences

12 Other features to be wary of scheduler governor cpuidle, power management, thermal limiting SMP bugs, like core lockdown NEON startup See “Understanding the Linux Kernel” ch10: http://oreilly.com/catalog/linuxkernel/chapter/ch10.html

13 Putting things together and running We use: timer built into the app run five times collect everything post process

14 Statistics



17 Standard deviation Dispersion / coefficient of variance t-scores t-test

18 t= ̄X 1 − ̄X 2 √ s1 2 N 1 + s2 2 N 2 See http://en.wikipedia.org/wiki/Welch%27s_t_

19 Compiler Mean Std CV gcc-4.6.2 1.00 134u 134u gcc-linaro-4.6-2011.11 4.25 5035u 1178u t = 30,300

21 Variant Mean Std CV Plain 1.00 320u 320u With SMS 1.01 430u 426u t = 118

22 Variant Mean Std CV Plain 1.00 320u 320u With vectoriser 1.001 404u 404u t = 8.27 - significant

23 Our tools perf difftest betters tabulate Python LibreOffice! Other statistical tools like scipy.stat, R, Judge, and ministat

24 http://help.libreoffice.org/Calc/Applying_AutoFilter

25 www.linaro.org / wiki.linaro.org people.linaro.org/~michaelh/presentations

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Q2/2012 Report Conference Call - aap Implantate AG

Bone Cements and Cementing Techniques: +0.7 Mio. € ... •No change in balance sheet total Q2/12: 67.4 Mio. € ... Above benchmark! 10
Read more

Jatin Kumar | LinkedIn

View Jatin Kumar's professional ... Industry/market analysis using various tools & techniques, Competitor Benchmarking ... Gold Award-Q2'12" for ...
Read more

Intel® Core™ i3-3217U Processor (3M Cache, 1.80 GHz ...

Intel® Core™ i3-3217U Processor (3M Cache, ... Q2'12: Processor Number: i3 ... For benchmarking data see http://www.intel.com/performance.
Read more

Intel® Xeon® Processor E5-1410 (10M Cache, 2.8 GHz ...

Intel® Xeon® Processor E5-1410 (10M Cache, ... Q2'12: Processor Number: E5-1410: L3 Cache: 10 MB: ... For benchmarking data see http://www.intel.com ...
Read more

Deliverable Title: Demo 5 Specification and Early Benchmarking

... BACKGROUND PREVIOUS STUDY TECHNIQUES METHODOLOGY Aspects of the Presence ... Benchmark testing of the BENOGO ... Bird sound’ (Q2-12 ...
Read more

Kronos Incorporated - Put Labor Performance Standards to ...

... enabling you to measure your operations against productivity goals and benchmarks. ... 2012– Discover how to apply proven Lean techniques to the ...
Read more