Published on June 2, 2016
1. Adaptive Media Streaming: The Role of Standards Dynamic Adaptive Streaming over HTTP Priv.-Doz. Dr. Christian Timmerer [Ack: Ali C. Begen, MediaMelon, Inc., OzyeginUniversity] Alpen-Adria-Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Information Technology (ITEC) w Multimedia Communication (MMC) w Sensory Experience Lab (SELab) http://blog.timmerer.com w http://selab.itec.aau.at/ w http://dash.itec.aau.at w email@example.com Chief Innovation Officer (CIO) at bitmovin GmbH http://www.bitmovin.comw firstname.lastname@example.org Tutorial @ QoMEX 2016, June 2016 http://www.slideshare.net/christian.timmerer
2. Importance of Multimedia Delivery • Multimedia is predominant on the Internet • Real-time entertainment – Streaming video and audio – More than 70% of Internet traffic at peak periods • Popular services – YouTube (17.85%), Netflix (37.05%), Amazon Video (3.11%), Hulu (2.58%) – All delivered over-the-top (OTT) June 2016 QoMEX 2016 Tutorial 2 Global Internet Phenomena Report: Dec 2015
3. Open Digital Media Value Chain June 2016 QoMEX 2016 Tutorial 3 Create Content Aggregate Monetize Distribute Content Consume Content Any Content Any Storefront Any Network Any Device CDNsMedia Protocols Internet Transport DRM Encoding Encapsulation Dynamic Ads Clients Happy User
4. What is Streaming? June 2016 QoMEX 2016 Tutorial 4 Streaming is transmission of a continuous content from a server to a client and its simultaneous consumption by the client Two Main Characteristics 1. Client consumption rate may be limited by real-time constraints as opposed to just bandwidth availability 2. Server transmission rate (loosely or tightly) matches to client consumption rate
5. Common Annoyances in Streaming • Wrong format • Wrong protocol • Plugin requirements • DRM issues • Long start-up delay • Poor quality • Frequent stalls • Quality oscillations • No seeking features June 2016 QoMEX 2016 Tutorial 5
6. Over-The-Top – Adaptive Media Streaming • In a nutshell… June 2016 QoMEX 2016 Tutorial 6 Adaptation logic is within the client, not normatively specified by the standard, subject to research and development
7. Multi-Bitrate Encoding and Representation Switching June 2016 QoMEX 2016 Tutorial 7 Contents on the Web Server Movie A – 200 Kbps Movie A – 400 Kbps Movie A – 1.2 Mbps Movie A – 2.2 Mbps . . . . . . Movie K – 200 Kbps Movie K – 500 Kbps Movie K – 1.1 Mbps Movie K – 1.8 Mbps . . . . . . Time (s) Start quickly Keep requesting Improve quality Loss/congestion detection Revamp quality . . . . . . Segments
8. Adaptive Streaming over HTTP June 2016 QoMEX 2016 Tutorial 8 … … … … HTTPGETs Client Buffer Media Player HTTP Server
9. Formats and Standards • Adobe – HTTP Dynamic Streaming (HDS) – Almost dead • Apple – HTTP Live Streaming (HLS) – Required for iOS • Microsoft – Smooth Streaming – Almost dead, too! Or not? • MPEG Dynamic Adaptive Streaming over HTTP (DASH) – Supported by Netflix, YouTube, Bitmovin, etc. June 2016 QoMEX 2016 Tutorial 9 Source: http://xkcd.com/927/
10. Scope of DASH: what is specified? June 2016 QoMEX 2016 Tutorial 10 Media Presentation on HTTP Server DASH-enabled ClientMedia Presentation Description . . . Segment … . . .Segment … . . . Segment … . . .Segment … … Segments located by HTTP-URLs DASH Control Engine HTTP/1.1 HTTP Client MPD Parser Media Engine On-time HTTP requests to segments
11. Scope of DASH: what is specified? June 2016 QoMEX 2016 Tutorial 11 Media Presentation on HTTP Server DASH-enabled ClientMedia Presentation Description . . . Segment … . . .Segment … . . . Segment … . . .Segment … … Segments located by HTTP-URLs DASH Control Engine HTTP/1.1 HTTP Client MPD Parser Media Engine On-time HTTP requests to segments
12. DASH Data Model June 2016 QoMEX 2016 Tutorial 12 MPD Period id = 1 start = 0 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 2 audio english Adaptation Set 1 BaseURL=http://abr.rocks.com/ Representation 2 Rate = 1 Mbps Representation 4 Rate = 3 Mbps Representation 1 Rate = 500 Kbps Representation 3 Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4 Segment Access Initialization Segment http://abr.rocks.com/3/0.mp4 Media Segment 1 start = 0 s http://abr.rocks.com/3/1.mp4 Media Segment 2 start = 10 s http://abr.rocks.com/3/2.mp4 Adaptation Set 3 audio german Adaptation Set 1 video Period id = 2 start = 100 s Representation 3 Rate = 2 Mbps Selection of components/tracks Well-defined media format Selection of representations Splicing of arbitrary content like ads Chunks with addresses and timing
13. June 2016 QoMEX 2016 Tutorial 13 type=static typically, for on demand content Base URL of the segments Subtitles Audio adaptation set with different representations (bw) Video adaptation set with different representations (bw) Different codecs (profiles) Segment URL constructed with template and base URL
14. http://www.dash-player.com/demo/ June 2016 QoMEX 2016 Tutorial 14
15. O Standard, Where Art Thou? What is a Standard? http://en.wikipedia.org/wiki/Standards A technical standard is an established norm or requirement. It is usually a formal document that establishes uniform engineering or technical criteria, methods, processes and practices. è I N T E R O P E R A B I L I T Y Types of Standard • Open standards – ITU-T, W3C, IETF, MPEG, Linux, … • Closed standards – MS MediaPlayer, … • De jure standards – ANSI, ISO, DIN, … • De facto standards – MS Windows, OS X, Unix, Linux, … • Mandatory standards – Health, radiation, … • Voluntary standards – JPEG, … June 2016 QoMEX 2016 Tutorial 15
16. Adaptive Streaming Content Workflow June 2016 QoMEX 2016 Tutorial 16 Source Transcoding Encapsulation Encryption Origin Server HelperDistribution Client Linear: Multicast VoD: FTP, RTMP, HTTP, etc. Unicast HTTP (PUSH), FTP, etc. HTTP GET small objects Single highest-bitrate stream Multiple streams at target bitrates Multiple streams at target encapsulation formats Large video/virtual files and manifests
17. Adaptive Streaming Content Workflow Simplified June 2016 QoMEX 2016 Tutorial 17 Standard Delivery Infrastructure (CDN) Source Transcoding Encapsulation Encryption Multiple streams: video [bitrate (32000-20000000), profile (baseline, main, high), preset (standard, professional, premium), height (128- 7680), width (96-4320), frame rate (1-120), codec (h264, hevc)], audio: [bitrate (8000-256000), sample rate (0, 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000, 64000, 88200, 96000)] Single highest-bitrate stream: HTTP, FTP, RTMP; mp4, ts; AVC, AAC, Subtitles Multiple streams at target encapsulation formats: DASH (MPD + mp4), HLS (m3u8, ts) Multiple streams with multiple DRM formats: MPEG-CENC, Widewine, PlayReady, PrimeTime, Fairplay Player Heterogeneous Clients e.g. Bitmovin HTML5 Adaptive Player DASH, HLS, HTML5, MSE, EME
18. QoE for DASH Services • Different application domains have different QoE requirements – Need to provide specializations of the general QoE definition – Take into account requirements formulated by means of influence factors and features of QoE • QoE influence factors for DASH – Initial/start-up delay (low) – Buffer underruns, stalls, freezes (zero) – Quality switches (low) – Media throughput (high) – … June 2016 QoMEX 2016 Tutorial 18
19. ! http://www.bitcodin.com/ http://www.dash-player.com/ QoE Evaluation for DASH-based Services • Test sequence – Many datasets available – Adopted Big Buck Bunny & DASHed it with bitcodin • Players – bitdash – Proprietary solutions (smooth, HLS, HDS) – YouTube, dash.js, DASH-JS – …and compare it with ten different adaptation algorithms • Objective evaluation – Common test setup using network emulation & bandwidth shaping – Predefined bandwidth trajectory (or real network traces) • Subjective evaluation – Lab [ITU-T B.500 / P.910] vs. crowdsourcing with special platforms or social networks June 2016 QoMEX 2016 Tutorial 19
20. Crowdsourced QoE Evaluation • Quality of Experience … – Mean Opinion Score [0..100] – [other objective metrics: start-up time, throughput, stalls] • … Web-based Adaptive HTTP Streaming Clients … – HTML5+MSE: DASH-JS (dash.itec.aau.at), dash.js (DASH-IF, v1.1.2), YouTube • … Real-World Environments … – DASH-JS, dash.js hosted at ITEC/AAU (~ 10Gbit/s) – YouTube hosted at Google data centers – Content: Tears of Steel @ 144p (250 kbit/s), 240p (380 kbit/s), 360p (740 kbit/s), 480p (1308 kbit/s), and 720p (2300 kbit/s); segment size: 2s – Users access content over the open Internet (i.e., real-world environment) • … Crowdsourcing – Campaign at Microworker platform (others also possible: Mechanical Turk, social networks) limited to Europe, USA/Canada, India – Screening Techniques: Browser fingerprinting, stimulus presentation time, QoE ratings and pre- questionnaire June 2016 QoMEX 2016 Tutorial 20 B. Rainer, C. Timmerer, “Quality ofExperience ofWeb-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”, Proceedings of International Workshop on VideoNext: Design,Quality andDeployment of Adaptive Video Streaming, Sydney, Australia,Dec. 2014.
21. MOS and Average Bitrate • 288 microworkers, 33 screened (Fingerprinting: 20, presentation time: 6, QoE ratings and pre- questionnaire: 7), 175 male and 80 female, majority (80%) is aged between 18 and 37 June 2016 QoMEX 2016 Tutorial 21
22. Startup Time and Number of Stalls June 2016 QoMEX 2016 Tutorial 22
23. Results Summary • DASH-JS – High start-up time – Low number of stalls – Good throughput, QoE • dash.js – Low start-up time – High # stalls – Low throughput – Low QoE • YouTube – Low start-up time – Low number of stalls – Best throughput, QoE June 2016 QoMEX 2016 Tutorial 23
24. Now, 10 different adaptation logics … June 2016 QoMEX 2016 Tutorial 24 Adaptation logics well-known in research literature Predefined bandwidth trajectory and test setup Different segment sizes, RTTs, HTTP/2, etc. C. Timmerer, M. Maeiro, B. Rainer, “Which Adaption Logic? An Objective and Subjective Performance Evaluation of HTTP-basedAdaptiveMedia Streaming Systems”, arXiv cs.MM, June 2016, http://arxiv.org/abs/1606.00341.
25. June 2016 QoMEX 2016 Tutorial 25
26. June 2016 QoMEX 2016 Tutorial 26
27. DASH-JS vs. bitdash June 2016 QoMEX 2016 Tutorial 27 C. Timmerer, D. Weinberger, C. Mueller, and S. Lederer, “Ultra-High-Definition-Quality of Experience with MPEG-DASH”, Proceedings of the Broadcast EngineeringConference (BEC), NAB2015, Las Vegas, NV, USA, April 2015.
28. Objective Evaluations June 2016 QoMEX 2016 Tutorial 28 Stalls (lower is better)Average Bitrate (higher is better)
29. Stalls are really bad… June 2016 QoMEX 2016 Tutorial 29 Conviva: Viewer Experience Report. 2014
30. Conclusions (1) • MPEG-DASH defines formats only – Media Presentation Description (MPD) – Segment format: mp4, ts • MPEG-DASH is not – System, protocol, presentation, codec, interactivity, DRM, client specification – Other standards required for a complete ecosystem: e.g., DASH-IF, WAVE, HMTL5, MSE, EME • Do we need MPEG-DASH? (for adaptive media streaming) – Not necessarily: e.g., WebM + VPx + manifest & control end-to-end – Required to address heterogeneous environments • Role of standards sometimes overrated but often underestimated June 2016 QoMEX 2016 Tutorial 30
31. Conclusions (2) • QoE for DASH-based services (a rule of thumb) – Startup delay (low [but live vs. on-demand & short vs. long-tail content]) – Buffer underrun / stalls (zero) – Quality switches (low) and media throughput (high) – Energy- and cost-awareness (data plan) • No general applicable QoE model for DASH – (Too) many factors influencing / features of QoE for DASH-based services – Methodology for reproducible research is in place and well established – Ample research opportunities June 2016 QoMEX 2016 Tutorial 31 Main QoE factors for DASH ➡ Come up with our own QoE factor and design, conduct, analyze a small-scale experiment
The role of adaptive streaming standardization activities in ... Media Platform Architect Microsoft The role of ... an adaptive streaming standard.
Adaptive streaming technologies share several critical aspects. First, they produce multiple files from the same source file to distribute to viewers ...
The role of adaptive streaming ... definingbroadband television standards Broadband Television ... adaptive streaming Addresscombinatorial ...
... issue of Streaming Media magazine ... groups working on dynamic adaptive streaming was a double ... independent streaming standard is ...
Adaptive bitrate streaming is a technique used ... HTTP Live Streaming is a standard feature in ... that enables adaptive streaming of media to ...
Understanding MPEG-DASH for Adaptive Streaming ... on the MPEG Dynamic Adaptive Streaming over ... how standards are implemented in a media ...
Dynamic Adaptive Streaming over ... HTTP-based streaming solution that is an international standard. ... of major streaming and media ...
of the essential role a media player ... also comes standard with the plug-ins required for ... companies are leveraging adaptive streaming ...
... finally ratified the version of its next generation adaptive streaming standard: ... streaming media ... Encrypted Media Extensions standards ...
Dynamic Adaptive Streaming over HTTP – Design Principles and Standards Thomas Stockhammer ... media delivery from standard HTTP Web servers.