Audio And Video Over Internet

Audio and Video over Internet Sudarsun S., M.Tech Checktronix India Pvt Ltd Chennai 600034 [email_address]

Introduction What is Audio/Video over Internet? What are the benefits of AV on Internet? Cost of traditional communication Availability/Mobility with Wireless Entertainment Industry On demand Movie, Songs Duplication Protection Distance does not matter! Decentralization What is and Why Streaming data? Real-time Communication Protocols

What is Audio/Video over Internet?

What are the benefits of AV on Internet?

Cost of traditional communication

Availability/Mobility with Wireless

Entertainment Industry

On demand Movie, Songs

Duplication Protection

Distance does not matter!

Decentralization

What is and Why Streaming data?

Real-time Communication Protocols

Agenda Learn the fundamentals of streaming TCP, UDP and their limitations RTP, RTSP to overcome TCP limitations Error Correction and Detection Low Quality Piggybacking Interleaving Understand “On demand” multimedia data On demand cinema Video conferencing

Learn the fundamentals of streaming

TCP, UDP and their limitations

RTP, RTSP to overcome TCP limitations

Error Correction and Detection

Low Quality Piggybacking

Interleaving

Understand “On demand” multimedia data

On demand cinema

Video conferencing

Overview The big picture

The big picture

Media Data types Audio : WAV, MP3, RM Video: AVI, MPG, MOV, RM Bandwidth calculation 3GPP, 3GPP2 3rd generation, high-speed wireless networks MPEG4 Professional-quality audio and video streams H264 formats MPEG-4 standard (Part 10) used across 3G for mobile devices to iChat AV for video conferencing to HD for broadcast and DVD

Audio : WAV, MP3, RM

Video: AVI, MPG, MOV, RM

Bandwidth calculation

3GPP, 3GPP2

3rd generation, high-speed wireless networks

MPEG4

Professional-quality audio and video streams

H264 formats

MPEG-4 standard (Part 10) used across 3G for mobile devices to iChat AV for video conferencing to HD for broadcast and DVD

TCP & UDP Connection Oriented – Is that a Overhead ? Connectionless – Will that guarantee timely delivery? Maximum Effort Reliability vs Just In Time with permissible error rate Packet Sequence – Is that an Advantage ? Summary of general deficiencies

Connection Oriented – Is that a Overhead ?

Connectionless – Will that guarantee timely delivery?

Maximum Effort Reliability vs Just In Time with permissible error rate

Packet Sequence – Is that an Advantage ?

Summary of general deficiencies

RTP (RFC 3550) Real-time Transport Protocol on UDP Uses RTCP (special set of messages) to exchange periodic reports One RTP session per media flow 16 bit sequence numbers to detect packet loss 32 bit time stamp to handle jitter due to network

Real-time Transport Protocol on UDP

Uses RTCP (special set of messages) to exchange periodic reports

One RTP session per media flow

16 bit sequence numbers to detect packet loss

32 bit time stamp to handle jitter due to network

RTP Format (V) Version; 2 bits (P) Padding; 1 bit. (X) Extension; 1 bit. (CC) CSRC Count; 4 bits. (M) Marker; 1 bit. (PT) Payload Type; 7 bits. Sequence Number ; 16 bits. Time Stamp ; 32 bits. SyncSRCId ; 32 bits. ContributingSRCId List ;

How RTP Works? IP UDP RTP HDR Audio Payload IP UDP RTP HDR Video Payload Uses separate sessions for audio and video Uses sequence number to synchronize audio/video

Uses separate sessions for audio and video

Uses sequence number to synchronize audio/video

RTP Vs OSI Data Link Frame Relay Ethernet Network IP IP UDP UDP Transport RTP RTP Application MPEG2 video PCMA audio

RTCP Supervise QoS Reception reports, Sender reports, Source description Reports: packets sent/lost, inter-arrival jitter Helps modify sender transmission rate and for diagnostics purpose RTCP bandwidth = 5% of session BW Commands: SR, RR, SrcDESc, BYE, APP

Supervise QoS

Reception reports, Sender reports, Source description

Reports: packets sent/lost, inter-arrival jitter

Helps modify sender transmission rate and for diagnostics purpose

RTCP bandwidth = 5% of session BW

Commands: SR, RR, SrcDESc, BYE, APP

Continuous Media Real-time (Video Conferences) Interactive Tight timing relationship Error rate/Jitter to be near Zero Streaming Playback stored media Error rate need not be very low Loose timing relationship

Real-time (Video Conferences)

Interactive

Tight timing relationship

Error rate/Jitter to be near Zero

Streaming

Playback stored media

Error rate need not be very low

Loose timing relationship

RTSP (RFC 2326) Application-level protocol for control over the delivery of data with real-time properties. Works on top of HTTP Extensible framework to enable controlled, on-demand delivery of real-time data. Supports live data feeds and stored clips Controls multiple data delivery sessions (UDP, multicast UDP, TCP). Delivery mechanisms based on RTP.

Application-level protocol for control over the delivery of data with real-time properties.

Works on top of HTTP

Extensible framework to enable controlled, on-demand delivery of real-time data.

Supports live data feeds and stored clips

Controls multiple data delivery sessions (UDP, multicast UDP, TCP).

Delivery mechanisms based on RTP.

 

RTSP Methods OPTIONS- obtain available methods SETUP- Establish transport ANNOUNCE- alter description of media object. DESCRIBE- get low level description of media object. PLAY- start playback. RECORD- Start recording PAUSE- suspend delivery, keep state SET_PARAMETER- device or encoding control TEARDOWN- remove state.

OPTIONS- obtain available methods

SETUP- Establish transport

ANNOUNCE- alter description of media object.

DESCRIBE- get low level description of media object.

PLAY- start playback.

RECORD- Start recording

PAUSE- suspend delivery, keep state

SET_PARAMETER- device or encoding control

TEARDOWN- remove state.

Error Correction Piggy backing (FEC) Interleaving Combine both

Piggy backing (FEC)

Interleaving

Combine both

Low Quality Piggybacking Works for single packet loss, How about multi-packet loss ?

Interleaving Data blackout is recovered with intermittent jitter

Network Strategies Multicasting reduces amount of redundant data transferred Streaming media delivery networks allow for multiple, geographically scattered servers (achieved either by splitting or caching ) Dedicated networks guarantee bandwidth Multiple-access transmissions over Internet—requested info sent from several different locations simultaneously

Multicasting reduces amount of redundant data transferred

Streaming media delivery networks allow for multiple, geographically scattered servers (achieved either by splitting or caching )

Dedicated networks guarantee bandwidth

Multiple-access transmissions over Internet—requested info sent from several different locations simultaneously

Streaming Media Three modes in which video information can be streamed: Live broadcasting On-demand streaming Conferencing

Three modes in which video information can be streamed:

Live broadcasting

On-demand streaming

Conferencing

Live Broadcasting System SLTA: Simulated Live Transfer Agent (Rebroadcasting)

On-demand System Structure

Streaming Media – On demand Distribution system similar to live broadcasting system, except: No direct connection between encoder and server Remote proxy servers can cache popular files Clients allowed to rewind, fast forward, etc. Server proxy transfers only initiated by client

Distribution system similar to live broadcasting system, except:

No direct connection between encoder and server

Remote proxy servers can cache popular files

Clients allowed to rewind, fast forward, etc.

Server proxy transfers only initiated by client

Where to Get More Information http://www.mediacollege.com/video/streaming/overview.html http://www.apple.com/quicktime/technologies/ http://www.teamsolutions.co.uk/streaming.html http://archive.dstc.edu.au/RDU/staff/jane-hunter/video-streaming.html http://www2.umist.ac.uk/isd/lwt/clickgo/the_guide/the_guide.htm http://emoney.al.ru/capture-streaming-video-and-audio/ http://www.cit.cornell.edu/atc/itsupport/streaming.shtml http://www.yolinux.com/TUTORIALS/LinuxTutorialRealVideoStreaming.html http://nms.csail.mit.edu/projects/oxygentv/

http://www.mediacollege.com/video/streaming/overview.html

http://www.apple.com/quicktime/technologies/

http://www.teamsolutions.co.uk/streaming.html

http://archive.dstc.edu.au/RDU/staff/jane-hunter/video-streaming.html

http://www2.umist.ac.uk/isd/lwt/clickgo/the_guide/the_guide.htm

http://emoney.al.ru/capture-streaming-video-and-audio/

http://www.cit.cornell.edu/atc/itsupport/streaming.shtml

http://www.yolinux.com/TUTORIALS/LinuxTutorialRealVideoStreaming.html

http://nms.csail.mit.edu/projects/oxygentv/

Limitations Bandwidth limitation Congestion and Quality of network Buffer Overhead in Portable devices Compression and Decompression overhead Security issues Eavesdropping Duplication by Data Augmentation

Bandwidth limitation

Congestion and Quality of network

Buffer Overhead in Portable devices

Compression and Decompression overhead

Security issues

Eavesdropping

Duplication by Data Augmentation

Some Servers Audio Winamp’s Shoutcast Server Streamsicle Broadcast server RealMedia Stream server Video Quicktime Server Helix universal server Macromedia communication server Ffmpeg ( http:// sourceforge .net/projects/ ffmpeg / )

Audio

Winamp’s Shoutcast Server

Streamsicle Broadcast server

RealMedia Stream server

Video

Quicktime Server

Helix universal server

Macromedia communication server

Ffmpeg ( http:// sourceforge .net/projects/ ffmpeg / )

Conclusion Streaming multimedia poses many challenges to video encoders Variability of channel performance High server workloads Need to adapt coded bit rate to match channel capacities; must be done efficiently and on the fly

Streaming multimedia poses many challenges to video encoders

Variability of channel performance

High server workloads

Need to adapt coded bit rate to match channel capacities; must be done efficiently and on the fly

Thank You Sudarsun S., M.Tech Director – Research and Development Checktronix India Pvt Ltd [email_address] November 2005