Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

33 %
67 %
Information about Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

Published on February 5, 2014

Author: igalia



By Guillaume Emont.

GStreamer 1.0 introduced a new architecture that makes memory management easier. In particular, handling non-system memory and sharing it between elements is made easier.

While building a video sink for cairogles, Guillaume wanted to explore the possibility of sharing GPU memory (in his case, GL or GLES textures) between elements of a pipeline. In this talk, Guillaume will explain how he made the various parts fit together, and discuss the challenges that he has met and how he has worked on them.

Cairosink, GPU Buffer Sharing ...and other random ramblings Guillaume Emont - Igalia

ongoing experiment, I may not have the best answers yet

Work sponsored by Samsung Research America

I. Technologies, past and present

Cairo: 2D graphics Cairogles: 2D graphics in OpenGL(ES)

support for EGL/GLX/WGL

Sharing video buffers: before - Add a specific cap (e.g. video/x-raw-gl) - subclass GstBuffer

Sharing video buffers: nowadays - GstContext - GstVideoGLTextureUploadMeta - GstAllocator and GstMemory

- GstContext, system to share a GL context and more between elements - GstVideoGLTextureUploadMeta: a way to tell how to upload a buffer to GPU - GstAllocator and GstMemory: cleaner way to handle non-CPU memory

Sharing video buffers: nowadays "modern" gl(es) sink implementations known to the author: - eglglessink (and libgstegl) - glimagesink

II. Integrating cairogles and GStreamer

Many possibilities: - xvimagesink + GstVideoOverlay - eglglessink - glimagesink - make our own sink

xvimagesink with overlay: not so fast, I suspect it triggers a download to CPU to get the image from xvimagesink's Drawable to a cairo-gl surface. eglglessink: promising, but obviously doesn't support openGL or GLX glimagesink started work on proper integration, likely will require addition of a few features there

First attempt: a cairosink that does everything

Cairosink: status - GstAllocator and GstMemory subclasses - Use either OpenGL or OpenGLES - Possibility to use PixelBuffer Objects (PBOs) (inefficiently)

allocator and memory not that useful as it's the only element using it. Main advantage: _could_ allow to start uploading the frame as soon as possible. PBOs in a nutshell: a way to asynchronously upload a frame to a texture. The upload is then done through the magic of DMA, without the CPU being involved. More on why cairosink is currently inefficient at using it later. FIXME: graph explaining PBO?

Cairosink: simplistic benchmark

Synchronous / PBO / XV XV: limited by refresh rate? Looks like its genuine limit (often below 60) frames 720x400, RGB or YUV

Cairosink: difficulties (or rants) - cairogles can be compiled with either gl or glesv2 (not both) - driver bugs and regressions - cairo only supports RGB - desktop vs mobile GPUs

driver regression: GL+EGL used to work on my machine...

Desktop vs mobile GPU - EGLMakeCurrent() is costly on many mobile GPUs/drivers - bandwidth differences

context acquisition (makeCurrent) slowness is likely to counter the effects of asynchronous direct upload

Cairosink: what's to do - more flexibility on code path - compatibility with other elements (not our own memory/allocator)

we want to be able to tailor to various GPUs various "options": is it faster to do a synchronous or asynchronous upload?

Cairosink: other ideas to explore - libgstgl backend (ongoing experiment) - wrapper around glimagesink

libgstgl issues: mechanism to MakeCurrent() or save/restore states share GMainContext? (all in one thread)

Thank you Questions?

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Cairosink and GPU Buffer Sharing (GStreamer Conference ...

By Guillaume Emont. Slides at GStreamer 1.0 introduced a new architecture that makes memory management ...
Read more

Cairosink and GPU Buffer Sharing (GStreamer Conference ...

Slides at GStreamer 1.0 introduced a new ... Cairosink and GPU Buffer Sharing (GStreamer Conference 2013 ...
Read more

Cairosink and GPU Buffer Sharing -

Speakers bio and abstracts - GStreamer Conference 2013 Edinburgh, Scotland, UK, 22-23 October 2013. Back to conference main page. Back to conference timetable
Read more

GStreamer Conference 2013 Schedule

GStreamer Conference 2013 Edinburgh, ... Cairosink and GPU Buffer Sharing Guillaume ... GStreamer and the Move From a Hardware Centric Product Design ...
Read more

Cairosinki and GPU Buffer Sharing - GStreamer conferences

Permalink. Tweet about this video. Share on LinkedIn. Share on Facebook. Share this video by mail
Read more

GStreamer Conference 2013 - My GStreamer-Enabled ... - YouTube

GStreamer Conference 2013 ... Cairosink and GPU Buffer Sharing (GStreamer Conference 2013) ... Program, Gstreamer and Ubuntu on YouTube Live ...
Read more

Multimedia on Vimeo

... GStreamer Conference 2015) 8 months ago. Cairosink and GPU Buffer Sharing (GStreamer Conference 2013) 2 years ago. WebKit and GStreamer (GStreamer ...
Read more

Multimedia - Igalia: Open Source Consultancy

WebKit Multimedia; GStreamer; Grilo; ... WebKit and GStreamer (GStreamer Conference 2013) ... Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)
Read more

Gstreamer | LinkedIn

View 3579 Gstreamer posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn. LinkedIn Home What is LinkedIn? Join Today
Read more