by Klaus Lüthje and Filip Šuba

Size: px

Start display at page:

Download "by Klaus Lüthje and Filip Šuba"

Millicent White
6 years ago
Views:

1 by Klaus Lüthje and Filip Šuba

2 What is GStreamer? Open source multimedia framework Based on: C language, uses GObject to implement objects GLib library LGPL licence GUI independent Usable on servers, desktops and embedded systems

3 History Based on a research project at Portland University Loosely modeled on DirectShow First focus on embedded devices Integrated in the GNOME desktop since 2.2

4 Supported OS Enviroments Primarily aimed at UNIX like enviroments Competes with xine, helix, other frameworks Special integration functions with GNOME Volume, audio output etc. Mac OS X Can be used on Windows Uses cygwin

5 Graph system Pipelines handle everything Source => Sink connections between modules Sink Source Sink MPEG Video decoder X Video output Filesource Source Sink Video MPEG Demux Audio Sink Source MP3 Decoder Sink OSS Audio output Simplified Graph of the playback of a MPEG video file

6 XML Support Pipelines can be saved as XML-format files that are easy to load in code Example: int main(int argc, char *argv[]) { GstXML *xml; GstElement *bin; gboolean ret; gst_init(&argc, &argv); xml = gst_xml_new(); ret = gst_xml_parse_file(xml, "xmltest.gst", NULL); g_assert(ret == TRUE); bin = gst_xml_get_element(xml, "bin"); g_assert(bin!= NULL); gst_element_set_state(bin, GST_STATE_PLAYING); while(gst_bin_iterate(gst_bin(bin))); gst_element_set_state(bin, GST_STATE_NULL); exit(0); }

7 MIME Support Plug-ins detect data types with MIME and type specific properties OGG Vorbis plug-in: Sink with audio/x-vorbis format Source with audio/x-raw-float format and a rate of 8-48kHz Build pipelines automatically based on data types

8 Other Features Small core library, 150KB Lightweight data passing Built-in threads Clocking for synchronization

9 Debugging gst-inspect for information on plug-ins OGG Vorbis plug-in output: Pad Templates: SRC template: 'src' Availability: Always Capabilities: audio/x-raw-float rate: [ 8000, ] channels: [ 1, 2 ] endianness: 1234 width: 32 buffer-frames: 0 SINK template: 'sink' Availability: Always Capabilities: audio/x-vorbis GST_DEBUG enviroment variable for configuring debug info level on different categories. Written to stderr.

10 Language Bindings Released: Python Perl Guile Ruby Under Development:.NET C++ Java

11 Plug-ins Dynamically loaded on demand No need to recompile software when a new plug-in is installed Core/Base plug-ins Additional plug-ins separated into three modules: Good: Stable with free licence Ugly: Stable with other licences Bad: Unstable API for plug-in development

12 Plug-ins - Base ALSA audio output X and Xv video output GnomeVFS URI file input CD, OGG Vorbis audio Theora video Basic audio/video conversion and filtering tools: Volume, bitrate, colorspace etc.

13 Plug-ins - Good Formats: AVI, DV video/audio, FLAC, JPEG, PNG, Speex, WAV, Matroska, id3 tags, etc.. GOOM audio visualization HAL audio output and input shout/icecast streams OSS Audio UDP, RTP, RTSP streams

14 Plug-ins - Ugly MP3 encoding with LAME library ID3 tags and MP3 decoding with mad library MPEG stream parsing RealMedia demuxing SID file support

15 Plug-ins - Bad DirectFB, OpenGL and SDL video output Formats: AAC, cdxa, GSM, mods, musepack, quicktime, tta, h264 alsa S/PDIF output bz2 compression FFT on audio signals Some video and audio effects

16 Application types Media players Video Editors Streaming media broadcasting

17 Application Examples Embedded: Nokia 770 multimedia support Server: Icecast server Desktop: Totem video player

18 Prototyping gst-launch Commandline pipeline building Enables fast prototyping Examples: gst-launch filesrc location=thesong.mp3! mad! audioconvert! audioresample! osssink gst-launch filesrc location=my-random-media-file.mpeg! decodebin name=decode decoder.! ffmpegcolorspace! xvimagesink decoder.! audioconvert! audioresample! osssink

19 Prototyping gst-editor Graphical tool for pipeline building XML load/save Capable of running the pipelines Supports gst-launch syntax prototyping Integrated with GStreamer s debugging subsystem

20 gst-editor - Screenshot

21 Code Example #include <gst/gst.h> /* * Global objects are usually a bad thing. For the purpose of this * example, we will use them, however. */ GstElement *pipeline, *source, *parser, *decoder, *conv, *sink; static gboolean bus_call (GstBus *bus, GstMessage *msg, gpointer data) { GMainLoop *loop = data; switch (GST_MESSAGE_TYPE (msg)) { case GST_MESSAGE_EOS: g_print ("End-of-stream\n"); g_main_loop_quit (loop); break; case GST_MESSAGE_ERROR: { gchar *debug; GError *err; } } gst_message_parse_error (msg, &err, &debug); g_free (debug); g_print ("Error: %s\n", err->message); g_error_free (err); g_main_loop_quit (loop); break; } default: break; return TRUE; static void new_pad (GstElement *element, GstPad *pad, gpointer data) { GstPad *sinkpad; /* We can now link this pad with the audio decoder */ g_print ("Dynamic pad created, linking parser/decoder\n"); sinkpad = gst_element_get_pad (decoder, "sink"); gst_pad_link (pad, sinkpad); gst_object_unref (sinkpad); }

22 int main (int argc, char *argv[]) { GMainLoop *loop; GstBus *bus; /* initialize GStreamer */ gst_init (&argc, &argv); loop = g_main_loop_new (NULL, FALSE); /* check input arguments */ if (argc!= 2) { g_print ("Usage: %s <Ogg/Vorbis filename>\n", argv[0]); return -1; } /* create elements */ pipeline = gst_pipeline_new ("audio-player"); source = gst_element_factory_make ("filesrc", "file-source"); parser = gst_element_factory_make ("oggdemux", "ogg-parser"); decoder = gst_element_factory_make ("vorbisdec", "vorbisdecoder"); conv = gst_element_factory_make ("audioconvert", "converter"); sink = gst_element_factory_make ("alsasink", "alsa-output"); if (!pipeline!source!parser!decoder!conv!sink) { g_print ("One element could not be created\n"); return -1; } /* set filename property on the file source. Also add a message * handler. */ g_object_set (G_OBJECT (source), "location", argv[1], NULL); bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline)); gst_bus_add_watch (bus, bus_call, loop); gst_object_unref (bus); /* put all elements in a bin */ gst_bin_add_many (GST_BIN (pipeline), source, parser, decoder, conv, sink, NULL); /* link together - note that we cannot link the parser and * decoder yet, becuse the parser uses dynamic pads. For that, * we set a pad-added signal handler. */ gst_element_link (source, parser); gst_element_link_many (decoder, conv, sink, NULL); g_signal_connect (parser, "pad-added", G_CALLBACK (new_pad), NULL); /* Now set to playing and iterate. */ g_print ("Setting to PLAYING\n"); gst_element_set_state (pipeline, GST_STATE_PLAYING); g_print ("Running\n"); g_main_loop_run (loop); /* clean up nicely */ g_print ("Returned, stopping playback\n"); gst_element_set_state (pipeline, GST_STATE_NULL); g_print ("Deleting pipeline\n"); gst_object_unref (GST_OBJECT (pipeline)); return 0; }

23 Code Example - Pipeline

24 Future Plans Core will not change much MIDI subsystem More subtitle formats for video Interactivity support for DVD menus, Flash animations etc. Proposed to be integrated into KDE 4.0 A touchy topic - might not happen

25 Web Links Official site - Nokia GObject reference

GStreamer Application Development Manual ( ) Wim Taymans Steve Baker Andy Wingo Ronald S. Bultje

GStreamer Application Development Manual (0.10.1.2) Wim Taymans Steve Baker Andy Wingo Ronald S. Bultje GStreamer Application Development Manual (0.10.1.2) by Wim Taymans, Steve Baker, Andy Wingo, and