How to write a GStreamer Source Element in C
A 10-Step guide culminating with creating a flip book video with your code
How does it sound to be able to write your own GStreamer Element? It’s not a daunting task. It just takes you to know which functions do what in which GStreamer classes. In the last tutorial we learned the basics of how to use a GStreamer pipeline in C and Python. In this tutorial we will write a source element. The only assumption made your are familiar with writing C code, especially GStreamer code.
Getting Started With GStreamer: A beginner's guide
I will outline the steps in the 10 Step process to write a GStreamer Source Element starting with gst-element-maker from gst-plugins-bad. The element we will write will insert jpeg/jpg files from a directory into the pipeline. We will demonstrate a flip book (each picture will appear on the screen for 1 second) with jpeg/jpg files and in the end create a video with the pictures.
The 10-Step Process:
Get the sources
Generate the source element from the template
Delete unused functions
Search and Replace to change from GstBaseSrc to GstPushSrc
Correct include file
Set the Capabilities
Describe the functioning of the element
Add a Property to take input from the user, configure our element
Write the code for the init, start, fill, dispose function
Compile and Test
Let’s begin!
Step 1: Get the sources
Let’s start by downloading all the necessary sources. We need gst-element-maker which is in the gst-plugins-bad tarball. I downloaded version 1.20.4 here https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.20.4.tar.xz . Next, clone my github repo writings, it contains the source code for the source element in the source_element directory
#Download and unpack gst-plugins-bad
wget https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.20.4.tar.xz
tar Jxvf gst-plugins-bad-1.20.4.tar.xz
#CLone the repository containing the source element
git clone https://www.github.com/mndar/writings
Step 2: Generate the source element from the template
GStreamer elements are written by sub-classing a base class and over riding functions in them. To write a source element, you can either use GstBaseSrc or GstPushSrc. GStPushSrc is a actually a subclass of GstBaseSrc. This is the one we will use. However, there is no template for GstPushSrc. So, let’s start with GstBaseSrc and then edit manually.
cd gst-plugins-bad/tools
./gst-element-maker my_jpeg_streamer basesrc
This will generate the source code for the element and compile it. It will also show you the output of gst-inspect-1.0. The source code files generated will be gstmyjpegstreamer.c and gstmyjpegstremaer.h. The compile command for the element is
gcc -Wall -Werror -fPIC $CPPFLAGS `pkg-config --cflags gstreamer-1.0 gstreamer-base-1.0` -c -o gstmyjpegstreamer.o gstmyjpegstreamer.c
gcc -shared -o gstmyjpegstreamer.so gstmyjpegstreamer.o `pkg-config --libs gstreamer-1.0 gstreamer-base-1.0`
Step 3: Delete unused functions
The way to write an element in GStreamer is to over ride functions in the base class. So, open up gstmyjpegstreamer.c and keep only the over rides we are going to use. The function to edit is gst_my_jpeg_streamer_class_init
static void
gst_my_jpeg_streamer_class_init (GstMyJpegStreamerClass * klass)
{
...
...
GstBaseSrcClass *base_src_class = GST_BASE_SRC_CLASS (klass);
...
...
gobject_class->set_property = gst_my_jpeg_streamer_set_property;
gobject_class->get_property = gst_my_jpeg_streamer_get_property;
base_src_class->start = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_start);
push_src_class->fill = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_fill);
gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_dispose);
...
...
}
Step 4: Search and Replace to change from GstBaseSrc to GstPushSrc
Since we have generated the code with gst-element-maker using the basesrc class, short for GstBaseSrc, we have the do a case-sensitive search and replace of the following terms
Repace GstBaseSrc with GstPushSrc in the implementation file gstmyjpegstreamer.c
Replace base_src with push_src in the implementation file gstmyjpegstreamer.c
Replace base with push in the header file gstmyjpegstreamer.h
Step 5: Correct the include file
Change gstbasesrc.h to gstpushsrc.h since we are sub-classing GstPushSrc and not GstBaseSrc.
...
#include <gst/base/gstpushsrc.h>
...
Step 6: Set the Capabilities
Since this is a source element, it has just one pad. We set it’s capabilities to image/jpeg as we will be inserting jpeg files into the pipeline.
/* pad templates */
static GstStaticPadTemplate gst_my_jpeg_streamer_src_template =
GST_STATIC_PAD_TEMPLATE ("src",
GST_PAD_SRC,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("image/jpeg")
);
Step 7: Describe the functioning of the element
The element will read a property from the user, the directory, scan the directory for all jpeg/jpg files, make a list and send these files one at a time over the output pad of the source element.
To do so, we will add a property called directory, initialize variables in the init function, call functions that need to be executed only at the begininig in start, call functions that fill the GstBuffer in fill and call all functions that clean up memory in dispose. Here is a snippet of what we will use
base_src_class->start = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_start);
push_src_class->fill = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_fill);
gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_dispose);
Step 8: Add a Property to take input from the user, configure our element
We use a property called directory to get input from the user. The images from this directory will be used by our element. Make sure all the images of the same resolution. As we are making a flip book, we need all images of the same size.
We add the property to our element in class_init and implement set and get in set_property and get_property respectively.
enum
{
PROP_0,
PROP_DIRECTORY
};
...
...
static void
gst_my_jpeg_streamer_class_init (GstMyJpegStreamerClass * klass)
{
...
...
g_object_class_install_property
(gobject_class, PROP_DIRECTORY,
g_param_spec_string ("directory", "Directory",
"Value of the Directory containing JPEG files",
"myjpegstreamer", G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
}
...
...
void
gst_my_jpeg_streamer_set_property (GObject * object, guint property_id,
const GValue * value, GParamSpec * pspec)
{
...
...
case PROP_DIRECTORY:
const gchar *directory;
directory = g_value_get_string (value);
myjpegstreamer->directory = g_strdup (directory);
g_print ("Setting JPEG Directory to %s\n", myjpegstreamer->directory);
break;
...
...
}
void
gst_my_jpeg_streamer_get_property (GObject * object, guint property_id,
GValue * value, GParamSpec * pspec)
{
...
switch (property_id) {
case PROP_DIRECTORY:
g_value_set_string (value, myjpegstreamer->directory);
break;
...
...
}
Step 9: Write the code for the init, start, fill, dispose function
The init function initializes all the variables we will be using. start is called once when the element starts, dispose is called when the element is destroyed and fill is called everything the GstBuffer needs to be filled.
The snippets of code are given below. In start, we scan the directory for all jpeg/jpg files. In fill we read part of the file and fill GstBuffer to its size.
The important part to remember is how to fill the GstBuffer. All you to do is call gst_buffer_map containing GstMapInfo. GstMapInfo.data points to the data inside the buffer. We have to fill the entire buffer. To get the size of the buffer, use gst_buffer_get_size. Once you are done, unmap the buffer using gst_buffer_unmap. Incase you are not able to fill the GstBuffer, use gst_buffer_resize to resize it to a smaller size.
//all operations on GstBuffer
...
gst_buffer_map (buf, &info, GST_MAP_WRITE);
buf_size = gst_buffer_get_size (buf);
...
...
count = fread (info.data, 1, buf_size, src->curr_file);
...
...
gst_buffer_resize (buf, 0, count);
...
...
gst_buffer_unmap (buf, &info);
// In gstmyjpegstreamer.h
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
...
...
struct _GstMyJpegStreamer
{
GstPushSrc push_myjpegstreamer;
gchar *directory;
GList *jpeg_files;
FILE *curr_file;
guint file_index;
guint num_files;
};
// In gstmyjpegstremaer.c
static void
gst_my_jpeg_streamer_class_init (GstMyJpegStreamerClass * klass)
{
...
...
base_src_class->start = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_start);
push_src_class->fill = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_fill);
gobject_class->dispose = GST_DEBUG_FUNCPTR (gst_my_jpeg_streamer_dispose);
...
...
}
static void
gst_my_jpeg_streamer_init (GstMyJpegStreamer *myjpegstreamer)
{
myjpegstreamer->directory = NULL;
myjpegstreamer->jpeg_files = NULL;
myjpegstreamer->curr_file = NULL;
myjpegstreamer->file_index = 0;
}
static gboolean
gst_my_jpeg_streamer_start (GstBaseSrc * src)
{
GstMyJpegStreamer *myjpegstreamer = GST_MY_JPEG_STREAMER (src);
GST_DEBUG_OBJECT (myjpegstreamer, "start");
GDir *dir;
GError *error = NULL;
const gchar *file;
dir = g_dir_open(myjpegstreamer->directory, 0, &error);
while ((file = g_dir_read_name(dir))) {
if (g_str_has_suffix (file, ".jpeg") || g_str_has_suffix (file, "jpg")) {
g_print("%s\n", file);
myjpegstreamer->jpeg_files =
g_list_append (myjpegstreamer->jpeg_files, g_strdup (file));
}
}
g_dir_close (dir);
myjpegstreamer->num_files = g_list_length (myjpegstreamer->jpeg_files);
return TRUE;
}
gboolean
stream_list_file (GstMyJpegStreamer *src, gchar *directory, gchar *filename, GstBuffer *buf) {
g_print("Copying File %s To Buffer\n", filename);
long count;
gsize buf_size;
gchar *file_path;
GstMapInfo info;
if (!src->curr_file) {
file_path = g_strdup_printf ("%s/%s", directory, filename);
src->curr_file = fopen (file_path, "rb");
g_free (file_path);
}
gst_buffer_map (buf, &info, GST_MAP_WRITE);
buf_size = gst_buffer_get_size (buf);
count = fread (info.data, 1, buf_size, src->curr_file);
g_print ("File Index: %d Count: %ld, Buf Size: %ld\n", src->file_index, count, buf_size);
if (count < buf_size) {
fclose (src->curr_file);
src->curr_file = NULL;
gst_buffer_resize (buf, 0, count);
gst_buffer_unmap (buf, &info);
return FALSE;
}
gst_buffer_unmap (buf, &info);
return TRUE;
}
static GstFlowReturn
gst_my_jpeg_streamer_fill (GstPushSrc * src, GstBuffer * buf)
{
GstMyJpegStreamer *myjpegstreamer = GST_MY_JPEG_STREAMER (src);
GList *file;
gchar *filename;
gboolean ret;
GST_DEBUG_OBJECT (myjpegstreamer, "fill");
if (myjpegstreamer->num_files == 0)
return GST_FLOW_EOS;
if (myjpegstreamer->file_index > myjpegstreamer->num_files - 1) {
return GST_FLOW_EOS;
}
file = g_list_nth (myjpegstreamer->jpeg_files, myjpegstreamer->file_index);
filename = (gchar *) file->data;
g_print ("Streaming File %s\n", filename);
ret = stream_list_file (myjpegstreamer, myjpegstreamer->directory, filename, buf);
if (!ret)
myjpegstreamer->file_index++;
return GST_FLOW_OK;
}
void free_list_string (gpointer data) {
g_print("Freeing List Data\n");
g_free (data);
}
static void
gst_my_jpeg_streamer_dispose (GObject *gobject) {
GstPushSrc *src = GST_PUSH_SRC (gobject);
GstMyJpegStreamer *myjpegstreamer = GST_MY_JPEG_STREAMER (src);
g_free (myjpegstreamer->directory);
g_list_free_full (myjpegstreamer->jpeg_files, free_list_string);
G_OBJECT_CLASS (gst_my_jpeg_streamer_parent_class)->dispose (gobject);
}
Step 10. Compile and Test
To compile our code, run the following commands.
gcc -Wall -Werror -fPIC $CPPFLAGS `pkg-config --cflags gstreamer-1.0 gstreamer-base-1.0` -c -o gstmyjpegstreamer.o gstmyjpegstreamer.c
gcc -shared -o gstmyjpegstreamer.so gstmyjpegstreamer.o `pkg-config --libs gstreamer-1.0 gstreamer-base-1.0`
# Check the element with
GST_PLUGIN_PATH=. gst-inspect-1.0 myjpegstreamer
The output of the gst-inspect-1.0 command should be
$ GST_PLUGIN_PATH=. gst-inspect-1.0 myjpegstreamer|cat
Factory Details:
Rank none (0)
Long-name FIXME Long name
Klass Generic
Description FIXME Description
Author FIXME <fixme@example.com>
Plugin Details:
Name myjpegstreamer
Description FIXME plugin description
Filename ./gstmyjpegstreamer.so
Version 0.0.FIXME
License LGPL
Source module FIXME_package
Binary package FIXME_package_name
Origin URL http://FIXME.org/
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSrc
+----GstPushSrc
+----GstMyJpegStreamer
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
image/jpeg
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SRC: 'src'
Pad Template: 'src'
Element Properties:
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
directory : Value of the Directory containing JPEG files
flags: readable, writable
String. Default: null
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "myjpegstreamer0"
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false
Let’s generate few test images of size 1280x720 which we will use in the flip book
# Generate test images of size 1280x720
mkdir generated
cd generated
gst-launch-1.0 videotestsrc pattern=0 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=1.jpg
gst-launch-1.0 videotestsrc pattern=1 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=2.jpg
gst-launch-1.0 videotestsrc pattern=2 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=3.jpg
gst-launch-1.0 videotestsrc pattern=3 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=4.jpg
gst-launch-1.0 videotestsrc pattern=4 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=5.jpg
gst-launch-1.0 videotestsrc pattern=5 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=6.jpg
gst-launch-1.0 videotestsrc pattern=6 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=7.jpg
gst-launch-1.0 videotestsrc pattern=7 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=8.jpg
gst-launch-1.0 videotestsrc pattern=8 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=9.jpg
gst-launch-1.0 videotestsrc pattern=9 num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=10.jpg
To view the flip book, use the following pipeline
GST_PLUGIN_PATH=. gst-launch-1.0 myjpegstreamer directory=`pwd`/generated/ ! jpegdec ! videoconvert ! video/x-raw,width=1280,height=720,framerate=1/1 ! videorate ! autovideosink
To generate a H264 encoded Matroska video file with these images use the following pipeline
GST_PLUGIN_PATH=. gst-launch-1.0 myjpegstreamer directory=`pwd`/generated/ ! jpegdec ! videoconvert ! video/x-raw,width=1280,height=720,framerate=1/1 ! videorate ! openh264enc ! h264parse ! matroskamux ! filesink location=flipbook.mkv
And there you go, you wrote a GStreamer source element. To summarize what we did here, we use the GStremer base class GstPushSrc, added a property to take input from the user and set the function pointers to point to our functions that do what we want to do to insert data into the GStremaer pipeline.
Hope you enjoyed creating the flip book. See you next time!