sys/ximage/ximagesink.c: When XShm is not available, we might get row strides that are not rounded up to multiples of...

Original commit message from CVS:
* sys/ximage/ximagesink.c: (gst_ximagesink_ximage_new):
When XShm is not available, we might get row strides that are not
rounded up to multiples of four; this is bad, because virtually
every RGB-processing element in GStreamer assumes rowstrides are
rounded up to multiples of four, so let's allocate at least enough
memory to avoid crashes in this case. The image will still be
displayed distorted though if this happens, so that still needs
fixing (maybe by allocating a bigger image with an 'even' width
and then clipping it appropriately when rendering - something for
Xlib aficionados in any case).

2 files changed, 34 insertions, 1 deletions

diff --git a/ChangeLog b/ChangeLog
index 4253064e..56e45997 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2007-05-03  Tim-Philipp Müller  <tim at centricular dot net>
+
+	* sys/ximage/ximagesink.c: (gst_ximagesink_ximage_new):
+	  When XShm is not available, we might get row strides that are not
+	  rounded up to multiples of four; this is bad, because virtually
+	  every RGB-processing element in GStreamer assumes rowstrides are
+	  rounded up to multiples of four, so let's allocate at least enough
+	  memory to avoid crashes in this case. The image will still be
+	  displayed distorted though if this happens, so that still needs
+	  fixing (maybe by allocating a bigger image with an 'even' width
+	  and then clipping it appropriately when rendering - something for
+	  Xlib aficionados in any case).
+
 2007-05-03  Michael Smith <msmith@fluendo.com>
 
 	* gst/audiorate/gstaudiorate.c: (gst_audio_rate_chain):
diff --git a/sys/ximage/ximagesink.c b/sys/ximage/ximagesink.c
index af50cc0f..321a1fc0 100644
--- a/sys/ximage/ximagesink.c
+++ b/sys/ximage/ximagesink.c
@@ -500,6 +500,8 @@ gst_ximagesink_ximage_new (GstXImageSink * ximagesink, GstCaps * caps)
   } else
 #endif /* HAVE_XSHM */
   {
+    guint allocsize;
+
     ximage->ximage = XCreateImage (ximagesink->xcontext->disp,
         ximagesink->xcontext->visual,
         ximagesink->xcontext->depth,
@@ -519,9 +521,27 @@ gst_ximagesink_ximage_new (GstXImageSink * ximagesink, GstCaps * caps)
       goto beach;
     }
 
+    /* upstream will assume that rowstrides are multiples of 4, but this
+     * doesn't always seem to be the case with XCreateImage() */
+    if ((ximage->ximage->bytes_per_line % 4) != 0) {
+      GST_WARNING_OBJECT (ximagesink, "returned stride not a multiple of 4 as "
+          "usually assumed");
+    }
+
     /* we have to use the returned bytes_per_line for our image size */
     ximage->size = ximage->ximage->bytes_per_line * ximage->ximage->height;
-    ximage->ximage->data = g_malloc (ximage->size);
+
+    /* alloc a bit more for unexpected strides to avoid crashes upstream.
+     * FIXME: if we get an unrounded stride, the image will be displayed
+     * distorted, since all upstream elements assume a rounded stride */
+    allocsize =
+        GST_ROUND_UP_4 (ximage->ximage->bytes_per_line) *
+        ximage->ximage->height;
+    ximage->ximage->data = g_malloc (allocsize);
+    GST_LOG_OBJECT (ximagesink,
+        "non-XShm image size is %" G_GSIZE_FORMAT " (alloced: %u), width %d, "
+        "stride %d", ximage->size, allocsize, ximage->width,
+        ximage->ximage->bytes_per_line);
 
     XSync (ximagesink->xcontext->disp, FALSE);
   }