Thursday, August 15, 2013

A Sample to Use for Debugging and Testing Performance of the QtMultimedia Backend

In the github project for PiOmxTextures I recently uploaded also a sample player (tools/POCPlayer) that can be used for debugging and testing the performance. This is supposed to be a sample for the performance of the QtMultimedia backend and Qt itself on the Pi. It is entirely Qt/QML based, with the GUI written entirely in QML/Javascript. It shows the common controls for a media player and the service for metadata extraction. This is a demo of the current state:

How to Build

This is simple: it is a standard Qt application, with C++/Qt and QML code. A simple build of Qt version >= 5.1 should be sufficient. There is no interaction whatsoever with the plugin during build time.

Limitations

The current version can handle images and videos. Unfortunately it seems to perform pretty good in 1080p with 720p video, but 1080p videos are quite difficult to handle when other animations are running concurrently. Animations not involving video perform really good.
What you see in the video is running on the eglfs plugin: running on some more interesting layers like Wayland would require some changes in how EGL images are handled in the QtMultimedia plugin and PiOmxTextures. Unfortunately, it seems the QtWayland module is not currently perfectly working, so a hard work might be needed.

9 comments:

  1. NIce work !

    But still get problem with gradient !

    This "correct" the gradient, but image/video show ( width good gradient for 1sec ) and then auto hide ...

    QQuickView view;
    view.engine()->rootContext()->setContextProperty("utils", &qmlUtils);
    view.setSource(QUrl("qrc:///qml/main.qml"));
    view.setResizeMode(QQuickView::SizeRootObjectToView);


    QSurfaceFormat surfaceFormat;

    // CORRECTION DEPTH
    QSurfaceFormat surfaceFormat;
    surfaceFormat.setAlphaBufferSize(8);
    view.setClearBeforeRendering(true);
    view.setColor(QColor(Qt::transparent));
    view.setFlags(Qt::FramelessWindowHint);

    view.setFormat(surfaceFormat);

    // CORRECTION DEPTH END

    #ifdef RASPBERRY
    view.showFullScreen();
    #else
    view.resize(800, 400);
    view.show();
    #endif // RASPBERRY

    ReplyDelete
  2. Hello. What I have found, that ffmpeg blocks mainthread.
    When you try to connect to for example udp://@239.255.12.1:1234/ the mainthread is blocked by avformat_open_input in OMXReader.cpp for connection time, so app freezes. I noticed that when I call OMX_MediaProcessor::setFilenameInt with QMetaObject::invokeMethod with QueuedConnection it runs without mainthread blocking, but application crashes in OMX_TextureProviderQQuickItem::instantiateTexture on EGLContext eglContext = get_egl_context();.
    I tried to set a global eglContext, but then EGLint eglErr = eglGetError(); said 0x300c
    Maybe someone has an idea how to pass over it?
    Regards,
    Marek Bronowicki

    ReplyDelete
    Replies
    1. Yes, using the main thread is necessary because it is where the OpenGL texture is allocated the first time. You can't use a different thread unless you bind a new OpenGL context.

      Delete
  3. Hello. I've been checking on this project from time to time and it looks great. Thank you for posting your code. Are you currently available to do some contracting work related to this subject matter (accelerated QT with video on the RPI)?

    ReplyDelete
    Replies
    1. Hello, thanks! It highly depends on the task. You can contact me with the form if you want.
      Bye!

      Delete
    2. I sent you a message through the contact form, hope you received it. Look forward to talking!

      Delete
  4. Hi Luca, I tried to build from your instructions using Qt 5.3.0. I have all the compile ffmpeg working and patches also with prepare_openmaxil_backend completed. What might be the cause of having the video playing perfectly but with out any sound coming through (I'm using the 3.5mm jack)?

    ReplyDelete
    Replies
    1. A bit more to add, When I list QAudio available output devices I had ALSA there. The QAudio output does play .WAV files well it had the sounds coming, but with Mediaplayer (QML) I've had no luck.

      Delete
    2. Hi Luca,

      By digging into the code and stuff I must say your work is pretty amazing and I totally appreciated it! I have got the problem sorted out and now everything works perfectly! Keep on with the good stuff...Cheers!

      Delete