Sunday, March 19, 2017

Experimenting with Pi3 optimisations - POT 5.6.0-beta1 with Qt 5.8.0 built for armv8 with GCC 4.9.4

At the moment I still don't want to test custom firmwares or 64 bit arch builds, but I started to test a couple of new features: a new compiler from Linaro (the one provided by the foundation keeps giving me headaches), version 4.9.4 instead of 4.8, and optimised compiler flags for the Rapsberry Pi 3, which is an armv8.
In this build, Qt, ffmpeg and POT are all built with 4.9.4 Linaro toolchain and optimised compiler flags for Pi3. This will only work on Pi3.
You won't probably see much difference in GPU intensive apps, but it is a step on the road of optimisation!

Have fun! Bye! ;-)

Download the toolchain here.
Download POT 5.6.0-beta1 for Raspbian Jessie Lite Pi3 here (md5: 0eec41ef02e9369fc7e569030b8ff868).

Saturday, January 28, 2017

Hardware Decoding in Chromium through QtWebEngine on Raspberry Pi

A few months back I decided to try to implement hardware decoding in WebKit. Unfortunately this task is always pretty long and complex for many reasons. I found the time to draft an implementation for WebKit1, which is pretty useless as WebKit1 in Qt is only used outside QML and JS is executed in the main thread. Unfortunately I never found the time to implement this in WebKit2, which runs in QML and is suitable for more fluid UIs. This was the result:
This is how YouTube was running with this implementation:




Now Qt has deprecated QtWebKit and is working heavily on QtWebEngine, which is built on Chromium, so I wanted to try this road. Unfortunately these kind of things always claim a lot of time, and I don't typically have that much, but I was able to start and get something done already.

Writing a complete solution in Chromium to decode and render video takes much time, so I thought of a shortcut: creating a custom VDA (Video Decode Accelerator) that loads the POT library and reusing its entire codebase to implement decode and rendering with little modifications. This proved to be possible and now I get something on the screen.

So, to summarise: a little patch to Chromium is needed to create a VDA that dynamically loads POT library into memory and uses it with a common interface. Data and calls are translated to POT structures and are sent to POT, which then processes the buffers properly. The result of the decode operation is then sent back to the VDA through the same interface and textures are then sent to Chromium for rendering.

Still many problems remain open, there is much to be done yet as you can see from the video, but something is drawn. Have a look at the demo:


Have fun! ;-)

Tuesday, December 27, 2016

Binaries for POT 5.5.0-beta1 on Qt 5.8.0-rc1 and Experimental Accelerated QtWebKit

A new version of Qt is out and some interesting changes are being developed. Interesting work was done on WebEngine. This package contains an experimental build of Qt 5.8.0-rc1 with the POT driver to provide hardware acceleration to video decoding and rendering. POT was updated to build on the new plugin architecture of Qt and includes ffmpeg 3.2.2. For info on the content of the package refer to this article. The youtube player still runs on WebKit 1 and so is still hardly useful.

Please report issues on github. Have fun ;-)

Download POT 5.5.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: 7ca1b961ab8c70176f6aeb13d0bc4f9a).

Sunday, December 11, 2016

Binaries for POT 5.4.0 on Qt 5.7.0 and Experimental Accelerated QtWebKit

So far, no real complains about POT 5.4.0-beta1 so I suppose it is sufficiently safe to consider it POT 5.4.0. Refer to this article for more info.

Monday, July 25, 2016

Binaries for POT 5.4.0-beta1 on Qt 5.7.0 and Experimental Accelerated QtWebKit

This is a new binary package including the same content of POT 5.3.0-beta1 package, with some updates:

  • new improvements in POT taken from omxplayer code;
  • bump to ffmpeg 3.1;
  • based on Qt 5.7.0.
The build includes support for network protocols and WebKit 1 supporting video html5 elements (including youtube).
This version is still experimental. Please report bugs on github.
Have fun! Bye! ;-)

Download POT 5.4.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: e6b6a1f92b71ac2bdd3d9756efd7669c).

Monday, June 27, 2016

Streaming from VLC to a Qt App on HTTP Using POT

The latest version of POT (5.3.0-beta1) implements some kind of buffering which makes it also possible to do some streaming. ffmpeg provides what is needed to work on protocols like HTTP. This video is a little demo showing how simple it is to stream from VLC on HTTP, remuxing to mpeg-ts, and decoding/rendering on Raspberry Pi in an accelerated Qt application using POT.
Such feature can be useful when dealing with streams coming from many sources, like an IP camera for instance.
Have fun! Bye ;-)

Sunday, June 12, 2016

Binaries for POT 5.3.0-beta1 on Qt 5.6.0 and Experimental Accelerated QtWebKit

This is an experimental version of POT that includes a few new features. The driver now includes an experimental version of playback of remote content including proper buffering of the media and the QtWebKit library is able to play videos using proper hardware acceleration.

Features

The driver can now be used through the regular Qt/QML interface to play content over protocols different from file://. I only tested HTTP but others should be available.
This new feature allowed me to also implement a new player in QtWebKit using the POT driver to stream content to the regular Qt QWebView. Unfortunately I could only find the time to implement this in the WebKit 1 branch, not in WebKit 2 which requires more work. This makes the implementation hardly useful, but it is a start. If anyone wanted to contribute this is where to start. This video shows the new implementation in action:
As QtWebKit now uses the GPU properly, I could also implement a YouTube player app using the same QWebView with just a few lines of code of regular Qt code:

In the video you can also see an application I wrote to implement a YouTube fullscreen player using the YouTube iframe API. Such a sample app is also included in the package.
Of course the real target would be to implement this in QtWebEngine. I was never able to find a version of QtWebEngine with video acceleration for Pi. Anyone who knows one?


Package

The package now includes:

  • build_valgrind.tar: a build of valgrind to analyse your code;
  • libpiomxtextures_qmlutils.so: QML plugin to provide a video probe in QML;
  • piomxtextures_browser_we: an application that tests the QtWebEngine module (run passing a URL, ./piomxtextures_browser_we http://www.youtube.com);
  • piomxtextures_browser_wk: an application that tests the QtWebKit module:
    • you can run on WebKit 1 like ./piomxtextures_browser_wk http://www.youtube.com;
    • or you can run on WebKit 2 like ./piomxtextures_browser_wk --wk2 http://www.youtube.com;
  • piomxtextures_pocplayer: a sample QML player;
  • piomxtextures_pocplayer_yt: a youtube sample player (run passing a video ID like ./piomxtextures_pocplayer_yt 71UvXMzVgx4);
  • qtdeps.tar: the usual libs needed by Qt and POT;
  • Qt-rasp2-5.6.0.tar: a build of Qt 5.6.0 stable including:
    • regular Qt 5.6.0 modules;
    • untested bluetooth module including BLE support;
    • untested MySQL plugin;
    • untested QtFtp module;
    • untested Qt hat tools module;
    • QtWebEngine;
    • QtWebKit.
Please report any bug you find on github. Don't use in production. Sources will be available shortly. Have fun! ;-)

Download POT 5.3.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: edbe0ad2a552a5c8280e3876d16b237d).