There’s been a lot of noise on the forums about hardware accelerated decoding of video in Plex lately, largely due to the release of Snow Leopard, which enables hardware decoding of Quicktime videos, and the Broadcom Crystal HD mini-PCIe card, which also supports H.264 and MPEG-2 decoding.
Obviously the question is whether these and other performance-enhancing technologies will make it to Plex. The answer depends very much on what these technologies bring to the table, so it’s probably worth spending some time on explaining them.
The new version of Quicktime uses NVIDIA’s VP3 (Pure Video 3) API for hardware acceleration of H.264 video. This is currently only available on the 9400M GPUs in Macbooks and Mac Minis. The actual hardware is a separate decoder chip on the board, not directly using the GPU itself. This differs from OpenCL, which aims to use the GPU as a general-purpose CPU, and is not well suited to video decoding.
Unfortunately the API is only available when using QTKit, Apple’s new Cocoa framework for Quicktime. This means the support is limited to formats QuickTime X supports, which for our purposes isn’t particularly useful. Installing Perian helps by allowing MP4/MOV with AC3 audio to play, but MKV support even with Perian is pretty poor. Surround sound isn’t very well supported either, although future versions of Plex will have the capability to take multichannel sound being played through Quicktime, re-encode it to AC3 and send it to your receiver, in much the same way as the AC3 encoder works now. This may change if Apple broadens the API to allow acceleration of raw H.264 streams.
The other difficulty with QTKit is that rendering is only possible to a QTMovieLayer or a QTMovieView. This makes it incompatible with the current video player in Plex, so QT X support needs to wait for our new player, and is not an ideal solution for all video.
This is a H.264/MPEG-2 decoder card made by Broadcom. It can be installed in the mini-PCIe slot of an older (non-NVIDIA 9400M) Macbook or Mini, replacing the WiFi card, or in a desktop using a mini-PCIe to PCIe adaptor. XBMC supports this and has made an OS X driver. Unfortunately the driver does not yet support Snow Leopard or 64 bit kernels. Once the XBMC team makes an improved driver available, we’ll look at implementing support in Plex.
64 bit FFmpeg / ffmpeg-mt
Compiling ffmpeg for 64 bit (the AV decoding library Plex uses) gives a small but measurable performance increase. Additionally, the ffmpeg developers are always improving the H.264 decoder, and we benefit from that as we upgrade Plex.
An experimental branch of ffmpeg called ffmpeg-mt is being developed, which uses multithreading to speed up video decode. In our development builds, this has also resulted in improved video playback performance.
The dvdplayer code in XBMC isn’t ideal from design point of view anyway. Its threading is poor as a legacy from the XBox, and often results in slow performance in general, let alone playing video.
It doesn’t use timestamps for A/V sync, instead relying on audio playback speed to sync video. This is why we have so many problems with poor AV sync and drifting sync.
Apple released a new video framework in 10.4 called Core Video. Essentially this is a high-performance thread attached to a timer, which fires when the active video device is ready for a frame. It provides a clock which players can use to sync video and audio, and the advantage of outputting frames as the hardware is ready means we get smooth playback of 24p video on 50/60Hz TVs.
We’re developing a player based on this technology, which not only gives higher performance, as we only render when the display needs it, but A/V sync is much improved, as we can match DTS/PTS timestamps in the video to the audio and display hardware.
We’re certainly not sitting still on the issue of better performance, but we’re aware for most people, their videos play fine, and the overall user experience is what needs work. We’re moving forward on all fronts, and hope to have some exciting things to show you soon.17 comments