Video Playback Improvements in Plex
There’s been a lot of noise on the forums about hardware accelerated decoding of video in Plex lately, largely due to the release of Snow Leopard, which enables hardware decoding of Quicktime videos, and the Broadcom Crystal HD mini-PCIe card, which also supports H.264 and MPEG-2 decoding.
Obviously the question is whether these and other performance-enhancing technologies will make it to Plex. The answer depends very much on what these technologies bring to the table, so it’s probably worth spending some time on explaining them.
Quicktime X
The new version of Quicktime uses NVIDIA’s VP3 (Pure Video 3) API for hardware acceleration of H.264 video. This is currently only available on the 9400M GPUs in Macbooks and Mac Minis. The actual hardware is a separate decoder chip on the board, not directly using the GPU itself. This differs from OpenCL, which aims to use the GPU as a general-purpose CPU, and is not well suited to video decoding.
Unfortunately the API is only available when using QTKit, Apple’s new Cocoa framework for Quicktime. This means the support is limited to formats QuickTime X supports, which for our purposes isn’t particularly useful. Installing Perian helps by allowing MP4/MOV with AC3 audio to play, but MKV support even with Perian is pretty poor. Surround sound isn’t very well supported either, although future versions of Plex will have the capability to take multichannel sound being played through Quicktime, re-encode it to AC3 and send it to your receiver, in much the same way as the AC3 encoder works now. This may change if Apple broadens the API to allow acceleration of raw H.264 streams.
The other difficulty with QTKit is that rendering is only possible to a QTMovieLayer or a QTMovieView. This makes it incompatible with the current video player in Plex, so QT X support needs to wait for our new player, and is not an ideal solution for all video.
Crystal HD
This is a H.264/MPEG-2 decoder card made by Broadcom. It can be installed in the mini-PCIe slot of an older (non-NVIDIA 9400M) Macbook or Mini, replacing the WiFi card, or in a desktop using a mini-PCIe to PCIe adaptor. XBMC supports this and has made an OS X driver. Unfortunately the driver does not yet support Snow Leopard or 64 bit kernels. Once the XBMC team makes an improved driver available, we’ll look at implementing support in Plex.
64 bit FFmpeg / ffmpeg-mt
Compiling ffmpeg for 64 bit (the AV decoding library Plex uses) gives a small but measurable performance increase. Additionally, the ffmpeg developers are always improving the H.264 decoder, and we benefit from that as we upgrade Plex.
An experimental branch of ffmpeg called ffmpeg-mt is being developed, which uses multithreading to speed up video decode. In our development builds, this has also resulted in improved video playback performance.
Core Video
The dvdplayer code in XBMC isn’t ideal from design point of view anyway. Its threading is poor as a legacy from the XBox, and often results in slow performance in general, let alone playing video.
It doesn’t use timestamps for A/V sync, instead relying on audio playback speed to sync video. This is why we have so many problems with poor AV sync and drifting sync.
Apple released a new video framework in 10.4 called Core Video. Essentially this is a high-performance thread attached to a timer, which fires when the active video device is ready for a frame. It provides a clock which players can use to sync video and audio, and the advantage of outputting frames as the hardware is ready means we get smooth playback of 24p video on 50/60Hz TVs.
We’re developing a player based on this technology, which not only gives higher performance, as we only render when the display needs it, but A/V sync is much improved, as we can match DTS/PTS timestamps in the video to the audio and display hardware.
Finally…
We’re certainly not sitting still on the issue of better performance, but we’re aware for most people, their videos play fine, and the overall user experience is what needs work. We’re moving forward on all fronts, and hope to have some exciting things to show you soon.
17 comments17 Comments so far
Leave a reply
Good decision to work with the user experience right now.
Keep it up!
Sounds great! Especially “core video” – would save me from firing up frontrow every so often to play a video which has sync problems in plex.
According to a comment left on this article:
http://www.xlml.com/aehso/2010/01/13/mac-mini-xbmc-broadcom-crystal-hd-1080p-video/
The Crystal HD driver works fine in Snow Leopard. It may not work in 64-bit, but really, most people use 32-bit kernels anyway. For the price of the card (cheap!), it seems a lot of Plex users would benefit from this, myself included!
In my testing, the driver is unable to reliably allocate DMA memory under the 4GB barrier on 32-bit, and I have not had time to debug it extensively. Additionally, I run a 64 bit kernel.
Does your plans include taking advantage of the Grand Central Dispatch and OpenCL technologies built into Snow Leopard?
As and where appropriate, yes, although neither GCD nor OpenCL have much utility in HD decoding.
i suggest you read the actual xbmc code, rather than a 2 year old fork of it. we can sync to display, and all that and have been able to for about a year now. stop spreading FUD and outright lies please.
Hi Ryan, Nice to see you are playing with CrystalHD. The current CrystalHD driver “should” be mapping >4GB down to < 4GB for a 32-bit kernels but IODMACommand is not playing attention to the request. The fix is trivial and I should have it done when I find the time. Support for 64-bit kernels will also show up then.
Ok. Unless I’m misinformed Plex does not use GPU hardware acceleration for decoding today, therefore I thought that maybe these technologies, in particular OpenCL, would be a great and easy way of tapping into the power of the GPU to take some load of the main processor.
I don’t have enough knowledge about these kind of things to know if it’s a possibility.
Well, emotional outbursts aside, the dvdplayer code in Plex is obselete, broken, and is being replaced with an OS X-centric solution.
Sounds good, thanks Scott.
As had been discussed on the forums, OpenCL has the ability to accelerate certain parts of the H264 decode, but lacks the logic support to perform all of it. Consequently data would have to be swapped between VRAM and system RAM while decoding, and this would mitigate the benefit.
If anyone is aware of an open source OpenCL decoder, I’d be very interested to hear about it.
Ok, thanks for the info Ryan. Looking forward to see what’s in store for future releases.
FFMpeg-mt + Core Video sounds delicious. Can’t wait to try it out…
Ah, I just went to the forums to add a request for broadcom support after seeing it in an AppleTV. I was surprised to see how behind the times I was and that the topic had been discussed at length.
I was dismayed to read this statement from Plex but Scott Davilla’s comment has cheered me up a bit since Ryan did say they would re-look if the xbmc team made a new driver available
As always, we can only do what’s physically possible…
[...] Ik kwam het volgende artikel tegen over eventuele hardware acceleratie binnen Plex: http://ryan.plexapp.com/?p=34 Categorieën:Apple, XBMC Tags: Reacties (0) Trackbacks (0) Reageer [...]