Are video codecs written in JavaScript really the future?

Describing it as “the future,” Mozilla has been showing off ORBX.js, a video codec roughly comparable to the industry-standard H.264 that can be decoded entirely in JavaScript.

ORBX.js was developed by a company called OTOY. OTOY’s major product is Octane Render, a 3D renderer that works exclusively on NVIDIA cards using CUDA. Working with customers including Autodesk, OTOY has developed technology to allow applications such as Autodesk’s 3ds Max modelling software to be accessed over the Web without using plugins.

3ds Max streamed to a browser

Central to this is the ORBX codec. The codec allows efficient, real-time encoding on GPUs, and can be decoded in JavaScript. The JavaScript decoder works in all modern browsers, both desktop and mobile, though with differences depending on the browser features available.

Like most video codecs, ORBX distinguishes between video frames that are encoded in their entirety—“I-frames”—and those encoded as deltas relative to another frame—“P-frames” and “B-frames”. I-frames can be decoded independently of one another, but P- and B-frames can only be decoded when their surrounding frames have also been decoded. ORBX, again like most other video codecs, can be used to create video streams made up entirely of I-frames, or streams that use a mix of I-frames and P-frames. I-frame-only modes don’t compress as efficiently, but are simpler to decode.

For browsers including Internet Explorer 10 and Safari on iOS, ORBX is used in I-frame-only mode. For other browsers, including Firefox and Chrome, it uses a more conventional mixed mode. That’s because the mixed mode depends on WebGL for part of its decoding. I-frames can be encoded entirely in JavaScript, but P-frames require the use of shader programs due to their greater complexity. Internet Explorer 10 and Safari on iOS don’t support WebGL, and so can’t be used to run shader programs. As a result, they use about twice as much bandwidth for the same level of video quality.

OTOY plans to develop ORBX to make it suitable for a wider range of applications than existing codecs like H.264. The company wants to include features such as support for floating point data, to make it suitable for both high dynamic range imagery and video, and to make it usable for storing 3D data such as depth buffers.

Its encoding efficiency is broadly comparable to H.264—the company says that it’s 25 percent “better” than H.264 in some tests, but in others H.264 pulls ahead. The new HEVC/H.265 codec has better encoding efficiency than both H.264 and ORBX (it produces smaller files for a given level of quality), but requires far more computational power to encode and decode.

Being designed for streaming live apps, the ORBX encoder boasts low latency. To prove this point, the company has demonstrated the game Left 4 Dead 2 being streamed to a browser.

Left 4 Dead 2 on a PC, playing in Firefox on a Mac.

Possibilities for ORBX

OTOY describes a number of future ideas for ORBX. One plan is for more powerful remote application support, not merely streaming a video of an application (similar to the way VNC provides remote application support), but streaming the graphical API calls that an application makes, using client-side WebGL to actually perform the drawing. This reduces the load on the server end, and can reduce bandwidth requirements substantially.

Another idea is to use ORBX for DRM-free streaming video. OTOY believes that ORBX’s efficient on-GPU encoding makes it practical to embed within each video stream a per-user watermark. Watermarked video wouldn’t be DRM-protected, and so would be relatively easy to record and redistribute, but the watermarking would make it possible to determine who released a given video, in turn opening the door to pursuing the original pirates.

Mozilla’s interest in ORBX is obvious. Mozilla is aggressively promoting the Web as the platform to replace all other platforms. This is the entire point of its Firefox OS smartphone platform: that rather than developing apps tied to Android or iOS or Windows Phone or BlackBerry, developers should write apps for the Web, using JavaScript, HTML, and related technologies.

Demonstrations of video decoding in JavaScript serve to bolster Mozilla’s argument that JavaScript performance is good enough to make it usable for almost every application—even those traditionally regarded as “compute-intensive.” Mozilla itself has experimented with JavaScript-based video decoding with its Broadway decoder. This was fast enough to decode H.264 at 30 frames per second in JavaScript alone.

The alternative to DRM is also attractive to an organization that promotes the open Web. DRM capabilities are set to be included in browsers, a decision that isn’t universally popular among open Web proponents. The use of watermarking instead of DRM has some precedent—iTunes Music Store, for example, made a successful transition from DRM to watermarks—and could be pushed as a demonstration of the lack of necessity for DRM in browsers.

But is this really “the future” as Brendan Eich, Mozilla CTO, greeted it? That’s harder to see.

Challenges remain

ORBX looks very cool for certain applications, but as a demonstration of the power of “openness” it’s currently a little lacking. Organizations such as Mozilla may not like H.264’s royalty-bearing status, but it’s nonetheless an open, documented, published standard. ORBX, however, is none of these things. It’s a proprietary algorithm created by a commercial organization. We spoke to OTOY, and the company told us it has no immediate, specific plans to publish it openly. That’s not to say that the company is necessarily opposed to doing so; rather, it’s not sure how to do without undermining its commercial products.

Even apart from this detail, using JavaScript for video decompression is problematic. After many years of refusal, Mozilla decided last year that it would start supporting H.264 compression in the HTML5 <video> tag.

The company was ideologically opposed to this, due to H.264’s patents and royalties, but came to realize that it was a practical necessity, especially for Firefox OS. H.264 video is abundant, and is unambiguously “the winner” of all the current major video codecs. Consequentially, H.264 is widely supported in hardware, enabling battery-efficient hardware-accelerated playback of H.264 video. For Firefox OS to be viable, it had to support this codec and this hardware.