Surprise! Mozilla can produce near-native performance on the Web

We put Mozilla's JavaScript subset—asm.js—to the test.

Read the whole story

 

[supafreak]

Great article! Peter please post more articles like this!

 

[dacjames]

It should be noted that asm.js is a very young project and the native compiler in Firefox still needs a lot of work. You should expect the performance to improve significantly in the next few versions. Things will get even better if/when Google and/or Apple starts competing on asm.js compilation.

The world is truly a bizarre place when the best hope for near-native performance on the web is a set of clever syntax hacks on top of fully-dynamic language.

 

[kripkenstein]

Most simply of all, the fast asm.js performance only exists in Firefox—and only in the unstable Nightly builds at that. This technology isn't in any stable version of Firefox, and it'll be some time before it will be.

Firefox's asm.js optimizations are in Beta right now, and will be in stable in about 6 weeks.

 

[Bad Monkey!]

I approve of this article. Great job Peter.

 

[trevnorris]

I'll be interested in seeing how the code comes out. Recently wrote a prime number generator by hand for v8 that comes within the 2x figure of optimized native c.

There's still a lot that can be done, but I'm not so excited at the prospect of needing to compile my javascript. The whole maintainability vs performance thing. But might just be me.

 

[ThermalNoise]

Very interesting article. Thanks for taking the time to thoroughly investigate the performance, Mozilla's numbers seemed a little too good to me too. I'm glad to be proven (mostly) wrong.

Edit to remove typo information

 

[DrPizza]

ThermalNoise[/url]":2to4wycd]The Native code with SSE vs. asm.js chart has the wrong legend. "Firefox regular JS" should be "Native with SSE".

Very interesting article. Thanks for taking the time to thoroughly investigate the performance, Mozilla's numbers seemed a little too good to me too. I'm glad to be proven (mostly) wrong.

Good spot, I will endeavour to get this fixed.

 

[Gallen]

It's worth mentioning that emscripten makes use of typed arrays for it's "regular JS" compile mode. Because of this, "regular JS" can easily be faster than most standard JavaScript.

 

[DownAndGoing]

You've given a very nice overview of asm.js, but I think the performance comparison is misleading. The output of Emscripten, whether asm.js or not, will not use JavaScript objects, dynamic types, etc., and will do virtually all of its computation on typed arrays. JavaScript written by humans tends to be much, much slower than equivalent native code (closer to 20x) because developers actually use the dynamic features of JavaScript on a regular basis, and spend little if any time manipulating typed arrays.

Minor note: Emscripten is not a Mozilla project. It's just one person hacking away on github. Quite impressive!

Also, I'm thoroughly surprised that there was no discussion of Google's NaCl (native client). Given a program in C or C++ (or any other language with an LLVM front-end) you could easily go either route. Of course NaCl doesn't play nicely with other platforms like asm.js does, but its performance is a lot more compelling than Dart so adoption looks just a bit more likely.

 

[theblop]

So the only selling point of asm.js over google's new scripting engine is that it's compatible with non-optimised browsers, so in theory the transition period would be smoother until all browsers implement their optimisations.

The problem is that most sites that will use asm.js will probably be unusable without an optimised browser anyway (e.g. a game running at 5 frames/s instead of 30), so the compatibility advantage isn't that great in the long term, especially considering the drawbacks of the large JS source files that asm.js requires, or the initial waste in compilation/debugging time.

 

[Faramir]

I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which seems a) unwarranted and b) overkill for the problem of untrusted code.

 

[kripkenstein]

ccurtsinger[/url]":207sgrqf]You've given a very nice overview of asm.js, but I think the performance comparison is misleading. The output of Emscripten, whether asm.js or not, will not use JavaScript objects, dynamic types, etc., and will do virtually all of its computation on typed arrays. JavaScript written by humans tends to be much, much slower than equivalent native code (closer to 20x) because developers actually use the dynamic features of JavaScript on a regular basis, and spend little if any time manipulating typed arrays.

True, but note that you can use typed arrays in handwritten JS too. In fact, the handwritten fannkuch benchmark here that outperforms asm.js uses typed arrays,

http://benchmarksgame.alioth.debian.org ... ng=v8&id=2

You probably won't write large amounts of code like that, so for large apps to get the typed array speedup you would likely want to compile into asm.js (let the compiler do the hard work!), but for small stuff you can definitely hand-optimize code to asm.js levels.

 

[Sobels]

Faramir[/url]":1tocvsj8]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which seems a) unwarranted and b) overkill for the problem of untrusted code.

I present to you: Malbolge. If something is easy, people will do it. If something is hellishly difficult, people will do it. In fact, people will do pretty much anything. (No, I'm not saying that asm.js is like Malbolge. Do I think Malbolge should be the de facto web language? No comment)

 

[styron]

Faramir[/url]":4yynjxh7]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which a) seems unwarranted and b) seems like overkill for the problem of untrusted code.

10 yrs ago I would wonder why people would want to stream music or video instead of downloading it and playing it locally.

That way they could play it again and again without wasting network traffic/bandwidth and they wouldn't have to put up with excessive compression, buffering headaches, broken connections etc.

Now I hardly care, and streaming has some advantages around multiple devices and not having to maintain media libraries etc.

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

It isn't about right now - it's about 5-10yrs time.

 

[RRob]

Matty[/url]":39j1j4ct]
Sure you can obfuscate Javascript to some extent, but it is rare.

You should check out URLquery.net and look at some of the browser exploit kits (the entries with high "Alerts/IDS"), and see how deep the eval loops go. Or look at a TDSS/tidserv update page that generate dynamic XOR keys.

 

[abhi_beckert]

JavaScript is a horrible language... But if we can make it really _really_ fast, that will open up the door to creating a superset language that compiles down to into javascript, similar to how Obj-C turns C into a pretty good language by simply adding some new syntax that compiles down into the old syntax.

(Objective-C started out as a pre-processor script, taking Obj-C code, spitting out C code, and then sending it to a C compiler).

For example, the way variable scope works in javascript is completely unacceptable. A superset language might replace all of that.

 

[Faramir]

styron[/url]":1ugwl425]Now I hardly care, and streaming has some advantages around multiple devices and not having to maintain media libraries etc.

You can stream in programs other than web browsers. In fact, a web browser isn't a terribly great streaming platform.

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

Programs can auto-update themselves without having to run in web browsers. In fact, your web browsers is such a program.

 

[DrPizza]

ccurtsinger[/url]":3vkym8i8]You've given a very nice overview of asm.js, but I think the performance comparison is misleading. The output of Emscripten, whether asm.js or not, will not use JavaScript objects, dynamic types, etc., and will do virtually all of its computation on typed arrays. JavaScript written by humans tends to be much, much slower than equivalent native code (closer to 20x) because developers actually use the dynamic features of JavaScript on a regular basis, and spend little if any time manipulating typed arrays.

The final graph on the third page includes some comparisons to human JavaScript. One of them uses typed arrays.

Also, I'm thoroughly surprised that there was no discussion of Google's NaCl (native client). Given a program in C or C++ (or any other language with an LLVM front-end) you could easily go either route. Of course NaCl doesn't play nicely with other platforms like asm.js does, but its performance is a lot more compelling than Dart so adoption looks just a bit more likely.

I would like to look at NaCl too, but making it work is a little more involved than asm.js. If there's enough interest, it's certainly something I could take a look at.

 

[examancer]

I just want to join others in saying this was a great article.

I'm a developer, and I contend that in many (most?) cases developer time is much more valuable than CPU time. Silicon is often so much cheaper than salary, which is why "slow" languages like Ruby, Javascript, PHP, Python, etc are hugely popular. But, there are obviously many performance sensitive tasks. Combined with the desire to write software that runs unmodified on many platforms, I think there is definitely a niche for asm.js, even if a small one.

Anyways, more articles that really dive into a subject are appreciated, even if I occasionally have to skip them because they are diving into a topic I don't have the requisite knowledge to understand. (Try as I might, I simply can't make it through many of the biology and quantum theory articles Ars publishes with any comprehension of what I've just read).

 

[pqr]

Very nice article, Peter, but with one serious flaw apparent omission. Who are the 'we' you keep referring to? You should clarify whether this is the academic self, or there is credit due to other people.

On the lighter side, the SSE benchmark game figure is likely mislabeled (native code is strangely not fastest).

In any case, such in-depth analyses are the best gems on Ars, IMO.

EDIT: better knowledge of Peter's past trajectory would have obviated the first comment.

 

[TheCrackLing]

styron[/url]":114ihudm]

Faramir[/url]":114ihudm]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which a) seems unwarranted and b) seems like overkill for the problem of untrusted code.

10 yrs ago I would wonder why people would want to stream music or video instead of downloading it and playing it locally.

That way they could play it again and again without wasting network traffic/bandwidth and they wouldn't have to put up with excessive compression, buffering headaches, broken connections etc.

Now I hardly care, and streaming has some advantages around multiple devices and not having to maintain media libraries etc.

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

It isn't about right now - it's about 5-10yrs time.

It's not that apps are moving to browsers, but apps are moving away from OS's. That is the the real goal. Firefox wants to be the "OS" for apps, just like Chrome wants to, and IE wants to. Let's face it, we are approaching the end of the OS as we know it today, and quite a few of us do not like it at all.

 

[DrPizza]

pqr[/url]":aq9mv9q2]Very nice article, Peter, but with one serious flaw. Who are the 'we' you keep referring to? You should clarify whether this is the academic self, or there is credit due to other people.

While the use of the academic "we" is commonplace, I wrote it first person singular, because some of the decisions might be considered idiosyncratic and were heavily dependent on my own value judgement; I felt that first person singular made it clearer who was making these decisions. In editing, however, it got rewritten as first person plural. For the purposes of this article, "we" refers exclusively to me.

On the lighter side, the SSE benchmark game figure is likely mislabeled (native code is strangely not fastest).

I am working on getting this addressed; unfortunately, I don't create the graphs myself, as they're done in Keynote.

 

[krimhorn]

From a programming nerd perspective this is pretty cool. From a practical programming standpoint I wonder what's the point? Are people seriously looking to Javascript as the language to develop highly complex web applications? This effort would be better off looking to create a new client-side language without the terrible language design decisions that permeate every facet of Javascript.

As a web-dev I use Javascript, but as a web-dev I hate using Javascript.

 

[pqr]

DrPizza[/url]":r0gtohlc]

pqr[/url]":r0gtohlc]Very nice article, Peter, but with one serious flaw. Who are the 'we' you keep referring to? You should clarify whether this is the academic self, or there is credit due to other people.

While the use of the academic "we" is commonplace, I wrote it first person singular, because some of the decisions might be considered idiosyncratic and were heavily dependent on my own value judgement; I felt that first person singular made it clearer who was making these decisions. In editing, however, it got rewritten as first person plural. For the purposes of this article, "we" refers exclusively to me.

Thank you for the clarification. So 'Editor' title aside, you are a programmer (software developer ) by trade.

 

[AreWeThereYeti]

Faramir[/url]"emos814]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which seems a) unwarranted and b) overkill for the problem of untrusted code.

I think you are completely misreading why this is worth doing. Of course not every program is ideally suited for the browser, that isn't the point. The point is to minimize the performance penalty of web coding as much as possible, which opens new doors in terms of web functionality. When web pages get faster, things that seemed "ill suited" before, suddenly may not seem so ill-suited any longer.

There is always a use for more speed.

 

[stux]

Great article,

Looks like the labels are wrong on :
http://cdn.arstechnica.net/wp-content/u ... -asmjs.png

That is unless regular JS is supposed to beat SSE3?

I'm curious about running some software through this, but the dealbreaker is support for threading APIs, whether its pthreads, or C++11 threads etc doesn't matter, but I personally don't write single threaded code anymore, and I doubt any serious developer does.

 

[thegeneral]

The wonders of a post Flash world.

 

[Major Major]

krimhorn[/url]":2hfbsj8z]From a programming nerd perspective this is pretty cool. From a practical programming standpoint I wonder what's the point? Are people seriously looking to Javascript as the language to develop highly complex web applications? This effort would be better off looking to create a new client-side language without the terrible language design decisions that permeate every facet of Javascript.

As a web-dev I use Javascript, but as a web-dev I hate using Javascript.

You're missing the point. The idea is that we're stuck with Javascript as the language of web browsers whether we like it or not. By treating a particular subset of it as the assembly language of browser, it's possible to make that subset somewhat fast. Then we can write highly complex applications in languages that don't suck, compile them into asm.js, and run them on browser machines that speak Javascript as their instruction set.

 

[TheFerenc]

smoofles[/url]":3erfliw3]Their rendering engine still sucks balls. Fast computing only helps so much if you can't even get image scaling right or you engine starts acting up with complicated (HTML-based) scenes.

But, hey, good that they can run a 3D engine in the browser and offer a Social API. Who needs HTML/CSS rendering quality, amirite?

Just try using the digikey search box. In IE, there is a clearly delineated search box. In firefox on my RHEL6 install, the word "Search" is in one white box, while the actual text entry field is another, identical, offset and weirdly placed white box. Makes it a real PITA to search for things, since the first thought of most people (IME), is to click on the thing that looks like a text box with the word Search in it.

Back on topic, I generally like the idea of this approach, but I'm not sure it really has legs. JS worked as the default scripting language on the web because it was supported, not because it was good. In point of fact, it's terrible.

Inertia is a powerful force in the programming world. People don't like learning new methods of doing tasks, especially when the old way still works. But as flash has been supplanted due to ill fit (not to mention performance and security), I really do think JS will be supplanted as well, for likely similar reasons. Let's be honest, security and JS are usually only in the same sentence when discussing the security holes of a new web app.

 

[helios01]

This is quite exciting, I'd definitely make use of this.

 

[fuzzyfuzzyfungus]

dacjames[/url]":1wbjoe6b]It should be noted that asm.js is a very young project and the native compiler in Firefox still needs a lot of work. You should expect the performance to improve significantly in the next few versions. Things will get even better if/when Google and/or Apple starts competing on asm.js compilation.

The world is truly a bizarre place when the best hope for near-native performance on the web is a set of clever syntax hacks on top of fully-dynamic language.

I wonder if any of the guys responsible for the (misleading, marketing driven) choice to call Javascript "javascript", and connect perception of it to the pretty-much-entirely-unrelated 'Java' language and VM find it amusing that, if current trends continue, a performance-optimized-subset of javascript will very likely be the web's target 'VM' architecture of choice, despite java actually being intended to do that and javascript being a series of wacky hacks originally intended for something else entirely?

 

[els]

styron[/url]":7wswfjbd]

Faramir[/url]":7wswfjbd]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which a) seems unwarranted and b) seems like overkill for the problem of untrusted code.

10 yrs ago I would wonder why people would want to stream music or video instead of downloading it and playing it locally.

That way they could play it again and again without wasting network traffic/bandwidth and they wouldn't have to put up with excessive compression, buffering headaches, broken connections etc.

Now I hardly care, and streaming has some advantages around multiple devices and not having to maintain media libraries etc.

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

It isn't about right now - it's about 5-10yrs time.

You will still need to maintain the code you write in a browser, wouldn't you?

Surely there're other efficient ways of writing applications that use remote services without having them run in a web browser. Just as there is a race to figure this fit all programs in a browser model there are others trying to figure out other models distributed app programming and deployment which will solve the problem of commonality on each platform.

 

[RichardJames13]

Another advantage of this is that any language that be compiled to LLVM can now be run in a web browser. The only real problem is libraries.

I discovered that I can convert python to LLVM and thus to asm.js so now my code can be portable across many platforms without worrying about packaging for each different system. The problem I face is that the libraries are not there. One day they will be.

 

[AreWeThereYeti]

fuzzyfuzzyfungus[/url]":3cwtvhdq]

dacjames[/url]":3cwtvhdq]It should be noted that asm.js is a very young project and the native compiler in Firefox still needs a lot of work. You should expect the performance to improve significantly in the next few versions. Things will get even better if/when Google and/or Apple starts competing on asm.js compilation.

The world is truly a bizarre place when the best hope for near-native performance on the web is a set of clever syntax hacks on top of fully-dynamic language.

I wonder if any of the guys responsible for the (misleading, marketing driven) choice to call Javascript "javascript", and connect perception of it to the pretty-much-entirely-unrelated 'Java' language and VM find it amusing that, if current trends continue, a performance-optimized-subset of javascript will very likely be the web's target 'VM' architecture of choice, despite java actually being intended to do that and javascript being a series of wacky hacks originally intended for something else entirely?

Yes, whoever made the Javascript name decision truly did something moronic for the ages. But until they can self-host, I'll pass on naming this the eventual "web target VM of choice". By self-host, I mean show that they can implement their own entire VM in asm.js, with acceptable slowdown.

The problem is that for something to be a universal VM, it can't be only static. Can asm.js do high-perfornance dynamic code modification? If not, it won't be a universal VM, because state-of-the-art implementations of dynamic languages require dynamic code modification (for things like in-line caching, etc). So there would be a large class of languages you couldn't host on that VM without a big performance cost. A real universal VM would support all the critical optimization tricks that VM's use, but that is an unsolved problem, for what should be obvious reasons.

 

[trevnorris]

Faramir[/url]":224ykej9]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

Eh, weak sauce. My entire operating system runs in javascript.