Surprise! Mozilla can produce near-native performance on the Web

We put Mozilla's JavaScript subset—asm.js—to the test.

Read the whole story

 

[talksense101]

ccurtsinger[/url]":zhsuuxsu]
Also, I'm thoroughly surprised that there was no discussion of Google's NaCl (native client). Given a program in C or C++ (or any other language with an LLVM front-end) you could easily go either route. Of course NaCl doesn't play nicely with other platforms like asm.js does, but its performance is a lot more compelling than Dart so adoption looks just a bit more likely.

NaCl is a royal pain that is not portable. Bundling executables with the Chrome plugin that you develop for every platform (64bit, 32bit, ARM) isn't intuitive. I don't see NaCl ever getting the popularity that ASM.JS is likely to get.

 

[Faramir]

Didn't support for PNaCl just land in canary?

 

[jrose]

Uh...


Since when is "half speed" (i.e., factor of 2) "near native"?

 

[DrPizza]

jrose[/url]":kpv1ue2e]Uh...


Since when is "half speed" (i.e., factor of 2) "near native"?

Atom systems would love to be able to run at half the speed of my Sandy Bridge.

 

[jrose]

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

It isn't about right now - it's about 5-10yrs time.

Whilst yes, speed will get better, etc.



We're still burning (according to the 2x performance penalty listed) 2x the CPU for the same task.

Which means 2x the CPU load, which means 2x the power burn.

I'd prefer to have a battery half the size, or life twice as long, thanks.

 

[jrose]

DrPizza[/url]":lev7mqd3]

jrose[/url]":lev7mqd3]Uh...


Since when is "half speed" (i.e., factor of 2) "near native"?

Atom systems would love to be able to run at half the speed of my Sandy Bridge.

Sure.

But running asm.js won't get them there, and see my CPU burn vs battery comment above...


To turn your point on its head - by running Javascript you can turn your Sandy Bridge into what an atom would "love to" perform like.

No thanks.

 

[Matt Raymond]

drfisheye[/url]":2z1ucgpq]

Matt Raymond[/url]":2z1ucgpq]

This doesn't mean that Chakra is necessarily slower in "real life"; the code produced by Emscripten is substantially unlike any normal code that browsers would run. It may not be altogether surprising that they don't handle it well.

Stop making excuses for Chakra. Emscripten outputs JavaScript that uses typed arrays to increase performance, and Internet Explorer 10 is supposed to support Typed Arrays. If not for things like Emscripten, why did they add support for Typed Arrays in the first place, because it certainly wasn't for WebGL.

IE11 will have WebGL. So "it certainly wasn't for WebGL" is not that certain.

Microsoft's stated reason for including support for Typed Arrays is to "handle binary data". In fact, several W3C specifications (FileAPI, XHR2, WebSockets) use Typed Arrays to access binary data. So, no, it wasn't for WebGL. In fact, the only reason they're implementing WebGL in IE 11 is because Firefox, Safari, Chrome and Opera already support it. Don't be surprised if they "innovate" but slipping in support for HLSL or something similar, though.

 

[Frederik Aalund]

Good article and an interesting read!

I just have one remark that I can perfectly understand that you left out of the article for brevity.

In a C++ program, an object in memory is typically represented by the memory address of the class's v-table (a table of all the functions belonging to the object's class) followed by the storage for the object's data

"Typical" in this context is debatable. The v-table only comes into play when the programmer has used virtual functions (e.g., for polymorphism). Some applications make use of this feature but other applications don't. The latter applications will not have v-tables stored with each class instance. This is the zero-overhead principle of C++ in action: If you don't use a feature, you won't pay for it.

Just to stretch the argument further: Even when virtual functions are used, the C++ standard does not mandate the use of v-tables. However, all the implementations I know of use v-tables.

[EDIT] Typo.

 

[web2nr]

abhi_beckert[/url]":3r2jk63k]JavaScript is a horrible language... But if we can make it really _really_ fast, that will open up the door to creating a superset language that compiles down to into javascript

Plenty of those already exist, look it up.

abhi_beckert[/url]":3r2jk63k]For example, the way variable scope works in javascript is completely unacceptable.

It's easy to dislike what you don't understand. The variable scope is one of things I enjoy about JavaScript.

 

[trollhunter]

Quite interesting reading. asm.js looks a promising redemption to Javascript as a programming language. But why restricting the tests against IE10 only ? I would like to see some comparisons against the latest Chrome, too.

 

[Hurda]

Whee, since when is the report-button working?
Just used it, great feeling.

 

[Sobels]

jrose[/url]":1ntig1dh]

DrPizza[/url]":1ntig1dh]

jrose[/url]":1ntig1dh]Uh...


Since when is "half speed" (i.e., factor of 2) "near native"?

Atom systems would love to be able to run at half the speed of my Sandy Bridge.

Sure.

But running asm.js won't get them there, and see my CPU burn vs battery comment above...


To turn your point on its head - by running Javascript you can turn your Sandy Bridge into what an atom would "love to" perform like.

No thanks.

Have you ever run a game that uses Python for certain parts (like Civ)? Python performs much more abysmally than JS, on the order of 10x or more.

 

[DrPizza]

trollhunter[/url]":3rvxjtzj]Quite interesting reading. asm.js looks a promising redemption to Javascript as a programming language. But why restricting the tests against IE10 only ? I would like to see some comparisons against the latest Chrome, too.

This was addressed explicitly: Chrome crashed all the time, apparently incapable of handling the large memory allocations. I expect this will be fixed sooner or later, and it's not impossible that it's been fixed already, but at the time I performed the tests and wrote the article, Chrome was too unreliable to compare.

 

[Voo42]

web2nr[/url]":2fghd6ms]

abhi_beckert[/url]":2fghd6ms]For example, the way variable scope works in javascript is completely unacceptable.

It's easy to dislike what you don't understand. The variable scope is one of things I enjoy about JavaScript.

Really? I mean really? Someone is defending JS' scoping rules? Apparently it's also easy to like something if you're ignorant about the alternatives. Are you aware that even Brendan Eich considers the scoping rules in Javascript broken and tried to fix them in Harmony? ("Hey let's adopt C scoping syntax and then break it in a totally useless way that surprises everybody and needs a hack to fix, because we have a very limited timeframe to implement the language and it's easier!" - yep great idea)

Paraphrasing Peter Norvig: Design patterns are generally deficiencies in the language and having to write anonymous functions to get the scoping behavior that's not only expected but also in pretty much every situation conceivable preferred is just horrible.

And let's not even think about free variables, stupendous hoisting rules (quick are function definitions hoisted out of conditionals or not? Are variable definitions?), the four different versions of this, coercion rules for ==/!=, with statement, for..in statement problems, and.. can we stop here now?

 

[quaint]

minifig404[/url]":3l9x75t1]Just a random side-note: C is the fastest language tested by The Computer Language Benchmarks Game. The second fastest langauge is C++, at < 3x slower than C. --

Food for thought.

Not that well-made. C++ is really better described as comparable to C in terms of speed in the Benchmarks Game (although, strictly speaking, what they do is compare the speed of implementations, not languages). In some tests it's faster, in some it's slower and they mostly cancel each other when looking at the big picture; although the C++ programs tend to use more memory (and asm.js even more so, I would guess).

 

[Voo42]

quaint[/url]":3uj8bixo]

minifig404[/url]":3uj8bixo]Just a random side-note: C is the fastest language tested by The Computer Language Benchmarks Game. The second fastest langauge is C++, at < 3x slower than C. --

Food for thought.

Not that well-made. C++ is really better described as comparable to C in terms of speed in the Benchmarks Game (although, strictly speaking, what they do is compare the speed of implementations, not languages). In some tests it's faster, in some it's slower and they mostly cancel each other when looking at the big picture; although the C++ programs tend to use more memory (and asm.js even more so, I would guess).

Looking at most of the really fast running C++ code you don't see many classes or other stuff though, some stuff in C++ though allows it to actually be faster than standard c code. Or to be more exact: Getting the equivalent speed in C would need *lots* of extra code (well templates really, sorting stuff in c using qsort and a function pointer is generally horribly slow)

 

[Faramir]

Given that c++ is (almost) a superset of c, and assuming that by c++ you mean the language and not the standard library you should be able to write a c in c++ program for most situations that would compile to very similar assembly code.

That said, I think that'd defeat the point of the shoot out.

 

[Sobels]

Voo42[/url]":3f6553sn]

quaint[/url]":3f6553sn]

minifig404[/url]":3f6553sn]Just a random side-note: C is the fastest language tested by The Computer Language Benchmarks Game. The second fastest langauge is C++, at < 3x slower than C. --

Food for thought.

Not that well-made. C++ is really better described as comparable to C in terms of speed in the Benchmarks Game (although, strictly speaking, what they do is compare the speed of implementations, not languages). In some tests it's faster, in some it's slower and they mostly cancel each other when looking at the big picture; although the C++ programs tend to use more memory (and asm.js even more so, I would guess).

Looking at most of the really fast running C++ code you don't see many classes or other stuff though, some stuff in C++ though allows it to actually be faster than standard c code. Or to be more exact: Getting the equivalent speed in C would need *lots* of extra code (well templates really, sorting stuff in c using qsort and a function pointer is generally horribly slow)

There's nothing that intrinsically makes classes slower than plain functions. A method is really just a plain C function with an additional argument for the "this" pointer. And as you said, sometimes it can actually reduce the indirection you use. It just comes down to how much time you're really willing to spend optimizing something that takes microseconds anyway.

 

[Voo42]

Sobels[/url]":1imhgkb6]
There's nothing that intrinsically makes classes slower than plain functions. A method is really just a plain C function with an additional argument for the "this" pointer. And as you said, sometimes it can actually reduce the indirection you use. It just comes down to how much time you're really willing to spend optimizing something that takes microseconds anyway.

Absolutely true, but the thing is if you're going to use classes you'll most likely also use OOP principles and that in general does induce some runtime cost (exceptions are situations where you need a class because the language demands it), So yes should've been more specific there.

 

[DrPizza]

Voo42[/url]":sw3s79po](exceptions are situations where you need a class because the language demands it)

No they aren't, and no it doesn't.

 

[Voo42]

DrPizza[/url]":3dshdkxo]

Voo42[/url]":3dshdkxo](exceptions are situations where you need a class because the language demands it)

No they aren't, and no it doesn't.

Huh? So you're saying actually writing OOP code in c++ does have absolutely no overhead and will give you generally just the same performance as vanilla c code? Unlikely that.

Yes if you use classes just to avoid passing a pointer to a struct to your methods you'll get the same performance, but as soon as you start using all the actual OOP stuff performance suffers to some degree or the other. From a glance at the well performing c++ code it generally looks pretty similar to the c code..

 

[quaint]

Voo42[/url]":2hakb2h2]

DrPizza[/url]":2hakb2h2]

Voo42[/url]":2hakb2h2](exceptions are situations where you need a class because the language demands it)

No they aren't, and no it doesn't.

Huh? So you're saying actually writing OOP code in c++ does have absolutely no overhead and will give you generally just the same performance as vanilla c code? Unlikely that.

Yes if you use classes just to avoid passing a pointer to a struct to your methods you'll get the same performance, but as soon as you start using all the actual OOP stuff performance suffers to some degree or the other. From a glance at the well performing c++ code it generally looks pretty similar to the c code..

Yes (well, I didn't say it but I do say it now).

That's because there is no real alternative to, say, polymorphism in many cases. The human factor limits the massive duplication of code and interfaces, which is why you'll see OOP practices mimicked even in languages without formal support (C for example).

If the problem at hand doesn't benefit from what you consider as "actual OOP stuff", then why would you use it? You don't, hence no performance penalty (even if you modelled your program in terms of objects instead of buffers).

 

[DrPizza]

Voo42[/url]":2ksenqik]

DrPizza[/url]":2ksenqik]

Voo42[/url]":2ksenqik](exceptions are situations where you need a class because the language demands it)

No they aren't, and no it doesn't.

Huh? So you're saying actually writing OOP code in c++ does have absolutely no overhead and will give you generally just the same performance as vanilla c code? Unlikely that.

I'm saying that exceptions in C++ do not require you to use a class. The language does not demand it. Throw an int if you want.

Yes if you use classes just to avoid passing a pointer to a struct to your methods you'll get the same performance, but as soon as you start using all the actual OOP stuff performance suffers to some degree or the other. From a glance at the well performing c++ code it generally looks pretty similar to the c code..

C++ is not synonymous with "virtual dispatch". C++ features such as templates are arguably as important as virtual dispatch--perhaps even more important, in fact, if you're to judge by the design of the C++ Standard Library--and can often reduce runtime overhead, relative to comparable C implementations.

 

[Voo42]

DrPizza[/url]":1b6mbhpc]
I'm saying that exceptions in C++ do not require you to use a class. The language does not demand it. Throw an int if you want.

I totally agree and neither did I say that, although I can see how my sentence could be misunderstood.

DrPizza[/url]":1b6mbhpc]
C++ is not synonymous with "virtual dispatch". C++ features such as templates are arguably as important as virtual dispatch--perhaps even more important, in fact, if you're to judge by the design of the C++ Standard Library--and can often reduce runtime overhead, relative to comparable C implementations.

Umn yes, which is why I said that C++ templates give you advantages compared to C which you could otherwise only get by copious code duplication (the obvious example being qsort with function pointers being horribly slow compared to std::sort).
But templates have nothing to do with OOP and I still claim that the number of C++ programs that perform comparably to vanilla c programs and use OOP principles is vanishingly small (that obviously assumes that we don't go by "There's a class definition in the code, it's obviously object oriented!" there's more to it than that).

That's because there is no real alternative to, say, polymorphism in many cases.

I disagree, just because you pick polymorphism, etc. in say Java, doesn't mean that a corresponding C or LISP program would reinvent polymorphism to solve the same problem. Clearly the language influences the design process and decisions.

 

[DrPizza]

Voo42[/url]":yles7tmh]

DrPizza[/url]":yles7tmh]
I'm saying that exceptions in C++ do not require you to use a class. The language does not demand it. Throw an int if you want.

I totally agree and neither did I say that, although I can see how my sentence could be misunderstood.

Oh, I think I see what you mean.

Umn yes, which is why I said that C++ templates give you advantages compared to C which you could otherwise only get by copious code duplication (the obvious example being qsort with function pointers being horribly slow compared to std::sort).
But templates have nothing to do with OOP and I still claim that the number of C++ programs that perform comparably to vanilla c programs and use OOP principles is vanishingly small (that obviously assumes that we don't go by "There's a class definition in the code, it's obviously object oriented!" there's more to it than that).

I just don't know what you mean by that. What are you using "OOP principles" for and what technique are you using in C to achieve the same thing? And if the answer is "nothing" then the question is: then why are you doing it in C++?

 

[jimjimjimjimjim]

Voo42[/url]":1262d77j]

That's because there is no real alternative to, say, polymorphism in many cases.

I disagree, just because you pick polymorphism, etc. in say Java, doesn't mean that a corresponding C or LISP program would reinvent polymorphism to solve the same problem. Clearly the language influences the design process and decisions.

Apparently you folks haven't seen the way unions, structs, and void pointers are used by a lot of C programmers. OOP is a way of thinking. An OOL is a way of expressing it. OOP techniques when implemented in C have overhead too.

You avoid the overhead in C the same way you do in C++: You avoid using the more expensive OOP tools.

 

[ngativ]

styron[/url]":15m8si1f]

Faramir[/url]":15m8si1f]I don't understand why people seem to think that every sort of program should run in a browser.

Triple A video games, scientific computing programs, and computer assisted design programs seem particularly ill suited to a web browser strait jacket, that nonetheless people are enthusiastically working on.

The number one answer seems to be that people trust the browsers' sandboxes which a) seems unwarranted and b) seems like overkill for the problem of untrusted code.

10 yrs ago I would wonder why people would want to stream music or video instead of downloading it and playing it locally.

That way they could play it again and again without wasting network traffic/bandwidth and they wouldn't have to put up with excessive compression, buffering headaches, broken connections etc.

Now I hardly care, and streaming has some advantages around multiple devices and not having to maintain media libraries etc.

Apps moving to browsers is the same kind of progression. Yes it kind of sucks at the moment (especially for complex apps), but year by year it sucks less, and the advantages around not having to maintain software start becoming more compelling.

It isn't about right now - it's about 5-10yrs time.

Ten years ago nobody wanted to stream music because music stores didn't exist , and all you had to do was to download the music and maintain a 'limitless' cache . Those download options were banned from the internet, becoming more difficult to access and enjoy . Nowadays, all you have to do is to download it legally , maintain a couple of gigabytes of cache and maybe suffer from some very annoying and intrusive ads if you don't want to pay a fee. Nowadays is not better than yesterday, it is just different and forcibly more convenient.

The problem with web applications is that they are trying to reinvent the wheel for their own convenience, not for the consumer convenience. Remember that the real code is in the server side not the client side.

So the problem is not just about if "it can be done", the problem is more about "why you want it to be done?" and "who wants it to be done?". There's no only demand from the consumer side, there's also demand from the corporative and developer side to gain more exclusive control over the applications and deployment . From the consumer desktop regular consumer perspective, most of their needs are already fulfilled with current desktop applications.

 

[Chuckstar]

I find it interesting that on the desktop, we're working to put more and more functionality in the browser, while on mobile platforms it's all getting moved out of the browser into apps.

Is this simply because so many people use computers at work and can't install apps? (I know that's why desktop apps are not at all compelling for me..)

Or is it also because the screen, bandwidth and keyboard limitations of mobile computing simply make standalone apps easier to use than a browser? (On the desktop, opening a browser, typing "weather.com", and waiting for the page to load is the quickest way to get to current weather info. For me, that's even faster than digging in the apps menu to find the weather app and waiting for it to load. On my iPhone, it's the other way around. Way easier to just find the weather app. I believe its a combination of how much slower typing is on a phone and, for mobile use, download speed is much slower; you want as much of the app's code/interface to be already local, just reaching out to the internet for content updates.)

 

[slacka]

ccurtsinger[/url]":3aimdzm8] I'm thoroughly surprised that there was no discussion of Google's NaCl (native client). Given a program in C or C++ (or any other language with an LLVM front-end) you could easily go either route. Of course NaCl doesn't play nicely with other platforms like asm.js does, but its performance is a lot more compelling than Dart so adoption looks just a bit more likely.

I was surprised at this omission too. Considering the massive incompatibility with other browsers described in the article along with the performance delta between asm.js on firefox and other browsers, it's nearly as restrictive as NACL. Other than that, thoroughly enjoyed this article.

 

[kccqzy]

t1oracle[/url]":a2af6tkg]Unless it is easy to debug in all web browsers on all platforms I have no use for this. If it only works in Firefox or cannot be debugged in one of the browser/OS combinations that I have to support, it is not viable.

So, they better get working on a some way to allow break points, stepping, pausing, and error messages directly on the original source instead of some compiled version of it. More importantly, this has to work the same on all browsers' debugging tools.

But the point of compiling C++ in the first place is that you can compile C++ as native executable, use whatever you're accustomed to to debug, be it GDB, DDD or whatever tools Windows people use. And then, just recompile for the web when debugging is done.

Of course this is based on the assumption that asm.js itself does not have bugs so as to render perfectly correct code incorrect on the web.

This is not some web-only language or a web-only variant of some language. It is portable.

 

[slacka]

kccqzy[/url]":2d0bwmpg]

t1oracle[/url]":2d0bwmpg]

But the point of compiling C++ in the first place is that you can compile C++ as native executable, use whatever you're accustomed to to debug, be it GDB, DDD or whatever tools Windows people use. And then, just recompile for the web when debugging is done.

Of course this is based on the assumption that asm.js itself does not have bugs so as to render perfectly correct code incorrect on the web.

This is not some web-only language or a web-only variant of some language. It is portable.

As the article mentioned, you have to use a subset of C and have to write any of your code that takes advantage of threading, SIMD, or the system timer.

Also, portability is still a pipe dream when it comes to HTML5 standards. For example HTML5 Sound is totally broken and even javascript needs it's own platform tweaks. Unreal complied in asm.js only ran on FF, not on Chrome, IE, or Opera's javascript engine.

 

[jimjimjimjimjim]

Chuckstar[/url]":1fld3m9j]I find it interesting that on the desktop, we're working to put more and more functionality in the browser, while on mobile platforms it's all getting moved out of the browser into apps.

Unless you're Mozilla. Then you're trying to get apps into browsers and have apps run everywhere.

 

[slacka]

trollhunter[/url]":1l3p053i]Quite interesting reading. asm.js looks a promising redemption to Javascript as a programming language. But why restricting the tests against IE10 only ? I would like to see some comparisons against the latest Chrome, too.

because HTML5 compatibility is a joke, Write once, run anywhere, has been and always will be a pipedream. The asm.js Javascript crashes Chrome.

From the article

We tried various versions of Chrome (stable, dev, and canary branches) but the tests would not run consistently in any of them... Chrome consistently produced sad tabs.

 

[jrose]

Sobels[/url]":35q72vvh]

Have you ever run a game that uses Python for certain parts (like Civ)? Python performs much more abysmally than JS, on the order of 10x or more.

Yes I have, and Python is not used for the performance critical parts - for good reason.

 

[jrose]

ngativ[/url]":27546w5h]
So the problem is not just about if "it can be done", the problem is more about "why you want it to be done?" and "who wants it to be done?". There's no only demand from the consumer side, there's also demand from the corporative and developer side to gain more exclusive control over the applications and deployment . From the consumer desktop regular consumer perspective, most of their needs are already fulfilled with current desktop applications.

Exactly.

Google and Mozilla have a vested interest in getting you to write your applications in javascript. Because it is native to their application.

Given the choice between native code (e.g., iOS app) and Javascript, native code will win on CPU load/battery drain by a long way, if we're still talking a 2x performance penalty for Javascript.

Unless someone makes a Javascript optimized mobile CPU or something. The mind is repelled at the thought though.

 

[Sobels]

jrose[/url]":1yjazo3i]

Sobels[/url]":1yjazo3i]

Have you ever run a game that uses Python for certain parts (like Civ)? Python performs much more abysmally than JS, on the order of 10x or more.

Yes I have, and Python is not used for the performance critical parts - for good reason.

My point was that few programs are CPU-bound nowadays, believe it or not. For games, the bottleneck is graphics. For web browsing, the bottleneck is the network. When you're loading a program your disk is where it's slow. Maybe JS isn't really the best choice for file compression, audio/image processing, CAD work, etc.- but few people will say it is. For most user applications there's little technical reason why asm.js would give a really poor experience.

 

[niico]

What version of IE did you use for this?

Benchmarks weem way to slow to be IE 9/10 - which is on a par with modern browsers now.

 

[BubbleRaptor]

I'd honestly be more interested in a JS to asm.js optimizer. That is, a tool that can scan an existing JavaScript code base, optimize functions to use asm.js where possible, and suggest other functions that are close to being compatible.

I don't seem much utility in running huge apps from in the web but optimizing particularly computationally expensive code would be really helpful.

 

[DrPizza]

niico[/url]":210fh52f]What version of IE did you use for this?

Benchmarks weem way to slow to be IE 9/10 - which is on a par with modern browsers now.

Internet Explorer 10, in Windows 8.

 

[mathewaporter]

This is a huge step forward, will keep an eye on it as it moves forward.