Browser Benchmark: New Chromium-based Edge vs. Edge, Chrome & Firefox

In December 2018 Microsoft announced, to the surprise of many, that they would be replacing their existing Edge browser with a new version based on the Chromium open-source browser project. Other browsers based on Chromium include Google’s Chrome, Opera, and Brave. Microsoft also announced that they would be actively contributing to the Chromium project, which appears to be the case, with over 275 commits as of April 8, 2019.

The replacement for the original Edge browser, hereafter referred to as Chromium-Edge, is still in early development, with no beta release available as of late-April 2019. However, being based on a mature engine like Chromium, it is still and loads most sites without issue already. What is currently offered via the Microsoft Edge Insider site, are versions for Windows 10 in a Dev build channel with targeted weekly updates, and a Canary channel offering daily builds. There is a Beta channel listed, suggesting updates every six weeks, but so far no beta is available. The Insider site also lists Windows 7, 8, 8.1 and macOS as platform targets coming soon. That last bit is interesting since Microsoft hasn’t developed a broswer for Mac since Internet Explorer 5 Mac Edition’s last update in 2003.

Given that this is a significant shift in browser strategy for Microsoft, I decided to test what the performance impact might be for Windows 10 users. Using a variety of benchmarks, I tested the new Chromium-based Edge browser against the current releases of Google Chrome (73.0), Mozilla Firefox (66.0), and the EdgeHTML-based version of Microsoft Edge (44.17763) in Windows 10 1809. Microsoft has only released early builds of Edge in its Chromium flavor, so I tested the initial Dev release (74.1) and one of the daily Canary builds (75.0.134). The benchmarks used for this test are:

  • Google Octane 2.0
  • Sunspider 1.02
  • Mozilla Kraken 1.1
  • Speedometer 2.0
  • JetStream 2

Each test was run three times and an average of the scores taken. Tests were run after a reboot, with ample time for startup tasks to complete and the computer to be in an idle state with minimal background tasks running. Each browser was started in its private mode with no active extensions, the benchmark page loaded, and then the browser was allowed to sit for about 30 seconds before starting the benchmark.

The test system was configured as follows: Intel Core i7-3770K (3.7GHz base/4.1GHz turbo), 32GB DDR3-1600, 500GB SSD, Nvidia GeForce GTX 950, and Windows 10 v1809 with latest updates.

Google Octane 2.0

Google Octane 2.0 results

Octane 2.0 uses 17 different tests based on various real world scenarios, including code loading, regular expressions, a Gameboy emulator, PDF reader, and complex TypeScript. The Dev and Canary builds of Chromium-Edge both see a massive 41% improvement in the Octane 2.0 benchmark over the classic Edge. They also hold a slight 2-3% lead over Chrome, while Firefox sits in the middle of the three rendering engines, about 20 percent behind the leaders.

Sunspider 1.02

While this benchmark has been deprecated, it was run just to get a wider view of browser performance. It is based on older specific JavaScript functions and does not accurately represent more complex modern web applications. Despite this, Edge pulled out a big win over the other browsers.

Mozilla Kraken 1.1

With Mozilla Kraken, another older benchmark that runs a variety of use cases, all of the browsers are within 4% of each other, except for classic Edge, which is about 26% slower than the leader in this case, Firefox.

Speedometer 2.0

In Speedometer 2.0, a benchmark that tests modern web app responsiveness using the same app implemented in different versions of JavaScript and various modern frameworks, there is a significant gap between the Chromium based browsers and the others. Chromium-Edge beats Chrome slightly, by a couple of percent, while soundly beating Firefox by about 36% and classic Edge by a whopping 81%.

Jetstream 2

Jetstream 2 is a newer benchmark that tests more advanced web application functionality, including web assembly and web workers, in addition to using some functionality from previous benchmark suites such as Octane 2, Jetstream 1, and Sunspider. It uses 64 tests and appears to be more difficult optimize for, as opposed to older tests such as Sunspider and Octane 1 where browser vendors would target those benchmarks specifically for optimizations to obtain better scores. In Jetstream 2, the Chromium based browsers are once again far ahead of the competition. Classic Edge was not able to complete the benchmark after repeated attempts. The Canary build of Chromium-Edge was about 68% faster than Firefox, while 5.7% faster than Chrome and 2.8% faster than the Dev build based on Chromium 74.

Conclusion

In switching from EdgeHTML to Chromium as a base for its browser, Microsoft appears to have gotten a significant performance uplift in more advanced browser benchmarks and applications. This still affords Microsoft the opportunity to implement customizations that make Edge unique, while contributing to the underlying open source project and ensuring that the newest web technologies are supported.

For Google and the Chromium project, it significantly broadens the scope of users of Chromium-based browsers, as Edge ships in Windows 10, and broadens participation in the project that should result in improvements for all of those users. However, this does reduce the number of competing browser engines, and will likely further consolidate the number of users on Chromium-based browsers. Per Netmarketshare, as of March 2019, Chrome and Opera combined had 67% market share. If the Edge and a fair number of Internet Explorer holdouts convert, Chromium could easily surpass 75% market share.

Given that there is not even a beta release of Chromium-Edge yet, EdgeHTML will be around for quite a while. However, the stability of the current Dev build suggests that adventurous users should have no problem in using it daily, though given the weekly build cadence, some caution is recommended. EdgeHTML also underpins Progressive Web Apps (PWA) in Windows 10, and the timeline for replacing the engine there is unknown.

Leave a Reply

Your email address will not be published. Required fields are marked *