🎁 🤞🏽 🥧 The price of JavaScript frameworks 🛅 🔳 💱

There is no faster way to slow down a site (such a pun) than using a bunch of JavaScript code on it. When using JavaScript, you have to pay for this with project performance at least four times. This is how the site’s JavaScript code loads user systems:

Download a file over the network.
Parsing and compiling unpacked source code after loading.
JavaScript code execution.
Memory consumption.

This combination is very expensive . And we are including more and more JS code in our projects. As organizations move towards sites based on frameworks and libraries like React, Vue, and others, we make the core functionality of sites very dependent on JavaScript.

I saw a lot of very heavy sites using JavaScript frameworks. But my vision of the issue is highly biased. The fact is that the companies I work with contact me precisely because they encounter complex problems in the field of website performance. As a result, I became curious to find out how widespread this problem is and what kind of “fines” we pay when we select one or another framework as the basis for a certain site.

The HTTP Archive project helped me figure this out .

Data

The HTTP Archive project, in total, tracks 4,308,655 links to regular sites dedicated to desktop devices, and 5,484,239 links to mobile sites. Among the many indicators associated with these links, there is a list of technologies found on the respective sites. This means that we can build a sample of thousands of sites that use various frameworks and libraries, and find out how much code they send to customers, and how much load the user systems create this code.

I collected data for March 2020, these were the latest data that I had access to.

I decided to compare aggregated HTTP Archive data for all sites with data for sites where React, Vue, and Angular were detected, although I was considering using other source material.

To make it more interesting - I also added sites that use jQuery to the source dataset. This library is still very popular. It also provides a website development approach that differs from the Single Page Application (SPA) model proposed by React, Vue, and Angular.

Links in the HTTP Archive representing sites that have discovered the use of technologies of interest to us

Framework or library	Links to mobile sites	Links to regular sites
jQuery	4615474	3714643
React	489827	241023
Vue	85649	43691
Angular	19423	18088

Hopes and dreams

Before we move on to data analysis, I want to talk about what I would like to hope for.

I believe that in an ideal world, frameworks should go beyond satisfying the needs of developers and give certain concrete advantages to ordinary users working with our sites. Performance is just one of these benefits. Here, accessibility and security still come to mind. But this is only the most important.

So, in an ideal world, a certain framework should facilitate the creation of a high-performance site. This should be done either due to the fact that the framework gives the developer a decent base on which to build the project, or due to the fact that it imposes restrictions on the development, puts forward requirements for it that complicate the development of something that turns out to be slow.

The best frameworks should do both: give a good base, and impose restrictions on the work, allowing to achieve a decent result.

Analysis of the median data values will not give us the necessary information. And, in fact, such an approach leaves a lot of important beyond our attention . Instead, I deduced from my data indicators expressed in percentiles. This is 10, 25, 50 (median), 75, 90 percentiles.

I am particularly interested in the 10th and 90th percentiles. The 10th percentile represents the best performance (or at least more or less close to the best) for a particular framework. In other words, this means that only 10% of sites using a particular framework reach this, or to a higher level. The 90th percentile, on the other hand, is the flip side of the coin — it shows us how bad things can be. The 90th percentile is the weaving sites - those last 10% of the sites that are characterized by the largest volumes of JS-code or the longest time required to process their code in the main stream.

JavaScript code volumes

To begin with, it makes sense to analyze the size of the JavaScript code transmitted by different sites over the network.

The amount of JavaScript code (Kb) transferred to mobile devices

	10	25	50	75	90
	93.4	196.6	413.5	746.8	1201.6
jQuery-	110.3	219.8	430.4	748.6	1162.3
Vue-	244.7	409.3	692.1	1065.5	1570.7
Angular-	445.1	675.6	1066.4	1761.5	2893.2
React-	345.8	441.6	690.3	1238.5	1893.6

JavaScript-,

JavaScript- (),

	10	25	50	75	90
	105.5	226.6	450.4	808.8	1267.3
jQuery-	121.7	242.2	458.3	803.4	1235.3
Vue-	248.0	420.1	718.0	1122.5	1643.1
Angular-	468.8	716.9	1144.2	1930.0	3283.1
React-	308.6	469.0	841.9	1472.2	2197.8

The amount of JavaScript code transmitted to desktop devices

If we talk only about the size of the JS code sent by sites to devices, then everything looks like you would expect. Namely, if one of the frameworks is used, this means that even in an ideal situation, the volume of the site’s JavaScript code will increase. This is not surprising - you can not make the JavaScript framework the basis of the site and expect that the volume of the JS-code of the project will be very low.

It is noteworthy in this data that some frameworks and libraries can be considered a more successful starting point of the project than others. JQuery sites look best. On desktop versions of sites they contain 15% more JavaScript than all sites, and on mobile versions - 18% more. (I must admit that here you can observe some distortion of the data. The fact is that jQuery is present on many sites, so it is natural that such sites are closer than others to the total number of sites. However, this does not affect the way source data is displayed for each framework.)

Although a 15-18% increase in code volume is a notable figure, comparing it with other frameworks and libraries, we can conclude that the “tax” charged by jQuery is very low. Sites on the Angular of the 10th percentile send 344% more data to desktop devices than all sites, and to mobiles - 377% more. React sites, the next in terms of severity, send 193% more code to desktop devices than all sites, and 270% more to mobile devices.

I have already mentioned that although the use of the framework means that a certain amount of code will be included in the project, at the very beginning of work on it, I hope that the framework is able to somehow limit the developer. In particular, we are talking about limiting the maximum amount of code.

Interestingly, jQuery sites follow this idea. Although they, at the 10th percentile level, are slightly heavier than all sites (15-18%), they, at the 90th percentile level, are slightly lighter than all - by about 3% in both desktop and mobile versions. This cannot be said to be a very significant benefit, but it can be said that jQuery sites at least do not differ in the huge size of JavaScript code, even in their largest versions.

But the same cannot be said about other frameworks.

As with the 10th percentile, the 90th percentile sites on Angular and React are different from other sites, but unfortunately they differ for the worse.

At the 90th percentile, Angular sites send 141% more data to mobile devices than all sites, and 159% more to desktop. React sites send 73% more to desktop devices than all sites, and 58% more to mobile. The code size of the React sites on the 90th percentile is 2197.8 Kb. This means that such sites send 322.9 Kb more data to mobile devices than their closest competitors based on Vue. The gap between the desktop sites based on Angular and React, and between other sites, is even greater. For example, desktop React sites send 554.7 KB more JS code to devices than similar Vue sites.

Time spent processing JavaScript code in the main thread

The above data clearly indicate that sites using the studied frameworks and libraries contain large amounts of JavaScript code. But, of course, this is only one part of our equation.

After the JavaScript code has arrived in the browser, it needs to be brought to a working state. Especially a lot of problems are caused by those actions that have to be carried out with the code in the main stream of the browser. The main thread is responsible for processing user actions, for calculating styles, for building and displaying the page layout. If you fill up the main stream with JavaScript-affairs, it will not be able to solve other tasks in time. This leads to delays and "brakes" in the work of the pages.

The HTTP Archive database contains information on how much time it takes to process JavaScript code in the main thread of the V8 engine. This means that we can collect this data and find out how much time the main thread takes to process the JavaScript of various sites.

CPU time (in milliseconds) related to script processing on mobile devices

Percentiles	10	25	fifty	75	90
All sites	356.4	959.7	2372.1	5367.3	10485.8
jQuery sites	575.3	1147.4	2555.9	5511.0	10349.4
Vue Sites	1130.0	2087.9	4100.4	7676.1	12849.4
Angular sites	1471.3	2380.1	4118.6	7450.8	13296.4
React Sites	2700.1	5090.3	9287.6	14509.6	20813.3

CPU time related to script processing on mobile devices

CPU time (in milliseconds) relating to script processing on desktop devices

Percentiles	10	25	fifty	75	90
All sites	146.0	351.8	831.0	1739.8	3236.8
jQuery sites	199.6	399.2	877.5	1779.9	3215.5
Vue-	350.4	650.8	1280.7	2388.5	4010.8
Angular-	482.2	777.9	1365.5	2400.6	4171.8
React-	508.0	1045.6	2121.1	4235.1	7444.3

CPU time related to processing scripts on desktop devices

Here you can see something very familiar.

For starters, sites with jQuery spend significantly less on JavaScript processing in the main thread than others. At the 10th percentile, in comparison with all sites, jQuery sites on mobile devices spend 61% more time processing JS code in the main thread. In the case of desktop jQuery sites, processing time is increased by 37%. At the 90th percentile level, jQuery site metrics are very close to aggregated metrics. Namely, jQuery sites on mobile devices spend 1.3% less time in the main thread than all sites, and on desktop devices - 0.7% less time.

On the other side of our rating, there are frameworks that are characterized by the greatest load on the main thread. This, again, is Angular and React. The only difference between them is that although Angular sites send larger amounts of code to browsers than React sites, it takes less processor time to process the code of Angular sites. Far less.

At the 10th percentile, desktop Angular sites spend 230% more main thread time processing JS code than all sites. For mobile sites, this figure is 313%. React sites have the worst performance. On desktop devices, they spend 248% more time on code processing than all sites, and on mobile devices - 658% more. 658% is not a typo. At the 10th percentile, React sites spend 2.7 seconds of the main thread time to process the code they have.

The 90th percentile, when compared with these huge numbers, looks at least slightly better. Angular projects, compared with all sites, spend 29% more time on desktop devices in the main thread, and 27% more on mobile devices. In the case of React sites, similar indicators look, respectively, as 130% and 98%.

Percentage deviations for the 90th percentile look better than similar values for the 10th percentile. But here it is worth remembering that the numbers indicating the time seem pretty scary. Say - 20.8 seconds in the main stream of a mobile device for a site built on React. (I believe that the story of what actually happens during this time is worthy of a separate article).

There is one potential difficulty (thank youJeremy for drawing my attention to this feature, and for carefully considering the data from this point of view). The fact is that many sites use several front-end tools. In particular, I have seen many sites that use jQuery together with React or Vue, as these sites migrate from jQuery to other frameworks or libraries. As a result, I again turned to the database, choosing this time only those links that correspond to sites that use only React, jQuery, Angular or Vue, but not some combination of them. That's what I did.

CPU time (in milliseconds) related to the processing of scripts on mobile devices in a situation where sites use only one framework or only one library

Percentiles	10	25	fifty	75	90
, jQuery	542.9	1062.2	2297.4	4769.7	8718.2
, Vue	944.0	1716.3	3194.7	5959.6	9843.8
, Angular	1328.9	2151.9	3695.3	6629.3	11607.7
, React	2443.2	4620.5	10061.4	17074.3	24956.3

CPU time related to the processing of scripts on mobile devices in a situation where only one framework is used on sites, or only one library

At first, it is not surprising: when only one framework or one library is used on a site, the performance of such a site is more often improving than not improving. The indicators for each instrument look better at 10 and 25 percentiles. It makes sense. A site that is made using one framework should be more productive than a site that is made using two or more frameworks or libraries.

In fact, the indicators for each studied front-end instrument look better in all cases, minus one curious exception. I was surprised that at the 50th percentile level and higher, sites using React have worse performance if React is the only library used on them. This, by the way, was the reason that I bring here this data.

This is a little strange, but I still try to look for an explanation of this oddity.

If a project uses both React and jQuery, then this project is most likely halfway through the transition from jQuery to React. Perhaps it has a code base in which these libraries are mixed. Since we have already seen that jQuery sites spend less time in the main thread than React sites, this may tell us that the implementation of some functionality on jQuery helps to slightly improve the performance of the site.

But as the project, moving from jQuery to React, relies more on React, the situation changes. If the site is made really high quality, and the developers of the site use React prudently, then with this site everything will be fine. But for the average React site, the widespread use of React means that the main thread is under increased load.

The gap between mobile and desktop devices

Another point of view from which I looked at the data under study was to study how wide the gap between working with sites on mobile and desktop devices. If we talk about comparing the volumes of JavaScript code, then such a comparison does not reveal anything terrible. Of course, it would be nice to see smaller amounts of downloaded code, but there is not much difference in the volumes of mobile and desktop code.

But if you analyze the time required to process the code, you will notice a very large gap between mobile and desktop devices.

The increase in time (in percent) related to the processing of scripts on mobile devices compared to desktop

Percentiles	10	25	fifty	75	90
All sites	144.1	172.8	185.5	208.5	224.0
jQuery sites	188.2	187.4	191.3	209.6	221.9
Vue Sites	222.5	220.8	220.2	221.4	220.4
Angular sites	205.1	206.0	201.6	210.4	218.7
React Sites	431.5	386.8	337.9	242.6	179.6

Although some difference in the processing speed of the code between the phone and the laptop is quite expected, such large numbers tell me that modern frameworks are not sufficiently focused on low-power devices, and on the desire to close the gap. Even at the 10th percentile, React sites spend 431.5% more time in the main stream of mobile devices than in the main stream of desktop devices. The smallest gap is observed in jQuery, but even here the corresponding indicator is 188.2%. When website developers make their projects so that it takes more processor time to process them (and this happens, and with time everything gets worse), owners of low-power devices have to pay for it.

Summary

Good frameworks should provide developers with a quality base for creating web projects (in terms of security, availability, performance), or should have built-in restrictions that make it difficult to create something that violates these restrictions.

This does not seem to apply to the performance of web projects (and, apparently, to their availability ).

It is worth noting that the fact that React- or Angular-sites spend more CPU time on code preparation than others does not necessarily mean that React-sites in the course of work load the processor more than Vue-sites. In fact, the data we have reviewed says very little about the performance of frameworks and libraries. They talk more about development approaches to which, consciously or not, these frameworks can push programmers. We are talking about documentation for frameworks, about their ecosystem, about common development techniques.

It’s worth mentioning that we didn’t analyze it here, namely, how much time the device spends to execute JavaScript code when navigating between pages on the site. The argument in favor of SPA is that as soon as a single-page application is loaded into the browser, the user, theoretically, will be able to open the pages of the site faster. My own experience tells me that this is far from a fact. But we do not have data to clarify this issue.

It is only clear that if you use a framework or library to create a site, you compromise in terms of initial loading of the project and preparing it for work. This applies even to the most positive scenarios.

In appropriate situations, it is quite possible to make some compromises, but it is important that the developers make conscious compromises.

But we have reason to be optimistic. I’m encouraged by how closely Chrome developers interact with some of the front-end tools we've reviewed in an effort to help improve the performance of these tools.

However, I am a pragmatic person. New architectures as often create performance problems as they solve them. And it takes time to fix the flaws. Just as we should not expect new network technologies to solve all performance problems, we should not expect this from new versions of our favorite frameworks.

If you want to use one of the front-end tools discussed in this material, this means that you will have to make additional efforts to ensure that, in the meantime, not harm the productivity of your project. Here are some considerations to consider before starting to use the new framework:

Test yourself in terms of common sense. Do you really need to use the chosen framework? Pure JavaScript is capable of a great deal today.
Is there an easier alternative to the chosen framework (like Preact, Svelte, or something else) that can give you 90% of the capabilities of this framework?
If you are already using some kind of framework, think about whether there is something that offers better, more conservative, standard parameters (for example, Nuxt.js instead of Vue, Next.js instead of React, and so on).
What will be your JavaScript performance budget ?
How can you limit the development process in order to complicate the implementation of a larger amount of JavaScript code into the project than is absolutely necessary?
If you use the framework for development convenience, consider whether you need to send the framework code to your clients. Maybe you can solve all the problems on the server?

Usually, these ideas are worth a look regardless of what exactly you have chosen to develop the frontend. But they are especially important when you are working on a project that, from the very beginning, lacks productivity.

Dear readers! How do you see the perfect JavaScript framework?