JavaScript DOM Manipulation Performance - DiVA portal

JavaScript DOM ManipulationPerformance

Comparing Vanilla JavaScript and Leading JavaScriptFront-end Frameworks

Morgan Persson

28/05/2020

Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology inpartial fulfillment of the requirements for the bachelor’s degree in software engineering. Thethesis is equivalent to 10 weeks of full-time studies.

Contact Information:Author(s):Morgan PerssonE-mail: [email protected]

University advisor:Emil FolinoDepartment of Computer Science

Faculty of Computing Internet : www.bth.seBlekinge Institute of Technology Phone : +46 455 38 50 00SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

Abstract

Background. Websites of 2020 are often feature rich and highly interactive ap-plications. JavaScript is a popular programming language for the web, with manyframeworks available. A common denominator for highly interactive web applica-tions is the need for efficient methods of manipulating the Document Object Modelto enable a solid user experience.Objectives. This study compares Vanilla JavaScript and the JavaScript frameworksAngular, React and Vue.js in regards to DOM performance, DOM manipulationmethodology and application size.Methods. A literature study was conducted to compare the DOM manipulationmethodologies of Vanilla JavaScript and the selected frameworks. An experimentwas conducted where test applications was created using Vanilla JavaScript and theselected frameworks. These applications were used as base for comparing applica-tion size and for comparison tests of DOM performance related metrics using GoogleChrome and Firefox.Results. In regards to DOM manipulation methodology, there is a distinct differencebetween Vanilla JavaScript and the selected frameworks. In Vanilla JavaScript DOMmanipulation is handled by direct interaction with the DOM interface. When usingthe selected frameworks the actual interaction with the DOM interface is abstractedaway from the developer and handled by the framework. While React and Vue.jsboth have implemented a Virtual DOM to optimize DOM interactions, Angular hasimplemented Incremental DOM. Vanilla JavaScript had the best DOM performancein all tests and the smallest application size. Amongst the frameworks React hadthe best DOM performance, Angular performed close to React in nearly all test, andVue.js was slightly slower in most tests. In nearly all tests the applications performedbetter in Google Chrome.Conclusions. Vanilla JavaScript and the selected frameworks, and thereby theirDOM manipulation methodologies, are all feasible alternatives for creating inter-active web applications with high DOM performance. Tests indicate that VanillaJavaScript and the selected frameworks achieves better DOM performance in GoogleChrome compared to Firefox.

Keywords: JavaScript, Framework, DOM performance

Contents

Abstract i

1 Introduction 11.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 32.1 Document Object Model . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 CSS Object Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Rendering the HTML document in the Browser . . . . . . . . . . . . 5

2.3.1 First render . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3.2 Repaint and reflow . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 Vanilla JavaScript and the selected frameworks . . . . . . . . . . . . 72.4.1 Vanilla JavaScript . . . . . . . . . . . . . . . . . . . . . . . . . 72.4.2 Angular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4.3 React . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4.4 Vue.js . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Research Questions 93.1 RQ 1: How does the DOM manipulation methodology of Vanilla

JavaScript and the selected frameworks compare? . . . . . . . . . . . 93.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.1.2 Expected Outcome . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 RQ 2: How does initial page rendering time of Vanilla JavaScript andthe frameworks compare using Google Chrome and Firefox? . . . . . 93.2.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.2 Expected Outcome . . . . . . . . . . . . . . . . . . . . . . . . 10

3.3 RQ 3: How does Vanilla JavaScript and the frameworks compare whenit comes to DOM manipulation performance using Google Chrome andFirefox? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.2 Expected Outcome . . . . . . . . . . . . . . . . . . . . . . . . 10

3.4 RQ 4: How does Vanilla JavaScript and the frameworks compare whenit comes to application size? . . . . . . . . . . . . . . . . . . . . . . . 103.4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4.2 Expected Outcome . . . . . . . . . . . . . . . . . . . . . . . . 11

ii

4 Method 124.1 Literature Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.2 Empirical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2.1 Experiment design . . . . . . . . . . . . . . . . . . . . . . . . 124.2.2 Test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2.3 Test script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2.4 Test applications . . . . . . . . . . . . . . . . . . . . . . . . . 154.2.5 Deployment of test script and test applications . . . . . . . . . 16

5 Results 175.1 DOM manipulation methodology comparison . . . . . . . . . . . . . . 17

5.1.1 Virtual DOM . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.1.2 Incremental DOM . . . . . . . . . . . . . . . . . . . . . . . . . 18

5.2 Performance comparison . . . . . . . . . . . . . . . . . . . . . . . . . 185.2.1 Initial page render . . . . . . . . . . . . . . . . . . . . . . . . 185.2.2 Create 10000 elements . . . . . . . . . . . . . . . . . . . . . . 205.2.3 Update all elements . . . . . . . . . . . . . . . . . . . . . . . . 225.2.4 Update every other element . . . . . . . . . . . . . . . . . . . 245.2.5 Delete all elements . . . . . . . . . . . . . . . . . . . . . . . . 26

5.3 Application size comparison . . . . . . . . . . . . . . . . . . . . . . . 28

6 Analysis and Discussion 316.1 RQ 1: How does the DOM manipulation methodology of Vanilla

JavaScript and the selected frameworks compare? . . . . . . . . . . . 316.2 RQ 2: How does initial page rendering time of Vanilla JavaScript and

the frameworks compare using Google Chrome and Firefox? . . . . . 326.3 RQ 3: How does Vanilla JavaScript and the frameworks compare when

it comes to DOM manipulation performance using Google Chrome andFirefox? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

6.4 RQ 4: How does Vanilla JavaScript and the frameworks compare whenit comes to application size? . . . . . . . . . . . . . . . . . . . . . . . 36

6.5 Results seen from a sustainability and societal perspective . . . . . . 37

7 Validity Threats 38

8 Conclusion 39

9 Future Work 40

References 41

A Empirical study links 47

iii

Chapter 1Introduction

Since its inception in 1995 JavaScript has evolved into one of the fundamental pro-gramming languages for the Web [68, 18]. With the rise in JavaScript usage andpopularity the language has gained traction in open-source communities, leading toan abundance of open-source libraries and frameworks being developed over the years[18]. The pace of evolution is high in the current JavaScript framework landscape,where existing frameworks are rapidly evolving and at the same time new frameworkssee the light of day each year [2].

For many years the average web page size has increased from year to year [9]. Sizeincreases as websites get more feature rich and interactive. At the same time, per-formance is key to deliver a satisfactory user experience, both in regards of a fastinitial page load time as well as timely responding to user interactions with as lowperceived waiting time as possible [70]. For a website the Page Load Time metric iscritical for several reasons. It has a direct impact on user experience [39] and Googlestatistics shows that 53% of users accessing a website from a mobile device leaves apage that takes longer than three seconds to load [3]. Google has acknowledged thatsite speed is a parameter in their search ranking algorithms [37, 71]. A significantpart of the Page Load Time is the Page Render Time, measuring the amount of timea page render took in the Internet browser [16].

Internet usage has rapidly increased ever since the early 1990s [7], and now morethan half of the world population is online [11, 61]. At the same time the Inter-nets environmental impact has increased, and the Internet now is on par with theaviation industry when it comes to CO2 footprint [44, 59]. The share of global elec-tricity usage that can be ascribed to communication technology is expected to reach21% by year 2030 [1]. As web page sizes increase the metrics Page Load Time andPage Render Time has a larger impact on energy usage on the client side. It alsocontributes to the digital divide as users with poor internet connection speed mightexperience problems using the web site and using low-performing or outdated hard-ware can amplify this effect.

With the rapid evolution taking place in the JavaScript front-end framework land-scape comparisons tend to get outdated quickly, and although several JavaScriptfront-end framework comparisons exists [19] [55], many of them are outdated anddoes not compare the combination of frameworks and areas for comparison as thisstudy.

1

1.1 ScopeThis study compares DOM performance in Vanilla JavaScript and a selection offront-end frameworks using the browsers Google Chrome and Firefox.

IEEE Standard 610[24] defines performance as “The degree to which a system orcomponent accomplishes its designated functions within given constraints, such asspeed, accuracy, or memory usage.”. The performance metrics compared in this the-sis are all directly related to speed.

This study compares Vanilla JavaScript and the following front-end frameworks:

• vue.js

• React

• Angular

Comparing other aspects is out of scope of this study, but comparing several otheraspects such as functionality, learnability, maintainability and more are importantwhen selecting a JavaScript front-end framework.

The frameworks selected are the three most loved and wanted JavaScript front-endframeworks on the market according to Stack Overflow Developer survey 2019 [56].The selection of Google Chrome and Firefox was based on their positions as the 2most popular desktop web browsers[38].

1.2 PurposeThe purpose of this study is to contribute with insights regarding DOM manipula-tion methodology and performance for Vanilla JavaScript and the selected frame-works. Comparing DOM performance in vanilla JavaScript and a selection of front-end frameworks using different Internet browsers will contribute with insights thatare especially valuable to organisations operating content-heavy websites where thecontent is dynamically rendered and updated. It will also contribute to end-users asthe comparison extends to using different Internet browsers.

2

Chapter 2Background

2.1 Document Object ModelA key enabler of interactive websites is the Document Object Model (DOM). TheDOM is an API for HTML and XML documents [14, 48]. Conceptually the DOMcan be represented as a tree. The DOM tree consists of nodes and objects, origi-nating from the root node (the document itself) child nodes branch out to form thestructure of the document. The nodes have properties and methods which allowsfor manipulation of the nodes themselves. Using the DOM programmers can createdocuments, navigate the documents hierarchy, and modify the content of the docu-ment by adding, removing or updating nodes and content. These modifications canbe done using a scripting language such as JavaScript.

(a) HTML document (b) DOM tree

Figure 2.1: Example HTML document represented as a Document Object Modeltree

Figure 2.1 depicts how a HTML document can be represented as a Document ObjectModel tree. The root element of a HTML document is the HTML element. In theexample in figure 2.1 the HTML element has 2 children, head and body which intheir turns have child elements. For client-side rendered JavaScript applications thecommon approach is to provide the browser with a bare-bones HTML document

3

with linked JavaScript file/s which handles the rendering of the complete applicationusing the DOM API.

2.2 CSS Object ModelThe CSS Object Model is a programming interface for CSS [13] allowing programmersto read and modify CSS styles using JavaScript. The CSS Object Model is createdwhen the browser has finished constructing the Document Object Model. It thenparses CSS from all the sources related to the document and creates the CSS ObjectModel. The CSSOM and DOM are separate data models and are parsed separately,but they share a similar tree-like structures and the purpose of both models is toenable the browser to compute the layout of the document. A document can havemultiple CSS sources, and there are multiple types of sources. Examples of differenttypes of sources include CSS embedded in the document using the <style> tag,CSS specified inline on the HTML element, or external sources linked to the HTMLdocument. If a document does not have any CSS sources it will be rendered usingonly the browsers default styles, called user-agent styles. Any CSS styles defined ina documents CSS sources will override the browsers default styles.

(a) HTML document (b) CSS document

Figure 2.2: Example HTML document linked to a CSS style document

In figure 2.2a a stylesheet link to an external CSS file is added to the example HTMLdocument. Figure 2.2b shows the content of the linked CSS file, which apply stylesto body, h1 and p elements.

4

Figure 2.3: Documents in figure 2.2 represented as a CSSOM tree

Figure 2.3 depicts how the CSS rules defined in the example in figure 2.2 are modeledinto a CSSOM tree. The figure shows how children elements inherit some CSSproperties from its parents. Inherited styles are marked with blue text in the figure.

2.3 Rendering the HTML document in the Browser

2.3.1 First render

In order to render a HTML document the browser goes through a multi-step process[22, 23, 53]:

1. The HTML document is downloaded from the server and parsed into the DOM.

2. External CSS sources referenced in the HTML document are downloaded andare together with styles defined in the HTML document parsed into the CS-SOM.

3. The DOM and CSSOM are combined into a rendering tree, containing only thenodes required to render the page.

4. Layout (also referred to as reflow) stage computes the size and position for eachof the visible elements in the HTML document.

5. The browser paints the HTML page and the result is displayed in the browser.

5

(a) Page rendered without de-fined styles

(b) Page rendered with theexample CSS file

Figure 2.4: Documents in figure 2.1 and 2.2 rendered in Chrome web browser

Figure 2.4 shows the example HTML document rendered in Chrome both withoutdefined CSS styles (a), and with the example CSS file (b).

2.3.2 Repaint and reflow

After the initial page render has completed the rendering process, or parts of it, willbe redone upon a user or script interactions with the page Depending on the typeof interaction, i.e. depending on what has changed on the page, one of the two subrendering processes referred to as repaint and reflow will happen.

A repaint happens when an interaction leads to element style changes where theelements position is unaffected. As the name suggests, a repaint paints the elementin the browser window according to the updated style changes. Examples of changesresulting in repaint of an element is changes to an elements visibility, backgroundand color.

A reflow is triggered when changes affects the layout of the page, i.e when one ormore elements has changed position on the page. The reflow process recalculates theposition and size of all elements. As reflow is a user-blocking operation developersaim to minimize reflow processing time and occurrences [54].

Example of changes that triggers a reflow:

• DOM manipulation such as adding, updating and deleting elements

• Updating elements CSS class attribute

• Browser window resizing and scrolling

6

2.4 Vanilla JavaScript and the selected frameworksIn this study Vanilla JavaScript is compared to a selection of JavaScript frame-works in regards to DOM manipulation performance and application size. There area multitude of JavaScript frameworks available on the market, and for this studythe selected frameworks are the three most loved and wanted JavaScript front-endframeworks according to Stack Overflow Developer survey 2019 [56].

2.4.1 Vanilla JavaScript

JavaScript is a programming language that was first released in 1995. It was originallyused in front-end development to bring interactivity to web sites, and while that isstill the most common use of JavaScript it has in recent years also been possible to useJavaScript on the back-end. This thesis focus exclusively on client-side JavaScript,that is executed in the browser on the client. JavaScript conforms to ECMA Inter-nationals standard ECMA-262[35] commonly referred to as ECMAScript. In thesisVanilla JavaScript is defined as standard JavaScript without any external librariesor frameworks.

2.4.2 Angular

Angular as a framework has been around the longest of the compared frameworks,with the initial release of AngularJS taking place in 2010. Since then the frameworkhas evolved significantly, and a complete re-write of the framework was launched 2016referred to as Angular2 or just Angular. The framework is developed and maintainedby Google who themselves uses Angular in web applications like Google Cloud Plat-form and AdWords.

On the Angular page on Googles Open Source website the framework is describedwith the following words [20]:

"Angular is a development platform that aims to make web development feel ef-fortless, focused on developer productivity, speed and testability. Applications builtwith Angular can be deployed to mobile devices and desktops as websites and nativeapplications."

2.4.3 React

React was initially released as an open-source project in 2013 by Facebook, whooriginally developed it and continues to maintain it with the support of the Reactcommunity. Out of the three selected frameworks React is considered the most lovedby stackoverflow.com users [56]. React has a strong footprint among tech enterpriseswith companies such as Uber, Netflix, Reddit and Paypal all using the framework.

On the React GitHub page the frameworks is described with the following words[46]:

7

"React is a JavaScript library for building user interfaces.

Declarative: React makes it painless to create interactive UIs. Design simple viewsfor each state in your application, and React will efficiently update and render justthe right components when your data changes. Declarative views make your codemore predictable, simpler to understand, and easier to debug.

Component-Based: Build encapsulated components that manage their own state,then compose them to make complex UIs. Since component logic is written inJavaScript instead of templates, you can easily pass rich data through your appand keep state out of the DOM.

Learn Once, Write Anywhere: We don’t make assumptions about the rest of yourtechnology stack, so you can develop new features in React without rewriting existingcode. React can also render on the server using Node and power mobile apps usingReact Native."

2.4.4 Vue.js

Vue.js is an open-source project that was initially released in February 2014. It wascreated by Evan You, who is still active in the team maintaining the framework.Unlike React and Angular, Vue.js is not backed by a large company and instead re-lies on crowdfunding via Patreon. According to 2019 JavaScript Rising Stars report,which tracks the number of stars added on GitHub for amongst other the selectedframeworks, Vue.js got more than 31400 stars added during 2019 which is more thanany other front-end framework [52].

On the Vue.js GitHub page the frameworks is described with the following words[62]: "Vue (pronounced /vju:/, like view) is a progressive framework for buildinguser interfaces. It is designed from the ground up to be incrementally adoptable,and can easily scale between a library and a framework depending on different usecases. It consists of an approachable core library that focuses on the view layer only,and an ecosystem of supporting libraries that helps you tackle complexity in largeSingle-Page Applications."

8

Chapter 3Research Questions

3.1 RQ 1: How does the DOM manipulation method-ology of Vanilla JavaScript and the selected frame-works compare?

3.1.1 Motivation

For client-side JavaScript applications DOM manipulation is a key ability that en-ables dynamic manipulation of HTML and CSS resources. However, DOM manipu-lation is expensive and the performance cost can potentially have a negative impacton the user experience. DOM manipulation is central to all the selected frameworks,but the frameworks have taken different approaches to implementing DOM manipu-lation. Providing an answer to this questions can give insights to better understandany differences in DOM manipulation performance.

3.1.2 Expected Outcome

The author expects that Vanilla JavaScript and the frameworks all offer efficientmethods of manipulating the DOM. Furthermore Vanilla JavaScript and the frame-works are expected to have taken different approaches when it comes to DOM ma-nipulation methodology.

3.2 RQ 2: How does initial page rendering time ofVanilla JavaScript and the frameworks compareusing Google Chrome and Firefox?

3.2.1 Motivation

The initial Page Load Time [16] has a significant impact on the users likeliness toabandon the site in search of faster alternatives [3]. Page Rendering Time is a partof the Page Load Time metric and measures the time taken to render the page inthe browser. Answering this question will provide insights regarding how VanillaJavaScript and the selected frameworks compares when it comes to Page RenderTime, and potential differences in results using Google Chrome and Firefox.

9


The author’s expectation is that Vanilla JavaScript will achieve lower initial page ren-dering times mainly because of its expected smaller application size. Regarding theframeworks the expectation is that they all will achieve low initial page render timesat similar levels. The browsers are expected to show a similar level of performance.

3.3 RQ 3: How does Vanilla JavaScript and theframeworks compare when it comes to DOMmanipulation performance using Google Chromeand Firefox?

3.3.1 Motivation

Modern client-side web applications are often content-rich and highly interactive witha common dependency to efficient DOM manipulation. DOM manipulations includecreating, updating and deleting DOM elements and by answering this question in-sights into how Vanilla JavaScript and the selected frameworks compares in this areais provided.


The author’s expectation is that Vanilla JavaScript will have better DOM perfor-mance than the frameworks. Regarding the frameworks the expectation is that allwill exhibit a high level of DOM performance, and the frameworks DOM perfor-mance levels will be similar with no framework performing better than the others inall tests. The browsers are expected to show a similar level of performance.

3.4 RQ 4: How does Vanilla JavaScript and theframeworks compare when it comes to applica-tion size?

3.4.1 Motivation

Application size is one parameter impacting performance. As application size in-creases so does the time needed for network transfer and processing on the clientside, especially for users with poor internet connection speeds. A framework addspayload size to the application, and by answering this question insights into howthe minimum additional payload size compares between the selected frameworks andVanilla JavaScript is gained.

10


The author’s expectation is that Vanilla JavaScript will have a significantly smallerapplication size than the frameworks. Amongst the frameworks the expectation isthat they will achieve similar applications sizes.

11

Chapter 4Method

4.1 Literature StudyTo answer RQ1 a literature study is conducted focusing on how DOM management ishandled by Vanilla JavaScript and the selected frameworks. Furthermore informationon how to create applications using the technologies are collected. The literaturestudy collects information mainly from official websites [47, 65, 6, 15, 22, 66, 14],well-known and trusted websites and web developer communities on the Internet[58, 57, 36] and from results found searching BTH Summon using combinations ofthe following search terms:

• JavaScript

• Document Object Model

• JavaScript Framework Comparison

• JavaScript Framework DOM

• JavaScript Framework Performance

• JavaScript Framework Rendering

4.2 Empirical StudyRQ2, RQ3 and RQ4 is answered by conducting an experiment.

4.2.1 Experiment design

In the experiment an identical web application is created using Vanilla JavaScript[15], vue.js [65], React [47] and Angular [6]. These applications are used as basis forperforming a number of comparative test cases in pursuit of answers to the researchquestions. The test suite is run by a test script written in Python and uses Selenium[51] and webdrivers for Chrome [8] and Gecko [12] to emulate Google Chrome andFirefox browsers. Each test case is performed 100 times per browser.

The experiment has been designed with reproducibility in focus. To ensure a highdegree of reproducibility the following steps is taken:

12

• Test process is fully automated, which increases reproducibility and allows forsignificant number of test iterations to be performed and it reduces the risk ofmanual errors.

• Build versions of the test applications is containerized using Docker [28], andthe images is published on Docker Hub available for anyone to use.

• Test applications are deployed as Docker containers to a standard Amazon WebServices EC2 [25] instance that is used as Docker host.

• Test script is deployed to and run from a standard Amazon Web Services EC2[25] instance that has network access to the EC2 instance running the testapplications.

• All application repositories and the test script is made publicly available onGitHub [32].

4.2.2 Test cases

In order to provide answers to RQ2 and RQ3 a number of test cases is performed.

1. Initial page render time

2. Create 10000 elements

3. Update all elements

4. Update every other element

5. Delete all elements

To answer RQ4 "How does Vanilla JavaScript and the frameworks compare when itcomes to application size?", a size comparison of the test applications will be doneby comparing test applications sizes using Google Chrome network tab.

Test case 1

This test is designed to provide answer to RQ2, "How does initial page renderingtime of Vanilla JavaScript and the frameworks compare using Google Chome andFirefox?". In this test the initial page render time is measured for each applicationusing both Google Chrome and Firefox.

The necessary metrics to calculate Page Render Time [16] are available in the Nav-igation timing API [67]. Figure 4.1 show the timing attributes defined in Perfor-manceTiming and PerformanceNavigation interface [67]. Using this interface it ispossible to measure the different stages involved in loading a web page in a browser.This test measures Page Render Time, which is visualized in the figure as the boxnamed "Processing". The Page Render Time metric is calculated as domComplete -domLoading [16].

13

Figure 4.1: Illustration from W3.org showing the timing attributes defined in Per-formanceTiming and PerformanceNavigation interface [67]

Test cases 2-5

These test cases are designed to provide an answer to RQ3, "How does VanillaJavaScript and the frameworks compare when it comes to DOM manipulation per-formance using Google Chrome and Firefox?".

Test methodology for test case 2-5 Test case 2-5 are orchestrated using meth-ods in the Performance interface [17], that is implemented in both Google Chromeand Firefox, to measure the time duration of the test cases. Just before a test case isinitiated a timestamp is created using the performance.mark() method which servesas the starting mark of the test case. Right after the test case is finished anothertimestamp is created using the same method, serving as end mark of the test case.By using the performance.measure() method the distance between the start and endmark is measured, providing the time duration for the test case.

Test case 2 - Create 10000 elements. In this test the applications creates 10000 paragraph elements in a containing div element on the page, and the performanceis measured by time duration to perform the operation.

Test case 3 - Update all elements. In this test the applications updates all10 000 elements in the containing div element on the page, and the performance is

14

New element

Figure 4.2: HTML code for elements created in test case 2.

measured by time duration to perform the operation.

Updated all

Figure 4.3: HTML code of the updated elements in test case 3.

Test case 4 - Update every other element. In this test the applications updatesevery other element in the containing div element on the page, and the performanceis measured by time duration to perform the operation.

Updated half

Figure 4.4: HTML code of the updated elements in test case 4.

Test case 5 - Delete all elements. In this test the applications deletes all 10000elements in the containing div element, and the performance is measured by timeduration to perform the operation.

4.2.3 Test script

A test script is developed to enable an automated test process. The script performsthe test cases detailed above. The script is written in Python [45] script and utilisesSelenium [51] and webdrivers for Chrome [8] and Gecko [12] in a headless setup. Thescript emulates a user clicking on the buttons in the test application to perform thedifferent test cases. Each test case is iterated 100 times per application and browser,and the results for each iteration is written to file.

4.2.4 Test applications

The applications are developed in adherence to these common guidelines:

• Development of applications done using methodologies in accordance with theofficial documentation and best practice

• Application is prepared for deployment according to the official documentation

• No external libraries/modules is used

The reference application is a single page application having only the bare-bonesGUI and functionality needed to be able to perform the DOM test cases (create/up-date/delete elements):

15

• Button that when pressed creates and adds 10 000 elements to the DOM

• Button that when pressed updates all elements

• Button that when pressed updates every other element

• Button that when pressed deletes all elements

Framework versions and tools

All framework applications uses the latest version available at the time of creationand are all initiated using their respective command line interface tool, Angular CLI[4], create-react-app [29] and vue-cli [64].

Framework: Version:Angular 9.1.1React 16.13.1Vue 2.6.11

Table 4.1: Framework versions used

4.2.5 Deployment of test script and test applications

The test applications are prepared for deployment according to the guidelines in therespective frameworks official documentation [6, 47, 65]. All applications are con-tainerized using Docker [28]. NGINX [33] is used as web server for all applications,and all Docker images is based on nginx:1.18.0-alpine [34] Docker image. A t3.micro[26] Elastic Compute instance on Amazon Web Services is used as Docker host. Onthis instance a container for each test application is started and used for the tests.

The test script is deployed on a t3.large [26] Elastic Compute instance on Ama-zon Web Services in the same Virtual Private Cloud [27] as the test web server,allowing for HTTP access between the servers.

16

Chapter 5Results

5.1 DOM manipulation methodology comparisonDOM manipulation refers to actions that change the structure of the document [14].Examples of DOM manipulation actions are for example creating, updating anddeleting elements on the page. DOM manipulation in Vanilla JavaScript is handleddirectly by using appropriate methods in the DOM interface [14]. This is a funda-mental difference compared to the selected frameworks where DOM manipulationis handled indirectly via the framework. Whilst this gives a high degree of flexibil-ity on how DOM manipulation is implemented in a Vanilla JavaScript application,great care needs to be taken as there are several potential pitfalls to avoid not leastfrom a DOM manipulation performance perspective. As a developer using VanillaJavaScript an understanding of how to efficiently use the DOM interface is necessaryto avoid introducing DOM performance issues into the application.

The selected frameworks uses the same DOM interface for executing DOM manipu-lations behind the scenes, but DOM manipulation is handled by the frameworks andnot by the developer directly. All frameworks strives to optimise DOM interaction,and has taken different approaches to realize this.

5.1.1 Virtual DOM

React and Vue.js both have implemented a DOM manipulation methodology referredto as Virtual DOM [31, 63]. The main purpose of the Virtual DOM is to keep thereal DOM updated with as little performance impact as possible.

The Virtual DOM is a virtual representation of the DOM. Whenever the frame-work detects a change a new Virtual DOM is created. The newly created VirtualDOM is then compared to the current Virtual DOM, to identify differences. Thisprocess is commonly referred to as "DOM-diffing", but React calls it "Reconcilia-tion" [30]. In this process a diffing algorithm is executed to calculate the optimal wayto update the real DOM with the identified differences. By using this methodologythe frameworks can update the real DOM with what has changed only, instead ofhaving to update the full DOM at every change.

The benefit of this method is the reduced DOM manipulation cost. However, creatinga new Virtual DOM at every change impacts memory usage [49, 43] and application

17

size as the Virtual DOM is not tree-shakable and has to be fully included in appli-cation builds. [49].

5.1.2 Incremental DOM

With the release of Angular version 9 (released February 6 2020 [10]) a new compilerand runtime named Angular Ivy was introduced as the default option, replacing theprevious View Engine [5]. Angular Ivy has implemented a methodology called In-cremental DOM, which is quite different to Virtual DOM. The main drivers for thisimplementation is to optimize memory usage and application size.

Incremental DOM does not use a virtual representation of the DOM, but instead in-teracts directly with the DOM. When the application detects and processes changesaffecting the DOM, this method updates only the parts of the DOM that are af-fected by the change [60]. Viktor Savik, a previous member of the Angular team,describes the main concept of Incremental DOM as "Every component gets compiledinto a series of instructions. These instructions create DOM trees and update themin-place when the data changes" [49]. The instructions generated at compilationincludes both instructions to be executed when the component is first rendered, andinstructions regarding DOM updates when the component changes [49]. As theseinstructions is generated at compile time, it is possible for Angular to apply tree-shaking [69] to remove dead code related to rendering from the application build[10, 49].

5.2 Performance comparison

5.2.1 Initial page render

Vanilla JS Angular React Vue0

20

40

60

80

Ren

deri

ngT

ime

(ms)

Chrome FireFox

Figure 5.1: Initial Page Render, average (ms)

18

Chrome FirefoxAverage Std. dev Min Max Average Std. dev Min Max

Vanilla JavaScript 3 2.1 2 23 19 5.0 15 60Angular 30 4.4 28 72 85 67.2 72 744React 7 3.0 5 36 42 9.8 36 132Vue 9 2.9 8 37 43 16.2 34 144

Table 5.1: Initial page render (ms)

As shown in figure 5.1 and table 5.1 Vanilla JavaScript has the lowest average ren-dering times in both Google Chrome (3 ms) and Firefox (19 ms). The correspondingresults for React is the lowest among the compared frameworks in both GoogleChrome (7 ms) and Firefox (42 ms), closely followed by Vue in both Google Chrome(9 ms) and Firefox (43 ms). Angular is behind in both Google Chrome (30 ms) andFirefox (85 ms). The results show that all test applications achieve a lower averagepage rendering time in Google Chrome. The standard deviation in the test resultsis consistently higher in Firefox compared to Chrome, especially for Angular.


10

20

30

40

50

60

Ren

deri

ngT

ime

(ms)

Figure 5.2: Initial Page Render, average results using both browsers (ms)

Average Std. devVanilla JavaScript 11 8.7Angular 58 55.0React 24 18.7Vue 26 20.3

Table 5.2: Initial page render, average and std. dev based on results using bothbrowsers (ms)

Figure 5.2 and table 5.2 shows the average render time for each application basedon test results from both browsers. Vanilla JavaScript clearly has the lowest averagerendering time (11 ms), followed by React (24 ms) and Vue (26 ms) and lastly Angular(58 ms). The standard deviation when comparing the results for both browsers show

19

that Vanilla JavaScript has the lowest standard deviation. Amongst the frameworksReact has the lowest standard deviation results, followed by Vue and Angular hasthe highest standard deviation.

Chrome Firefox0

10

20

30

40

50

Ren

deri

ngT

ime

(ms)

Figure 5.3: Average first page render time, all applications (ms)

As shown in table 5.3 Google Chrome achieves a lower average page render time (13ms) than Firefox (47 ms) when comparing the test results for all applications in eachbrowser.

5.2.2 Create 10000 elements


200

400

600

800

1,000

1,200

Tim

e(m

s)

Chrome FireFox

Figure 5.4: Create 10000 elements (ms)

20



Table 5.3: Create 10000 elements (ms)

As shown in figure 5.4 and table 5.3 Vanilla JavaScript has the lowest average timefor creating 10 000 elements in both Google Chrome (221 ms) and Firefox (329 ms).Amongst the compared frameworks the results differ between the browsers. Reactachieve the second lowest result in Google Chrome (570 ms), and third lowest inFirefox (1012 ms). Angular on the other hand achieves the second lowest resultin Firefox (732 ms) and third lowest result in Google Chrome (865 ms). Vue hasthe highest average times for creating 10 000 elements in both Google Chrome (903ms) and Firefox (1164 ms). The standard deviation in the test results for VanillaJavaScript is similar in both browsers, but for the frameworks it is consistently higherin Firefox compared to Chrome.


200

400

600

800

1,000

Tim

e(m

s)

Figure 5.5: Create 10000 elements, average (ms)


Table 5.4: Create 10000 elements, average and std. dev based on results using bothbrowsers (ms)

When comparing the results for each application based on test results from bothbrowsers as shown in figure 5.5 and table 5.4 Vanilla JavaScript has the lowest average

21

time for creating 10 000 elements (275 ms). The results for React (791 ms) are closeto those of Angular (798 ms), and Vue (1033 ms) achieves the highest average time inthe test. The standard deviation when comparing the results for both browsers showthat Vanilla JavaScript has the lowest standard deviation. Amongst the frameworksAngular has the lowest standard deviation results, followed by Vue and React hasthe highest standard deviation.

Chrome Firefox0

200

400

600

800

Tim

e(m

s)

Figure 5.6: Average create 10000 elements, all applications (ms)

As shown in figure 5.6 Google Chrome achieves a lower average time to create 10000 elements (640 ms) than Firefox (809 ms) when comparing the test results for allapplications in each browser.

5.2.3 Update all elements


200

400

600

800

Tim

e(m

s)

Chrome FireFox

Figure 5.7: Update all elements, average (ms)

22



Table 5.5: Update all elements (ms)

Figure 5.7 and table 5.5 shows that Vanilla JavaScript has the lowest average timefor updating all 10 000 elements in both Google Chrome (246 ms) and Firefox (346ms). React achieves the second lowest result in Google Chrome (347 ms) and Firefox(384 ms). Angular has the third lowest results in both Google Chrome (355 ms) andFirefox (431 ms), while Vue has the highest average time in both Google Chrome (596ms) and Firefox (850 ms). The standard deviation in the test results is consistentlylower in Google Chrome for all application, with the exception of Vanilla JavaScriptthat has a slightly lower standard deviation in Firefox.


200

400

600

Tim

e(m

s)

Figure 5.8: Update all elements, average results using both browsers (ms)


Table 5.6: Update all elements, average and std. dev based on results using bothbrowsers (ms)

When comparing the results for each application based on test results from bothbrowsers as shown in figure 5.8 and table 5.6 Vanilla JavaScript has the lowest averagetime for creating 10 000 elements (275 ms), followed by React (366 ms), Angular (393ms) and Vue (723 ms). The standard deviation when comparing the results for both

23

browsers show that React has the lowest standard deviation, followed by Angular,Vanilla JavaScript and Vue.

Chrome Firefox0

100

200

300

400

500T

ime

(ms)

Figure 5.9: Update all elements, all applications (ms)

Figure 5.9 shows that Google Chrome achieves a lower average time to update all 10000 elements (386 ms) than Firefox (503 ms) when comparing the test results for allapplications in each browser.

5.2.4 Update every other element


100

200

300

400

500

600

Tim

e(m

s)

Chrome FireFox

Figure 5.10: Update every other element, average (ms)

24



Table 5.7: Update every other element (ms)

As shown in figure 5.10 and table 5.7 Vanilla JavaScript archives the lowest averagetime to update every other element in Google Chrome (159 ms) and Firefox (278ms). React achieves the second lowest result in Google Chrome (188 ms) and Firefox(315 ms), followed by Angular (219 ms in Google Chrome, 352 ms in Firefox), andVue (325 ms in Google Chrome, 583 ms in Firefox). The standard deviation of theresults shows that all applications achive a similar result in Google Chrome. Thestandard deviation of the results in Firefox is all higher except for Vanilla JavaScript.Amongst the frameworks the results ranges from 42.1 (Angular) to 62.4 (Vue).


100

200

300

400

500

Tim

e(m

s)

Figure 5.11: Update every other element, average results using both browsers (ms)


Table 5.8: Update every other element, average and std. dev based on results usingboth browsers (ms)

Figure 5.11 and table 5.8 shows the results for each application based on the testresults from both browsers. Vanilla JavaScript achieves the lowest average time toupdate every other element (219 ms), followed by React (252 ms), Angular (285 ms)and Vue (455 ms). The standard deviation when comparing the results for both

25

browsers show that Vanilla JavaScript has the lowest standard deviation, followedby Angular, React and Vue.

Chrome Firefox0

100

200

300

400T

ime

(ms)

Figure 5.12: Update every other element, all applications (ms)

Figure 5.12 shows that Google Chrome achieves a lower average time to update everyother element (223 ms) compared to Firefox (382 ms) when comparing the test resultsfor all applications in each browser.

5.2.5 Delete all elements


100

200

300

400

Tim

e(m

s)

Chrome FireFox

Figure 5.13: Delete all elements, average (ms)

26



Table 5.9: Delete all elements (ms)

As shown in figure 5.13 and table 5.9 Vanilla JavaScript has the lowest average timefor deleting the 10 000 elements in both Google Chrome (84 ms) and Firefox (235ms). React achieves the second lowest result in Google Chrome (114 ms) and Firefox(262 ms). The results for Vue in Google Chrome (268 ms) is lower than Angular(426 ms), and in Firefox (Angular 365 ms, Vue 381 ms) their positions are reversed.The standard deviation in the test results is consistently higher in Firefox comparedto Chrome, except for Vanilla JavaScript that achieves a slightly lower standarddeviation in Firefox.


100

200

300

400

Tim

e(m

s)

Figure 5.14: Delete all elements, average results using both browsers (ms)


Table 5.10: Delete all elements, average and std. dev based on results using bothbrowsers (ms)

As shown in figure 5.14 and table 5.10 Vanilla JavaScript archives the lowest averagetime to delete all 10000 elements followed by React (188 ms), Vue (325 ms) andAngular (396 ms). The standard deviation when combining the results in both

27

browsers show that Vanilla JavaScript, React and Vue all achieve similar results,while Angular has a slightly higher standard deviation.

Chrome Firefox0

50

100

150

200

250

300

Tim

e(m

s)

Figure 5.15: Average delete all elements, all applications (ms)

Figure 5.15 shows that Google Chrome achieves a lower average time to delete all10 000 elements (223 ms) compared to Firefox (311 ms) when comparing the testresults for all applications in each browser.

5.3 Application size comparison

Vanilla JS Angular React Vue

0

50

100

150

3.2

157.1

135.3

108.9

Size

(kB

)

Figure 5.16: Application size comparison

As shown in figure 5.16 the Vanilla JavaScript test application has a total size of 3.2kB. The size of the Vue.js application is 108.9 kb, the React application is 135.3 kBand the Angular application is 157.1 kB.

28

JavaScript

61.7

HTML/CSS

38.3

Figure 5.17: Vanilla JavaScript application size distribution in percent

As shown in figure 5.17 the Vanilla JavaScript test application consists of 61,7%JavaScript and 38.3% HTML/CSS.

JavaScript 99.2 0.8 HTML/CSS

Figure 5.18: Angular application size distribution in percent

As shown in figure 5.18 the Angular test application consists of 99,2% JavaScriptand 0.8% HTML/CSS.


Figure 5.19: React application size distribution in percent

As shown in figure 5.19 the React test application consists of 98,1% JavaScript and1.9% HTML/CSS.

29


Figure 5.20: Vue application size distribution in percent

As shown in figure 5.20 the Vue.js test application consists of 98,9% JavaScript and1.1% HTML/CSS.

30

Chapter 6Analysis and Discussion

This chapter contains the analysis of the results and is presented per research ques-tion. The results from the empirical study is compared with results in relevantsimilar studies. The chapter ends with a summary where the answers to the researchquestions are put into a larger context.

6.1 RQ 1: How does the DOM manipulation method-ology of Vanilla JavaScript and the selected frame-works compare?

There is a distinct difference between Vanilla JavaScript on one hand, and the se-lected frameworks on the other. Using Vanilla JavaScript DOM manipulation ishandled directly by using appropriate methods in the DOM interface [14], whereas when using one of the selected frameworks the actual interaction with the DOMinterface is abstracted away from the developer. Vanilla JavaScript offers full flexi-bility to the developer regarding how to structure and perform DOM manipulationin the application. But this flexibility shifts the responsibility for efficient DOMmanipulation to the developer(s), requiring the developer(s) to have a solid knowl-edge of how to utilize the DOM interface to construct efficient DOM manipulationmethods. The DOM interface often offer several alternative methods to perform aspecific DOM operation, and there might be significant differences in performancebetween these methods. For a developer without adequate DOM interface knowledgethere is a risk that inefficient DOM manipulation is introduced into the application.Furthermore it is reasonable to claim that this risk is amplified in larger developerteams. As DOM manipulation in the selected frameworks is abstracted away fromthe developer, the same level of DOM interface knowledge and focus is not neededwhen utilizing a framework. On the other hand, using a framework requires frame-work specific knowledge instead.

All frameworks use the DOM interface for executing DOM manipulations behindthe scenes, and they all strive to optimize DOM interaction. While React and Vue.jsboth has implemented a Virtual DOM [31, 63] to optimize DOM interactions, An-gular (version 9) has implemented Incremental DOM. Most of the relevant materialfound on Virtual DOM points out its high performance DOM manipulation as themain strength. Furthermore the main downsides suggested is its higher memoryusage related to keeping a virtual representation of the DOM in memory, and the

31

addition to application size as the Virtual DOM is not tree-shakable. Material foundon Incremental DOM points out its smaller memory footprint due to not having ain-memory virtual representation of the DOM, and the positive effect its ability toutilize tree-shaking [69] to remove dead code related to rendering has on applicationsize as the main benefits.

The test result from the empiric study and other relevant comparison tests [36, 50]suggests that all the selected frameworks can be considered as feasible alternativesfor creating interactive web applications with high DOM manipulation performance,meaning both Virtual DOM and Incremental DOM can deliver adequate DOM per-formance.

6.2 RQ 2: How does initial page rendering time ofVanilla JavaScript and the frameworks compareusing Google Chrome and Firefox?

The importance of a fast loading web page has been described earlier in this thesis,and Page Render Time is a significant part of the total page load time impactinguser experience, SEO and bounce rates. In test case 1, "First page render", in theempirical study a comparison test for first page render time was performed using thetest application. It is worth noting that the test application is small with light-weightlanding page, and should not be considered equivalent to a real application in termsof size and complexity. As expected, Vanilla JavaScript achieved the best results,both for Google Chrome and Firefox. The author’s opinion is to consider the resultsfor Vanilla JavaScript as a baseline to which the frameworks can be compared to.Amongst the frameworks React and Vue are very close in terms of results in bothbrowsers, and Angular achieves the slowest rendering time. The author can concludethat these results correlate to a large extent with a similar benchmark [36], with theexception of the results for React which in the benchmark performed noticeably worsethan Vue. When comparing the average results for the applications based on thetest runs performed in both browsers Vanilla JavaScript achieves the lowest averagetime, followed by React and Vue archiving similar results with Angular having thehighest average time. When comparing the results in Google Chrome and Firefox,the author can conclude that all applications performed better in Google Chrome,achieving both lower and more consistent results.

6.3 RQ 3: How does Vanilla JavaScript and theframeworks compare when it comes to DOMmanipulation performance using Google Chromeand Firefox?

To answer this research question four test cases where performed in the empiri-cal study. The test cases covers DOM manipulation actions for creating, updating

32

and deleting elements from the web page using Vanilla JavaScript and the selectedframeworks. The author’s opinion is to consider the results for Vanilla JavaScript asa baseline to which the frameworks can be compared to.

Create 10 000 elementsThe results from the test measuring time to create 10 000 elements points out thatVanilla JavaScript is significantly faster compared to the selected frameworks in bothGoogle Chrome and Firefox. Amongst the frameworks the results range from at best570 ms (React using Google Chrome) to at worst 1164 ms (Angular using Firefox).The results for React stand out in this test, as the React test application takes almosttwice the time to create the 10 000 elements in Firefox compared to Google Chrome.Furthermore it is interesting to see the difference in results for React and Vue.jsgiven that both frameworks use a Virtual DOM. When comparing the results with asimilar benchmark creating 10 000 table rows performed using the same technologiesand Google Chrome [36], Vanilla JavaScript archives the fastest times in both testsbut the gap between Vanilla JavaScript and the frameworks is larger in the test inthis study. When comparing the average results for the applications based on thetest runs performed in both browsers Vanilla JavaScript achieves the lowest averagetime, followed by React and Angular archiving similar results, and Vue.js having thehighest average time. Comparing the test results in Google Chrome and Firefox, theauthor can conclude that all applications performed better in Google Chrome in thistest case except for Angular. When comparing the results in Google Chrome andFirefox in table 5.3, the author can conclude that all applications performed betterin Google Chrome, achieving both lower and more consistent results. The standarddeviation for the results in both browsers in table 5.4 shows a higher standard devi-ation for React(230 ms) and Vue(138.4 ms) than the other applications. This is dueto the significant difference in the results these applications achieve in the browsersas shown in table 5.3.

Update all elementsFrom the results of the test to update all 10 000 elements Vanilla JavaScript againachieves the lowest average result in both Google Chrome and Firefox. Both Reactand Angular archives average times that are within 100 ms of Vanilla JavaScript’sresults in both browsers, with React being only slightly faster. Vue.js comes in at thelast position being clearly behind the other frameworks, especially in Firefox whereVue.js takes almost twice as long to perform the test compared to React and Vue.js.It is worth noting the difference in results between React and Vue, given that bothuse Virtual DOM. When comparing these results with a similar benchmark usingthe same technologies and Google Chrome [36] the author can conclude that resultscorrelate in large with the exception for Vue.js that in the compared benchmark per-forms in line with the other frameworks. When comparing the average results for theapplications based on the test runs performed in both browsers Vanilla JavaScriptachieves the lowest average time, but React and Angular is following closely andVue.js is clearly behind in this test. The standard deviation in the test results forVanilla JavaScript is similar in both browsers, but for the frameworks it is consis-tently higher in Firefox compared to Chrome as shown in table 5.5. The table alsoshows that in this test there is only a slight difference in standard deviation be-

33

tween the browsers for all applications except for Vue. As a result table 5.6 showslower standard deviations for all the applications except Vue when combining thetest results from both browsers. This is due to the larger difference in the resultsVue achieve in the browsers as shown in table 5.5. The author concludes that allapplications performs better and with more consistent results in Chrome comparedto Firefox in this test case.

Update every other elementIn the results for the test to update every other element (totaling 5 000 elements)Vanilla JavaScript once again achieves the lowest average time both using GoogleChrome and Firefox. In this test the spread between the test results per browser issmaller, especially in Google Chrome. Amongst the frameworks React has the lowestaverage time in both browsers, followed by Angular and Vue had the highest averagetime. When comparing the results with a similar benchmark using the same tech-nologies and Google Chrome [36], the order of technologies in the test results is thesame in both tests. When comparing the average results for the applications basedon the test runs performed in both browsers Vanilla JavaScript achieves the lowestaverage time (219 ms), followed relatively close by React (252 ms) and Angular (285ms) archiving similar results with Vue (455 ms) having the highest average time.The standard deviation of the results in table 5.7 shows that the standard deviationin Firefox is higher for all applications except for Vanilla JavaScript. The differencein performance between the browsers for Vue leads to a higher standard deviationfor that application when looking at the results in table 5.8. It should be noted thatall applications achieved lower average times in Google Chrome compared to Firefox,and all applications except for Vanilla JavaScript achieved more consistent results inGoogle Chrome.

Delete all elementsThe results from the test measuring average time to delete all 10 000 elements showthat Vanilla JavaScript is significantly faster compared to the selected frameworks inboth Google Chrome and Firefox. Amongst the frameworks the results range fromat best 114 ms (React using Google Chrome) to at worst 426 ms (Angular usingChrome). The results for Angular stand out in this test, as the Angular test applica-tion is the only application to achieve a better result in Firefox compared to GoogleChrome. When comparing the results with a similar benchmark using the same tech-nologies and Google Chrome [36], the test results correlate with exception for Vue.jswhich performs at a similar level as React in the compared test. When comparingthe average results for the applications based on the test runs performed in bothbrowsers Vanilla JavaScript achieves the lowest average time, with React followingclosely behind. Vue.js is a bit React, and Angular has the longest average time inthis test. The standard deviation of the results in table 5.9 shows that the standarddeviation in Firefox is higher for all applications except for Vanilla JavaScript. An-gular stands out in this respect having a significant difference in standard deviationbetween the browsers, and also when compared to the other applications in Firefox.All applications performed better in Google Chrome compared to Firefox in this testcase, except for Angular.

34

DOM performance summaryIt is important to stress that the results observed in the empirical study cannot fullyrepresent the technologies respective DOM performance, the empirical study coverscertain specific test cases carried out on a small test application. With that said, thetests cases performed in this study gives an indication on the respective technologiesDOM performance using both Google Chrome and Firefox.The test result from the empiric study and other relevant comparison tests [36, 50]show that all the selected frameworks are feasible alternatives for creating interactiveweb applications with high DOM manipulation performance.


200

400

600

800

1,000

Tim

e(m

s)

Create 10000 elements Update all elements Update every other element Delete all elements

Figure 6.1: Summary of all DOM test, results from both browsers (ms)

As expected Vanilla JavaScript performs better that the selected frameworks in alltests. Of the compared frameworks React stands out with performing best in all fourtest cases when combining the test results from both browsers. The same resultsshows that Angular performs close to React in all test cases, except in the test todelete all 10000 elements where Angular has the highest average time. Vue.js isslightly slower than the other frameworks in all tests, having the highest averagetimes in three of four test cases.

Chrome Firefox Chrome Firefox Chrome Firefox Chrome FirefoxTest case 2 2 3 3 4 4 5 5Vanilla JavaScript 8.1 8.4 9.2 7.1 6.3 5.9 6.2 3.0Angular 21.4 65.1 14.9 32.2 6.4 42.1 14.4 110.8React 15.3 89.4 14.4 24.6 6.6 51.3 4.8 18.7Vue 25.1 60.0 16.2 84.1 8.6 62.4 7.0 71.1

Table 6.1: Overview of standard deviation per application and test in Google Chromeand Firefox (ms).

35

As shown in table 6.1 Vanilla JavaScript has similar standard deviation results inboth browsers throughout the tests. For the frameworks the standard deviation islower in Google Chrome for all applications and tests compared to Firefox, clearlyindicating that Google Chrome is able to deliver more consistent results than Firefox.

6.4 RQ 4: How does Vanilla JavaScript and theframeworks compare when it comes to applica-tion size?

Vanilla JS Angular React Vue

0

50

100

150

3.2

157.1

135.3

108.9

Size

(kB

)

Figure 6.2: Application size comparison

During the empirical study a comparison of the test applications size was conducted.As expected the Vanilla JavaScript application is significantly smaller than the frame-work test applications. This comparison illustrates that there is a baseline cost as-sociated with using a framework. As the test application is a very small application,the lion’s share of the application size is related to the frameworks themselves. Theframework application’s size consists of JavaScript to a great extent, with HTM-L/CSS only being 0,8%-1,9% of the application size. For a larger application theframework baseline cost has a lesser relative impact on application size, but for asmall application requiring a small footprint this should be taken into considerationwhen evaluation whether to use a framework or Vanilla JavaScript. Amongst theframework application size varies between 108,9 kB - 157,1 kB. The Vue.js applica-tion (108.9 kB) has the smallest footprint, followed by React (135.3 kB) and Angular(157.1 kB). Given the small difference in baseline cost of the compared frameworksin its current versions, this factor is likely to weigh less than many other factors whenselecting a framework. Other factors can for example include framework maturity,features, performance, learnability, developer productivity and community [41, 21].

36

6.5 Results seen from a sustainability and societalperspective

This section describes the authors reflection on the results from a sustainability andsocietal perspective. The pursuit of web site performance is often motivated by busi-ness reasons such as getting a competitive edge over the competition, increase salesand web page visits. But performance related metrics such as Page Load Time, PageRender Time and application size can all be linked to sustainability and the societalperspective. By monitoring and maintaining low results for these and other perfor-mance related metrics a web site operator can lessen the web site’s environmentalimpact. Leaner and more efficient applications requires less data to be transferred tothe user, and also requires less processing power on the client side. While over 50%of the worlds population is online, the difference in Internet connection speeds varygreatly between the richest and poorest countries. From an ethical and societal per-spective lowering these metrics lessens the negative effect poor connection speed canhave on usability and user experience, contributing to bridging the digital divide[40].

37

Chapter 7Validity Threats

There is a lack of recent related academic studies with a scope comparable to thisstudy. The lack of academic studies focusing on this topic might partly be explainedby the fast-paced changes that is constantly taking place in the somewhat imma-ture JavaScript framework area [42], rendering studies obsolete in a relatively shorttime after publication. Another threat is that there are many ways to implementthe functionality of the test application using Vanilla JavaScript and the selectedframeworks. Even though official documentation has been used as guidance, theremight be better performing solutions not clearly detailed there. Furthermore theremight be test cases with a higher degree of relevance then the ones performed in thisstudy.

38

Chapter 8Conclusion

In regards to DOM manipulation methodology, there is a distinct difference betweenVanilla JavaScript and the selected frameworks. In Vanilla JavaScript DOM manip-ulation is handled by direct interaction with the DOM interface, where as when usingthe selected frameworks the actual interaction with the DOM interface is handled bythe framework. While React and Vue.js both has implemented a Virtual DOM tooptimize DOM interactions, Angular (version 9) has implemented Incremental DOM.Test results from the empirical study and relevant comparison tests show that theDOM manipulation methodologies of the selected frameworks all are able to deliverhigh DOM manipulation performance.

In the empirical study a test application was created using Vanilla JavaScript andthe selected frameworks. Through the empirical study the author concluded thatVanilla JavaScript, as expected, had the lowest initial page render time and the bestDOM manipulation performance in all tests. Test results for initial page render timeshows that React and Vue are close in terms of results in both browsers, and Angularachieves the slowest rendering time of the frameworks. Amongst the frameworks Re-acts DOM manipulation performance stands out as React has the best performancein three of the four test cases when combining the test results from both browsers.Angular performs close to React in all DOM manipulation performance test cases,except in the test to delete elements. Vue.js is slightly slower than the other frame-works, having the highest average times in three of four test cases. Test results andrelevant comparison tests shows that all selected frameworks are viable options forapplications demanding high DOM performance. In all but one test case GoogleChrome archived better results than Firefox, and it produced more consistent resultsfor all framework applications in all tests compared to Firefox.

The Vanilla JavaScript application is significantly smaller than the framework testapplications which illustrates the baseline cost associated with using a framework.The author concludes that given the small difference in baseline cost of the comparedframeworks, other factors are likely to weigh heavier when selecting a framework.

39

Chapter 9Future Work

This study only scratches the surface when it comes to DOM performance usingJavaScript-based technologies. There are several interesting activities that could bedone to both broaden and deepen the comparison done in this study. This studyhas focused on Vanilla JavaScript, Angular, React and Vue and could be broad-ened by including additional frameworks to the comparison. Furthermore it wouldbe interesting to include additional web browsers like Safari, Samsung Internet andothers to provide better insights related to mobile user experience. The scope of thetest application could be extended to include a wider range of test cases. If timeconstraints allow, a larger and more feature rich test application can be developed,opening up for further comparisons such as learnability, developer experience, devel-opment speed and performance based tests on more production-like applications. Itwould be interesting to further explore these technologies in the perspective of greenweb and digital divide.

40

References

[1] T. Edler A. Andrae. On Global Electricity Usage of Communication Technology:Trends to 2030, volume 6. Challenges, 2015.

[2] I. Allen. The brutal lifecycle of javascript frame-works. https://stackoverflow.blog/2018/01/11/brutal-lifecycle-javascript-frameworks/, 2018. Accessed 19/02/2020.

[3] D. An. Using site speed in web search ranking. https://www.thinkwithgoogle.com/marketing-resources/data-measurement/mobile-page-speed-new-industry-benchmarks/, 2018. Accessed11/02/2020.

[4] angular.io. Angular cli. https://cli.angular.io/, 2020. Accessed01/05/2020.

[5] angular.io. Angular ivy. https://angular.io/guide/ivy, 2020. Accessed09/05/2020.

[6] angular.io. Angular official docs. https://angular.io/docs, 2020. Accessed21/02/2020.

[7] The World Bank. Individuals using the internet (% of population).https://data.worldbank.org/indicator/IT.NET.USER.ZS, 2018. Accessed11/02/2020.

[8] chromium.org. Chromedriver - webdriver for chrome. https://chromedriver.chromium.org/downloads, 2020. Accessed 21/02/2020.

[9] T. Everts. The average web page is 3mb. how much should we care?https://speedcurve.com/blog/web-performance-page-bloat/, 2017. Ac-cessed 10/05/2020.

[10] S. Fluin. Version 9 of angular now available —project ivy has arrived! https://blog.angular.io/version-9-of-angular-now-available-project-ivy-has-arrived-23c97b63cfa3,2020. Accessed 01/05/2020.

[11] Mozilla Foundation. Internet health report2019. https://internethealthreport.org/2019/more-than-half-of-the-world-is-online-but/, 2019. Accessed09/02/2020.

41

[12] Mozilla Foundation. Mozilla geckodriver. https://github.com/mozilla/geckodriver/releases, 2019. Accessed 21/02/2020.

[13] Mozilla Foundation. Css object model (cssom). https://developer.mozilla.org/en-US/docs/Web/API/CSS_Object_Model, 2020. Accessed 16/03/2020.

[14] Mozilla Foundation. Introduction to the dom. https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction, 2020. Ac-cessed 21/02/2020.

[15] Mozilla Foundation. Mdn web docs. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference, 2020. Accessed 21/02/2020.

[16] Mozilla Foundation. Navigation timing api. https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API, 2020. Accessed11/02/2020.

[17] Mozilla Foundation. Performance. https://developer.mozilla.org/en-US/docs/Web/API/Performance, 2020. Accessed 03/05/2020.

[18] Github.com. The state of the octoverse. https://octoverse.github.com/,2019. Accessed 19/02/2020.

[19] A. Goel. 10 best javascript frameworks to use in 2020. https://hackr.io/blog/best-javascript-frameworks, 2020. Accessed 19/02/2020.

[20] Google. Angular - a web application framework for mobile, desktop, and web.https://opensource.google/projects/angular, 2020. Accessed 25/04/2020.

[21] S. Greif. The 12 things you need to consider when evaluatingany new javascript library. https://www.freecodecamp.org/news/the-12-things-you-need-to-consider-when-evaluating-any-new-javascript-library-32018. Accessed 12/05/2020.

[22] I. Grigorik. Constructing the object model. https://developers.google.com/web/fundamentals/performance/critical-rendering-path/constructing-the-object-model, 2019. Accessed 21/02/2020.

[23] I. Grigorik. Render-tree construction, layout, and paint. https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction, 2019. Accessed 22/04/2020.

[24] IEEE. Ieee standard glossary of software engineering terminology. IEEE Std610.12-1990, pages 1–84, 1990.

[25] Amazon Web Services Inc. Amazon ec2. https://aws.amazon.com/ec2/?nc2=h_ql_prod_fs_ec2, 2020. Accessed 21/02/2020.

[26] Amazon Web Services Inc. Amazon ec2 t3 instances. https://aws.amazon.com/ec2/instance-types/t3/, 2020. Accessed 01/05/2020.

42

[27] Amazon Web Services Inc. Amazon virtual private cloud. https://aws.amazon.com/vpc/, 2020. Accessed 21/02/2020.

[28] Docker Inc. Docker official website. https://www.docker.com/, 2020. Accessed21/02/2020.

[29] Facebook Inc. Create react app. https://github.com/facebook/create-react-app, 2020. Accessed 01/05/2020.

[30] Facebook Inc. Reconciliation. https://reactjs.org/docs/reconciliation.html, 2020. Accessed 09/05/2020.

[31] Facebook Inc. Virtual dom and internals. https://reactjs.org/docs/faq-internals.html#what-is-the-virtual-dom, 2020. Accessed09/05/2020.

[32] GitHub Inc. Github official website. https://www.github.com/, 2020. Accessed21/02/2020.

[33] NGINX Inc. Nginx. https://www.nginx.com/, 2020. Accessed 01/05/2020.

[34] NGINX Inc. nginx:1.18.0-alpine dockerfile. https://github.com/nginxinc/docker-nginx/blob/70e44865208627c5ada57242b46920205603c096/stable/alpine/Dockerfile, 2020. Accessed 01/05/2020.

[35] ECMA International. Standard ecma-262. https://www.ecma-international.org/publications/standards/Ecma-262.htm, 2020.Accessed 29/05/2020.

[36] S. Krause. A comparison of the perfomance of a few popular javascript frame-works. https://github.com/krausest/js-framework-benchmark, 2019. Ac-cessed 10/05/2020.

[37] A. Singhal M. Cutts. Using site speed in web searchranking. https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html/, 2010. Accessed11/02/2020.

[38] NetApplications.com. Browser market share. https://netmarketshare.com/?options=%7B%22filter%22%3A%7B%22%24and%22%3A%5B%7B%22deviceType%22%3A%7B%22%24in%22%3A%5B%22Desktop%2Flaptop%22%5D%7D%7D%5D%7D%2C%22dateLabel%22%3A%22Trend%22%2C%22attributes%22%3A%22share%22%2C%22group%22%3A%22browser%22%2C%22sort%22%3A%7B%22share%22%3A-1%7D%2C%22id%22%3A%22browsersDesktop%22%2C%22dateInterval%22%3A%22Monthly%22%2C%22dateStart%22%3A%222019-06%22%2C%22dateEnd%22%3A%222020-05%22%2C%22segments%22%3A%22-1000%22%7D, 2020. Accessed29/05/2020.

[39] J. Nielsen. Website response times. https://www.nngroup.com/articles/website-response-times/, 2010. Accessed 19/02/2020.

43

[40] OECD. Bridging the digital divide. https://www.oecd.org/site/schoolingfortomorrowknowledgebase/themes/ict/bridgingthedigitaldivide.htm, 2020. Accessed 29/05/2020.

[41] C. Hiller J. White P. Selle, T. Ruffles. Choosing a JavaScript Framework. Bleed-ing Edge Press, 2014.

[42] P. Abrahamsson Pano, D. Graziotin. Factors and actors leading to the adoptionof a JavaScript framework, volume 23st. Empirical Software Engineering, 2018.

[43] S. Peyrott. React virtual dom vs incrementaldom vs ember’s glimmer. https://auth0.com/blog/face-off-virtual-dom-vs-incremental-dom-vs-glimmer/, 2015. Ac-cessed 09/05/2020.

[44] The Shift Project. Lean ict: Towards digital sobriety. https://theshiftproject.org/en/lean-ict-2/, 2019. Accessed 11/02/2020.

[45] python.org. Python official website. https://www.python.org/, 2020. Accessed21/02/2020.

[46] React. React github page. https://github.com/facebook/react, 2020. Ac-cessed 22/04/2020.

[47] reactjs.org. React official docs. https://reactjs.org/docs/getting-started.html, 2020. Accessed 21/02/2020.

[48] J. Robie. What is the document object model? https://www.w3.org/TR/WD-DOM/introduction.html, 1998. Accessed 4/03/2020.

[49] V. Savkin. Understanding angular ivy: Incremen-tal dom and virtual dom. https://blog.nrwl.io/understanding-angular-ivy-incremental-dom-and-virtual-dom-243be844bf36,2019. Accessed 09/05/2020.

[50] J. Schae. A realworld comparison of front-end frameworks withbenchmarks (2019 update). https://www.freecodecamp.org/news/a-realworld-comparison-of-front-end-frameworks-with-benchmarks-2019-update-4be02020. Accessed 10/05/2020.

[51] selenium.dev. Selenium official website. https://www.selenium.dev/, 2020.Accessed 21/02/2020.

[52] M. Rambeau S.Grief. 2019 javascript rising stars. https://risingstars.js.org/2019/en/, 2020. Accessed 22/04/2020.

[53] M. Shirshin. Render-tree construction, layout, and paint. http://frontendbabel.info/articles/webpage-rendering-101/, 2014. Accessed22/04/2020.

[54] L. Simon. Minimizing browser reflow. https://developers.google.com/speed/docs/insights/browser-reflow, 2018. Accessed 22/04/2020.

44

[55] J. Smith. 9 popular javascript frameworks for 2020. https://raygun.com/blog/popular-javascript-frameworks/, 2020. Accessed 19/02/2020.

[56] Stackoverflow.com. Stack overflow developer survey 2019. https://insights.stackoverflow.com/survey/2019/#technology, 2019. Accessed 11/02/2020.

[57] A. Svensson. Todomvc - helping you select an mv* framework. http://todomvc.com/, 2020. Accessed 21/02/2020.

[58] P. Irish T. Garsiel. How browsers work: Behind the scenes of mod-ern web browsers. https://www.html5rocks.com/en/tutorials/internals/howbrowserswork/, 2011. Accessed 21/02/2020.

[59] C. Trueman. Why data centres are the new frontier in the fight againstclimate change. https://www.computerworld.com/article/3431148/why-data-centres-are-the-new-frontier-in-the-fight-against-climate-change.html, 2019. Accessed 19/02/2020.

[60] M. Ubl. Introducing incremental dom. https://medium.com/google-developers/introducing-incremental-dom-e98f79ce2c5f, 2015.Accessed 10/05/2020.

[61] International Telecom Union. Itu statistics. https://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx, 2019. Accessed 19/02/2020.

[62] Vue.js. Vue.js github page. https://github.com/vuejs/vue, 2020. Accessed22/04/2020.

[63] vuejs.org. Render functions & jsx. https://vuejs.org/v2/guide/render-function.html#The-Virtual-DOM, 2020. Accessed 01/05/2020.

[64] vuejs.org. Vue cli. https://cli.vuejs.org/, 2020. Accessed 01/05/2020.

[65] vuejs.org. Vue official docs. https://vuejs.org/v2/guide/, 2020. Accessed21/02/2020.

[66] w3.org. What is the document object model? https://www.w3.org/TR/WD-DOM/, 1998. Accessed 21/02/2020.

[67] w3.org. Navigation timing. https://www.w3.org/TR/navigation-timing/,2012. Accessed 21/02/2020.

[68] w3techs.com. Usage statistics of javascript as client-side program-ming language on websites. https://w3techs.com/technologies/details/cp-javascript/, 2020. Accessed 19/02/2020.

[69] J. Wagner. Minimizing browser reflow. https://developers.google.com/web/fundamentals/performance/optimizing-javascript/tree-shaking,2018. Accessed 10/05/2020.

45

[70] J. Wagner. Why performance matters. https://developers.google.com/web/fundamentals/performance/why-performance-matters, 2020. Accessed10/05/2020.

[71] D. Phan Z. Wang. Using page speed in mobile searchranking. https://webmasters.googleblog.com/2018/01/using-page-speed-in-mobile-search.html/, 2018. Accessed 19/02/2020.

46

Appendix AEmpirical study links

This appendix contains links to the test script and the test application repositoriesand Docker images.

JavaScript test application:https://github.com/mope18/Vanilla-js-dom-perf-test-apphttps://hub.docker.com/repository/docker/morper/bachelor-js

Angular test application:https://github.com/mope18/angular-dom-perf-test-apphttps://hub.docker.com/repository/docker/morper/bachelor-angular

React test application:https://github.com/mope18/react-dom-perf-test-apphttps://hub.docker.com/repository/docker/morper/bachelor-react

Vue.js test application:https://github.com/mope18/vue-dom-perf-test-apphttps://hub.docker.com/repository/docker/morper/bachelor-vue

47

Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden

JavaScript DOM Manipulation Performance - DiVA portal

Documents

Transcript of JavaScript DOM Manipulation Performance - DiVA portal