Hacking on Blink and Chromium

This blog documents my attempts to use and improve the Web platform

Iterating on Mobile Apps at Web Speed

This article covers a collection of techniques that allow me to build HTML5 applications with native capabilities, and iterate on my code using the edit-save-refresh cycle that is characteristic of Web development.

Motivation

I love building Web applications, and I’m addicted to the edit-save-refresh cycle.

At the same time, I miss many features that mobile app developers take for granted, but will take years to get through the Web standards gauntlet, such as WakeLock and Push notifications. Security decisions that make perfect sense in the context of Web pages become really limiting for applications. For example, it is impossible to build an alarm clock, because the play() method on HTML5 media elements (<audio> and <video>) requires user interaction, at least on Android and iOS

HTML5 application wrappers rose to respond to this challenge. A wrapper essentially bundles a browser component (usually known as a WebView), glue code that exposes native capabilities to the JavaScript inside the WebView, and the HTML, CSS and JS code that you supply. The first wrapper that I am aware of is Titanium.

Today’s favorite wrapper, Apache Cordova, has made a lot of progress on the tooling front, but still uses the same bundling technique, which means that I have to build a native application, deploy it on my mobile device, and restart, whenever I want to make a change.

Outline and Code

The following sections will describe the tricks I used to restore the edit-save-refresh cycle inside a Cordova application container.

Some tricks in this article can be repurposed for other wrappers easily, while some tricks are specific to Cordova’s implementation. The latter may still suggest approaches to solving similar problems in different wrappers.

I used the methods described here in the html5-app-container project, which is open-sourced on GitHub. The code samples in this article lack some of the details in the GitHub project, in the interest of brevity and readability. I consider the GitHub project code to be the production-quality version that you should use for your applications.

Chapter 1: The Naive Solution

In order to be able to use edit-save-refresh, I have to get the Cordova container to load my application code from a server. The naive solution would be to simply navigate to the server’s URL from the application’s main page.

www/index.html
<script>
window.location = 'http://webapp.com:3000/&#39;;
</script>

This is essentially the application code shipped with the initial html5-app-container version. The key files are app/index.html and app/js/index.js.

While the application gets the edit-save-refresh cycle, it loses all the native Cordova functionality. This is because the native functionality uses the Cordova bridge, which is a fancy name for platform-specific hacks that let JavaScript code communicate with the native Cordova code. The Cordova bridge is initialized by loading a platform-specific cordova.js file that is placed along the HTML5 application’s files by the Cordova build chain.

At this stage, it is tempting to attempt to load cordova.js from the Web site. This could work on Android, where the HTML5 application is accessible from file:///android_asset/www/index.html, but is doomed to fail on iOS, because the application path has a GUID in it, e.g. file:///Users/pwnall/Library/Developer/CoreSimulator/Devices/CF0D2CC2-3E1B-49FB-BC59-AFE4FD4C65FD/data/Applications/75A1BEC5-6FBF-4E90-90B8-F4B8B22EF5B0/AppContainer.app/www/index.html.

Even if the complex path on iOS can be tamed, Chrome’s remote inspector reveals that accessing a file:///android_asset resource from a http origin causes a security error, as shown below.

Web Application Snippet
<script src="file:///android_asset/www/cordova.js"></script>
<!— produces the following security error
Not allowed to load local resource: file:///android_asset/www/cordova.js
—>

Before moving on to the next attempt, it’s worth noting that even the naive solution provides advantages over running the application from a Web browser. Cordova’s WebView uses a security model that is better suited for applications, e.g. the play() method of HTML5 media elements can be called without user interaction.

Furtermore, the Crosswalk project offers a Cordova fork that uses a WebView built on Chromium, instead of the platform’s native WebView. This approach removes a lot of variability coming from the wide range of WebView versions shipped by device firmware. I can’t recommend it strongly enough! In fact, I wasted a few months trying to build pretty much the same thing, before this existed.

Last, both Cordova and Crosswalk support debug and release builds. The debug builds set up the WebViews so that they can be debugged remotely. On iOS, the Cordova WebView can be debugged by Safari’s Remote Inspector. On Android, the Cordova WebView and the Crosswalk WebView can be debugged using Chrome’s Remote Debugger.

In both remote debuggers, hitting the Refresh keyboard shortcut (Command+R on Mac OS X, Control+R on all other systems) reloads the WebView. This is an important piece of the edit-save-refresh cycle, as having to reach out for the phone and close/reopen the application would be much slower than hitting Refresh on my development machine.

Chapter 2: The Iframe

The <iframe> element has introduced many security issues in the Web platform, so it seems fitting that we’d try to use it to work around Cordova’s restrictions.

A reasonable design would be to have the actual application running inside a host page’s <iframe>. The <iframe> takes up the host page’s entire display space. The host page is the default Cordova starting page, www/index.html, so it can load cordova.js and talk to the Cordova bridge.

www/index.html
<script src="cordova.js"></script>
<script src="js/index.js"></script>
<iframe id="iframe" src="blank.html" scrolling="no"></iframe>
<style>
html, body, iframe {
display: block;
width: 100%; height: 100%;
margin: 0; border: none; padding: 0;
}
</style>

The host page uses postMessage to communicate with the application running inside the <iframe>.

www/www/index.js
(function() {
var appUrl = origin + '/';
document.addEventListener('deviceready', function() {
var iframe = document.getElementById('iframe');
window.iframe = iframe;
iframe.src = appUrl;
}, false);
window.addEventListener('message', function(event) {
var iframe = document.getElementById('iframe');
if (event.origin !== origin)
return;
if (event.data.substring(0, 25) === 'html5-app-container-eval|') {
var javaScript = event.data.substring(25);
var result = null;
try {
result = window.eval(javaScript);
} catch(e) {
result = e;
}
iframe.contentWindow.postMessage(
'html5-app-container-evaled|' + result, '*');
}
}, false);
})();

To keep the initialization process simple, the host page first waits until the Cordova bridge is initialized, and then loads the Web application in the <iframe>.

The postMessage receiver only accepts messages whose origin matches the Web application. This prevents hostile content, such as embedded ads, from obtaining native capabilities.

The receiver shown above is simple by design. It essentially eval()s the received message and responds with a message that contains the evaluation result. Using eval means that I can iterate on all the interesting code by changing the Web application code.

The main drawback of this approach is that sending JavaScript strings to the host page is an unnatural technique for developing Web applications, and can get tedious as the code that interacts with native features becomes more complex. Rails (my Web framework of choice) does not have good support for putting together JavaScript strings, and this was a deal-breaker for me.

The second drawback is that debugging my application with Chrome or Safari’s remote inspector is a bit more tedious. In order to see my page’s elements, I have to peek into the host page’s <iframe>. To play with the JavaScript objects, I have to access the <iframe>’s contentWindow. The deviceready handler code posted above assigins the <iframe> element to the iframe property on the host page’s window, to make debugging slightly easier.

Last, I was worried that as I use data-intensive features, such as Cordova’s File API, shuttling large strings around might become a performance bottleneck.

Fortunately, we can do better!

Chapter 3: The Iframe Grows Stronger

This section describes a trick that removes the need to postMessage strings to the host page in most cases, by giving the Web application inside the <iframe> direct access to Cordova’s native objects. This makes the <iframe> approach more bearable.

In a nutshell, after the Web application loads inside the <iframe>, it posts a message to the host page, asking it to copy the references to Cordova’s objects to the <iframe>’s window object. For generality, the host page creates an empty object, copies all the enumerable properties of its window into the newly created object, then assigs the object to a property on the <iframe>’s window.

www/js/index.js
(function() {
var appUrl = origin + '/';
document.addEventListener('deviceready', function() {
var iframe = document.getElementById('iframe');
window.iframe = iframe;
iframe.src = appUrl;
}, false);
window.addEventListener('message', function(event) {
var iframe = document.getElementById('iframe');
if (event.data === 'html5-app-container-wire-me') {
var platform = {};
iframe.contentWindow.cordovaPlatform = platform;
var properties = Object.getOwnPropertyNames(window);
var length = properties.length;
for(var index = 0; index < length; ++index) {
var name = properties[index];
platform[name] = window[name];
}
iframe.contentWindow.postMessage('html5-app-container-wired-you', '*');
}
}, false);
})();

Note that the host page still waits for the Cordova bridge to be fully initialized before navigating the <iframe> to the Web application. This guarantees that the host page’s window contains Cordova’s native objects by the time the postMessage receiver is invoked.

The second major html5-app-container version implements the trick above, combined with the eval() receiver described in the previous section. The key files are app/index.html and app/js/index.js.

Unfortunately, this trick does not work in Crosswalk 10+, so I had to look for a better solution. However, it is worth knowing, as it still works in Cordova, and might come in handy when dealing with other HTML5 application wrappers.

Chapter 4: The Demise of the Iframe

A different approach to obtaining native capabilities is to attempt to initialize the Cordova bridge in the Web application.

In this case, we don’t embed any code in the Cordova application. Instead, we add a config.xml whose <access> points to our Web application.

assets/config.xml
<?xml version='1.0' encoding='utf-8'?>
<widget id="us.costan.app_container" version="1.0.0"
<name>AppContainer</name>
<description>
Web Application Container
</description>
<author email="victor@costan.us" href="www.costan.us">
Victor Costan
</author>
<access origin="*" />
</widget>

According to Cordova Bug 5988, the Cordova bridge can be used from a file:// origin, or from the top level page’s origin. So, if we can obtain the platform-specific JavaScript in the Web application, we can execute it and gain access to Cordova’s native features.

Fortunately, Cordova’s JavaScript is well-structured, and collected in the platform-specific assets directory for each platform. For example, the Android JavaScript is all in platforms/android/assets/www and the iOS JavaScript is in platforms/ios/www.

The first file we need is cordova.js, the build product of the cordova-js project. As far as we’re concerned, it implements Cordova’s module system (cordova.define and cordova.require), and contains the platform-specific bootstrap code.

The second file we must extract is cordova_plugins.js, in the same directory. It defines the cordova/plugin_list module, which exports a list of all the plugins installed in the application.

The other files needed to set up the bridge are spread throughout the plugins/ directory. A reasonable method for obtaining them is recusively looking for .js files inside the plugins/ directory. In order to avoid including any test code, we can filter out the files that don’t start with cordova.define.

We can collect and concatenate all the files for each platform into one big bundle file, e.g. android.js for Android, and ios.js for iOS. In the Web application, we need to detect whether we’re running under Cordova and, if so, load the appropriate bundle.

I use the Rails asset pipeline to serve the JavaScript files, as it does all the tricks needed to serve them with long-lived cache expiration dates. In order for this to work, I need to use asset pipeline-generated URLs to load the scripts.

app/assets/javascripts/platform_scripts.js.erb
window.PlatformScripts = {
android: <%= javascript_path('cordova/android.js').to_json %>,
ios: <%= javascript_path('cordova/ios.js').to_json %>
};

The JavaScript bundles must not be executed when the Web application does not run under Cordova. For example, the Android bundle currently uses window.prompt() to talk to the native Cordova code. If this code runs outside Cordova, the user will be presented with cryptic dialog boxes. In order to avoid that, we must make sure that we run under Cordova before executing any Cordova JavaScript.

app/assets/javascripts/platform.js
window.Platform = {};
// Returns "android", "ios", or null if not running under Cordova.
Platform.cordovaPlatform = function() {
var ua = navigator.userAgent;
if (/android/i.test(ua)) {
if(window._cordovaNative)
return 'android';
return null;
}
if (/iphone|ipad|ipod/i.test(ua)) {
if (</span>(</span>d+</span>)$/.test(ua))
return 'ios';
return null;
}
return null;
};

The code above takes advantage of Cordova implementation details, and was derived from Cordova 3.6. On Android, Cordova defines a _cordovaNative object. On iOS, Cordova appends a numeric nonce wrapped in parantheses to the WebView’s User-Agent property.

If the Web application detects that it runs under Cordova, a <script> tag pointing to the appropriate script is dynamically generated and added to the page.

app/assets/javascripts/platform.js
// Calls the callback with true (success) or false (failure).
Platform.bootCordova = function(callback) {
var platform = Platform.cordovaPlatform();
if (platform === null) {
callback(false);
return;
}
var element = document.createElement('script');
element.src = PlatformScripts[platform];
document.getElementsByTagName('head')[0].appendChild(element);
document.addEventListener('deviceready', function() {
callback(true);
});
};

While implementing this approach, I ran over a Crosswalk crashing bug that brings down the entire Android application when a non-file:// page attempts to initialize the Cordova bridge. Fortunately, I was able to find and submit a fix.

The third major html5-app-container version uses the approach outlined in this section. The only application file is app/config.xml. The Cordova JavaScript specific to Android and iOS is collected by two bash scripts, script/bundle-android.sh and script/bundle-ios.sh.

This method addresses the main issues raised by the <iframe> approach. The Web application does not need to postMessage JavaScript strings to a host page, and the Chrome and Safari remote inspectors display the Web application’s elements and interact directly with the application’s JavaScript world.

Unfortunately, this method’s requirement of hosting Cordova’s platform-specific JavaScript bndles on the server introduces significant complexity in long-lived production applications. For example, if a security issue is reported in the Cordova version used by the application wrappper, a new version of the wrapper must be released. New bundles must be produced and deployed on the production server before the new wrapper can be released. Furthermore, the Web application must retain the bundles for all the wrapper versions that were ever released.

From a performance standpoint, this approach has the downside of having to load the platform-specific JavaScript over the network. The Android-specific JavaScript file has almost 300kb, and the iOS-specific JavaScript is slightly biggger. Furthermore, the file is loaded via a dynamically-added <script> tag, which eludes the pre-loader.

The Rails asset pipeline sweetens the performance blow quite a bit, as the file is 26kb after minifying and gzipping, and is served with a long-lived cache expiration date. Still, it’s nice to know that the following refinement removes the performance issue completely, while being easier to maintain at the same time.

Chapter 5: The Return of the Iframe

In this section, we modify the approach presented above to remove the need for server-hosted JavaScript bundles. The high-level idea is rather straightforward, namely we must restructure our code so that the Web application receives the JavaScript bundle from the Cordova wrapper. However, we need a couple of tricks to bypass the security measures in Cordova’s WebView.

First, the build scripts for the Cordova-based wrapper must be augmented to extract the JavaScript bundle, as described in the previous section.

This is mildly tricky, because the Cordova build scripts wipe the www directory inside the platform-specific platforms/ subdirectory at the beginning of a build. Fortunately, the script copies the platform-specific cordova.js from the platform_www directory inside the platform’s platforms/ subdirectory, so we can stash our JavaScript bundle there.

Also, we minify the JavaScript bundle using UglifyJS 2. On the Andoid and iOS bundles, minification cut down about 300k of JavaScript to less than 100k. This matters because we’ll store the bundle in sessionStorage, which is read synchronously from JavaScript.

Second, we must send the JavaScript bundle to the Web application, despite the fact that it cannot read file:// resources. We bypass this limitation by resurrecting the <iframe> technique discussed earlier.

The application wrapper includes a host page that loads a bootstrap page from our Web application in a hidden <iframe>.

www/index.html
<iframe id="iframe" src="blank.html" style="display: none;"></iframe>
<script src="js/index.js"></script>

The bootstrap page receives the JS bundle from the host page via postMessage and stores it in its sessionStorage. Note that the bootstrap page must be hosted by the Web application, because the sessionStorage store is per-origin. After the bundle is stored, the bootstrap page sends a message to the host message telling it to navigate to the Web application’s main page. The main page URL is contained in the message so that the application wrapper only needs to hard-code one URL, namely the bootstrap page’s URL.

Sample Bootstrap Page
<script>
window.addEventListener('message', function(event) {
if (event.origin.substring(0, 7) !== 'file://')
return;
var data = event.data;
if (data.substring(0, 10) !== 'cordovaJs|')
return;
sessionStorage.setItem('cordova-js', data.substring(10));
window.parent.postMessage('jump|http://webapp.com:3000/&#39;, '');
});
if (window.parent !== window) {
window.parent.postMessage('getjs', '');
};
</script>

The host page JavaScript below looks large, but it can be de-constructed into small straightforward parts. The getBundle function uses XMLHttpRequest to read the JavaScript bundle. The onMessage function implements the postMessage-based protocol used to communicate with the Web application’s bootstrap page, and is similar to the code for the <iframe>-based approach presented above.

www/js/index.js
(function() {
var bootstrapOrigin = 'http://webapp.com:3000&#39;;
// Issues an XHR get and passes the response String to the callback function.
var getUrl = function(url, callback) {
var xhr = new XMLHttpRequest();
xhr.open('GET', absoluteUrl, true);
xhr.onreadystatechange = function() {
if (xhr.readyState !== 4)
return;
var data = xhr.responseText || xhr.response;
callback(data);
};
xhr.send();
};
// Loads the JS bundle via XHR and passes it to the callback function.
var getBundle = function(callback) {
var pageUrl = window.location.href;
var slash = pageUrl.lastIndexOf('/');
var absoluteUrl = pageUrl.substring(0, slash + 1) + 'cordova_all.min.js';
getUrl(absoluteUrl, callback);
};
// Load the bootstrap URL in the iframe.
var iframe = document.getElementById('iframe');
iframe.src = bootstrapUrl;
// The postMessage protocol used to talk to the bootstrap page.
var onMessage = function(event) {
if (event.origin !== bootstrapOrigin)
return;
var data = event.data;
if (data === 'getjs') {
getBundle(function(cordovaJs) {
iframe.contentWindow.postMessage(
'cordovaJs|' + cordovaJs, bootstrapOrigin);
});
}
if (data.substring(0, 5) === 'jump|') {
window.removeEventListener('message', onMessage);
window.location = data.substring(5);
}
};
window.addEventListener('message', onMessage, false);
})();

Note that after the JavaScript bundle transfer is complete, the host page’s code navigates the top-level window to the Web application’s main URL. So, after the bundle transfer, the Web application takes over the entire Cordova WebView, as opposed to being contained inside an <iframe>, so the drawbacks of the <iframe>-based solution do not apply.

This method also does not require the Web application to rely on Cordova implementation details to detect whether it runs inside a Cordova container or not. Instead, application code can simply probe sessionStorage for the Cordova JavaScript bundle.

app/assets/javascripts/platform.js
// Calls the callback with true (success) or false (failure).
Platform.bootCordova = function(callback) {
var cordovaJs = sessionStorage.getItem('cordova-js');
if (!cordovaJs) {
callback(false);
return;
}
document.addEventListener('deviceready', function() {
callback(true);
});
window.eval(cordovaJs);
};

This was implemented in the 4th major html5-app-container version uses the approach outlined in this section. The host page HTML is app/index.html and the loader code is in app/js/loader.js. The Cordova JavaScript bundles are collected by script/bundle-ios.sh, which (confusingly) builds the Cordova-based wrapper for both iOS and Android. script/bundle-android.sh builds an Android wrapper using the Crosswalk fork of Cordova.

Unfortunately, the code here has an XSS (cross-site scripting) vulnerability. If an attacker convinces a user to download a malicious HTML page and open the downloaded version, such that it gets the file:// origin, the malicious page can open our Web application’s bootstrap page in an <iframe> and use the postMessage protocol to convince the bootstrap page to store some evil JavaScript in sessionStorage. If the user then visits our Web application from the same computer, the application will run the attacker’s evil JavaScript.

Chapter 6: The Revenge of the Iframe

In order to fix the XSS vulnerability, we apply a well-known method for preventing against CSRF (cross-site request forgery) vulnerabilities.

The Web application’s bootstrap page generates a random token and places it inside a <div> attribute in its DOM tree. The application wrapper uses its privileges to extract the token from the <div> element, and sends the token together with the Cordova JavaScript. The bootstrap does not write to sessionStore if the token in the message doesn’t match the value that it generated.

The main source of complexity in the updated bootstrap page code (shown below) is the token generation code. We first attempt to generate a cryptographically secure token, using the WebCrypto API. If that is not supported, we fall back to using Math.random().

Sample Bootstrap Page
<div id="cordova-js-token" datatoken=""></div>
<script>
(function() {
var generateToken = function() {
var numbers = null;
// If possible, get cryptographically secure random values.
try {
if (window.crypto && crypto.getRandomValues && window.Uint32Array) {
numbers = new Uint32Array(4);
crypto.getRandomValues(numbers);
}
} catch(cryptoError) {
numbers = null;
}
// Fallback to weak random values.
if (numbers === null) {
numbers = new Array(4);
for (var i = 0; i < 4; ++i)
numbers[i] = Math.random();
}
var strings = new Array(4);
for (var i = 0; i < 4; ++i)
strings[i] = numbers[i].toString(36);
return strings.join('_');
};
var jsToken = generateToken();
var checkToken = function(submitted, baseline) {
return submitted === baseline;
};
var onMessage = function(event) {
if (event.origin.substring(0, 7) !== 'file://')
return;
var data = event.data;
if (data.substring(0, 10) !== 'cordovaJs|')
return;
var tokenEnd = data.indexOf('|', 10);
if (tokenEnd === 1)
return;
if (checkToken(data.substring(10, tokenEnd), jsToken))
sessionStorage.setItem('cordova-js', data.substring(tokenEnd + 1));
window.parent.postMessage('jump|http://webapp.com:3000/&#39;, '');
};
window.addEventListener('message', onMessage);
var tokenDiv = document.getElementById('cordova-js-token');
tokenDiv.setAttribute('data-token', jsToken);
window.parent.postMessage('getjs', '');
})();
</script>

Note that when the code above receives an incorrect token, it follows through the postMessage protocol, but doesn’t update sessionStorage. This reduces the amount of information that a potential attacker might obtain, at the cost of making it harder to debug the JavaScript bundle transmission process.

The host page’s code, embedded in the application wrapper, is identical to the previous version, up to the getIframeToken function. The main source of complexity here is working around browser support for the contentDocument property. The onMessage function was modified to call out to getIframeToken and embed the return value in the message that it posts to the bootstrap page.

www/js/index.js
(function() {
var bootstrapOrigin = 'http://webapp.com:3000&#39;;
// Issues an XHR get and passes the response String to the callback function.
var getUrl = function(url, callback) {
var xhr = new XMLHttpRequest();
xhr.open('GET', absoluteUrl, true);
xhr.onreadystatechange = function() {
if (xhr.readyState !== 4)
return;
var data = xhr.responseText || xhr.response;
callback(data);
};
xhr.send();
};
// Loads the JS bundle via XHR and passes it to the callback function.
var getBundle = function(callback) {
var pageUrl = window.location.href;
var slash = pageUrl.lastIndexOf('/');
var absoluteUrl = pageUrl.substring(0, slash + 1) + 'cordova_all.min.js';
getUrl(absoluteUrl, callback);
};
// Load the bootstrap URL in the iframe.
var iframe = document.getElementById('iframe');
iframe.src = bootstrapUrl;
// Returns the HTMLDocument inside an <iframe>, or null.
var getIframeDocument = function(iframe) {
return iframe.contentDocument || iframe.contentWindow.document;
};
// Gets the XSS protection token in the <iframe>.
var getIframeToken = function(iframe) {
var iframeDoc = getIframeDocument(iframe);
var tokenDiv = iframeDoc.getElementById('cordova-js-token');
return tokenDiv.getAttribute('data-token');
};
// The postMessage protocol used to talk to the bootstrap page.
var onMessage = function(event) {
if (event.origin !== bootstrapOrigin)
return;
var data = event.data;
if (data === 'getjs') {
getBundle(function(cordovaJs) {
iframe.contentWindow.postMessage(
'cordovaJs|' + cordovaJs, bootstrapOrigin);
});
}
if (data.substring(0, 5) === 'jump|') {
window.removeEventListener('message', onMessage);
window.location = data.substring(5);
}
};
window.addEventListener('message', onMessage, false);
})();

The security in this method can be proven by reasoning that an attacker who can extract the token from the bootstrap page can also extract a CSRF token embedded in a <form>, <input> or <meta> tag. Such an attacker can already execute requests on behalf of its victim.

Unfortunately, just like the contentWindow trick described in a previous section, the code above does not work in Crosswalk 10, because of this bug.

If you want to use Crosswalk versions impacted by the bug, the host page JavaScript must be modified to catch security exceptions and return a special CSRF token, such as 0000. The bootstrap page can accept the special token if it can verify that the JavaScript execution environment belongs to an impacted Crosswalk version.

Host Page Changes
var getIframeDocument = function(iframe) {
try {
if (iframe.contentDocument)
return iframe.contentDocument;
} catch(securityError) {
}
try {
if (iframe.contentWindow && iframe.contentWindow.document)
return iframe.contentWindow.document;
} catch(securityError) {
}
return null;
};
var getIframeToken = function(iframe) {
var iframeDoc = getIframeDocument(iframe);
if (iframeDoc === null)
return '0000';
var tokenDiv = iframeDoc.getElementById('cordova-js-token');
return tokenDiv.getAttribute('data-token');
};

The host page changes above make the token extraction code more robust, and have no negative impact, from a security perspective. Therefore, they are included in the html5-app-container code.

The Web application page gets to decide whether it accepts the special CSRF token. Therefore the modification below can simply be skipped by an application that never releases a wrapper based on a Crossswalk version impacted by the bug above. Although the changes

Crosswalk Bootstrap Page Changes
var checkToken = function(baseline, submitted) {
if (submitted === baseline)
return true;
// Work around XWALK-2905.
if (submitted !== '0000')
return false;
if (!window.cordovaNative)
return false;
if (cordovaNative.exec.toString() !== 'function () { [native code] }')
return false
if (navigator.userAgent.indexOf('Crosswalk/10.') === 1)
return false;
return true;
};

This was implemented in the 5th major html5-app-container version uses the approach outlined in this section. The new loader code is in app/js/loader.js.

The code presented above has a couple of performance issues, compared to a wrapper that loads the Web application directly into a WebView. Time-permitting, they may be the subject of future work.

First, loading the bootstrap page adds an HTTP round-trip to the application’s startup time. This can be significant for applications that are meant to be used on the go. The round-trip can be eliminated from most starts by serving the bootstrap page with a far-in-the-future cache expiration time.

A high-performance solution would remove the bootstrap page completely. Instead, a Cordova plugin would serve the JavaScript bundle from a custom URL scheme. The Web application could attempt to load the bundle via an XMLHttpRequest, and would report a Cordova boot failure if the request fails. This approach would be expensive in terms of engineering time, as a Cordova plugin requires different native code for each supported platform, and must be maintained to track updates to the Cordova API.

Second, storing the (rather large) JavaScript bundle in sessionStorage is a source of performance issues. The sessionStorage API consists of blocking calls, so the WebView’s JS thread is unresponsive while the mobile device reads or writes the JavaScript bundle into the sessionStorage backing store. Also, some WebView implementations store the sessionStorage contents in RAM, so the JavaScript bundle might use 200kb of RAM (the bundle is 100kb after minification, but is likely stored as a UTF-16 or UCS-2 string. So, we’re possibly taking up 200kb of RAM with a string that’s only used once during page load.

The storageSession is used in these samples because of its simplicity and wide support. A performance-conscious Web application could use IndexedDB and fall back to the deprecated Filesystem API and WebSQL database API. Unfortunately, all fallbacks are necessary to achieve reasonable platform coverage. On a brighter note, the choice of bundle backing store is completely decoupled from the code inside the Cordova wrapper, so a Web application can deploy this performance improvement without having to change the code in html5-app-container.

Of course, the problem of storing the JavaScript bundle goes away if the bootstrap page is eliminated by implementing a Cordova plugin.

JavaScript Bundle Signatures?

A noteworthy alternative to the CSRF token extraction method described here is signing the JavaScript bundles. The signatures would be produced during the application wrapper build process, and verified by the code in the bootstrap page.

Unfortunately, this leaves secret key signing methods, like HMAC, out of the question, because the signature would be embedded on the bootstrap page. Verifying asymmetric (e.g., RSA) signatures is slow when done in JavaScript. Furthermore, signing would burden the build process with the extra complexity of handling a secret key.

Last but not least, relying on signatures would mean that the bootstrap page would accept any previously signed JavaScript bundle. A troll could use this to load a Cordova bundle into the site when used from a normal browser, which would cause JavaScript prompts to pop up when the user browses to our Web application.

A more serious attacker could load an old (but still signed) version of a bundle with known security vulnerabilities, and exploit the vulnerabilities against the site.

General Security Considerations

Unfortunately, all the designs above have the drawback that any XSS (cross-site scripting) vulnerability can be used to obtain access to native features.

At the same time, it is worth mentioning that native applications also suffer from security vulnerabilities, such as push notifications spam and hijacking and Android intent hijacking. Furthermore, a security patch in a Web application requires a server push to be deployed, which can be done in minutes or at most hours. A mobile application patch requires an app store / market submission, which often includes a human review, and can take between hours and days.

Conclusion

This article described the tricks I used in the html5-app-container project, which allows me to build Web applications with the native capabilities afforded by Cordova, and iterate on my code using the edit-save-reload cycle that is typical of Web applications.

Blink's Public Interface

Blink is cleanly separated from Chromium’s content layer. This article gives a quick overview of Blink’s interface. Most code references in this article are relative to Blink’s root directory, third_party/WebKit.

This article is likely to get updated and grow as my understanding of the Blink codebase improves.

The Web and Platform APIs

The interface is defined by the files in the following directories:

  • public/web/ has headers for the classes implemented by Blink and used by the content layer; these classes are implemented in Source/web
  • public/platform/ has classes implemented by the content layer and used by Blink; these classes are implemented in Chrome’s content layer, in the content/ directory in Chrome’s source tree

Blink’s public interface classes are all defined in the blink namespace, and have the Web prefix in their name. In fact, when reading Blink code, it is safe to assume that any class whose name starts with Web belongs to Blink’s public interface.

Wrappers and Pointers

Headers in public/ are not allowed to include files outside of the public/ directory. This implies that all the blink::Web classes and structures can only use internal classes via pointers or references.

Many classes declared in public/web wrap Blink’s internal classes. For example, public/web/WebNode.h defines WebNode, which wraps WebCore::Node, defined in Sorce/core/dom/Node.h.

Some classes hold raw pointers to the internal classes that they wrap. For example, WebDatabase (defined in public/web/WebDatabase.h) holds a raw pointer to the WebCore::DatabaseBackendBase instance that it wraps. However, most classes use a smart pointer. For example, WebNode uses the WebPrivatePtr smart pointer class to reference the WebCore::Node instance that it wraps.

WebPrivatePtr, declared and completely defined in public/platform/WebPrivatePtr.h, is the most popular smart pointer class. It is used for wrapping reference-counted classes, and automatically calls ref(), deref(), etc. on the wrapped instance. The header has a very useful block comment right above the WebPrivatePtr class declaration. On a first read, I’d skip the boilerplate at the beginning of the file and go straight for the block comment.

WebPrivateOwnPtr, declared and completely defined in public/platform/WebPrivateOwnPtr.h, is used for wrapping classes that are allocated on the heap, but do not use reference-counting. The smart pointer automatically destructs the wrapped instance. This class' header also has a very useful block comment.

The Platform Object

The entry point to the implementations of the classes and functions defined in the public/platform headers is the blink::Platform class, defined in public/platform/Platform.h and implemented in content/child/blink_platform_impl.h in the Chromium souce tree.

Blink code obtains a reference to the current platform by calling the blink::Platform::current() static function, and then calls a Platform virtual method. Many methods are used to obtain instances of platform classes.

References

Comparing Against the Current Time in JavaScript

I recently had to write a Blink layout test ensuring that a piece of code returns the current time. I am documenting the stages I went through, hoping to provide both entertainment and an appreciation for the importance of using code that has been thoroughly tested instead of rolling your own.

Let’s start by formalizing the task at hand: given a function code, we want to write a test function test that returns true if code’s return value is the current time, and false otherwise.

Revision 1: Direct Comparison

The straight-forward comparison method is relatively straight-forward.

var test = function (code) {
var result = code();
if (result instanceof Date || typeof(result) == 'number') {
return false;
}
return result.valueOf() === Date.now().valueOf();
}

I’m using ===, like 99.99999% of JavaScript code should. I used valueOf() to convert Date instances into numbers, because different Date instaces are never equal. Conveniently, valueOf() works the same way for both numbers and dates, so I don’t have to worry about whether Date.now() returns a Date instance (which is true in some browsers) or a number, as the specification says.

new Date(100) === new Date(100) // Always false, can't use equality on dates.

This test will probably work on your machine, and will most likely pass a continuous integration suite. However, it is a flaky test, meaning that at some point it will fail. Let’s write some code to prove that.

var proveTestIsBroken = function () {
var good = function() { return new Date(); }
var i = 0;
while (true) {
if (!test(good)) break;
i += 1;
}
console.log(i);
}

On my machine, running this in the Chromium dev tools comes up with a number in less than a second.

Revision 2: Stubbing

In many cases, a great way to solve this problem is to stub the Date API used by the code being tested.

var test = function (code) {
var stubbedTime = 1384868334920;
var realDateNow = Date.now;
var realDateConstructor = Date;
Date = function () {
return new realDateConstructor(stubbedTime);
};
Date.now = function() { return stubbedTime; }
realDateConstructor.now = Date.now;
try {
var result = code();
} finally {
window.Date = realDateConstructor;
Date.now = realDateNow;
}
if (!(result instanceof Date) && typeof(result) !== 'number') {
return false;
}
return result.valueOf() === stubbedTime;
}

This code is a bit paranoid, and is probably overkill for specific cases. It stubs now() on both the stubbed Date constructor and on the real constructor, so all the code() variants below would be recognized as correct.

  • return new Date();
  • return Date.now();
  • return (new Date()).constructor.now()

The biggest advantage of this approach is that the stubbed current time is consistent across tests, which makes for very robust tests. In return, we pay the usual price of stubbing and mocking, namely our test doesn’t prove that code() returns the current time, it merely proves that it calls some API and passes down its return value. Assuming that the underlying APIs are solid and will not change, the trade-off is usually worth it!

At the same time, this approach does not work if code() uses an API that we can’t stub, or if we really want to assert that it returns the current time.

Revision 3: Time is Monotonic

The clever code below takes advantage of the fact that time is monotonic.

var test = function (code) {
var startTime = Date.now().valueOf();
var result = code();
var endTime = Date.now().valueOf();
if (!(result instanceof Date) && typeof(result) !== 'number') {
return false;
}
var time = result.valueOf();
return (startTime <= time) && (time <= endTime);
}

Sadly, this test is still not bulletproof. proveTestIsBroken() will terminate if our assumption of monotonic time breaks down. Most computers use NTP to keep their clock synchronized, so the clock might still be adjusted backwards. test is still flaky.

Revision 4: Sometimes, Time is Monotonic

A straight-forward fix for the NTP issue is below.

var test = function (code) {
while (true) {
var startTime = Date.now().valueOf();
var result = code();
var endTime = Date.now().valueOf();
if (startTime > endTime) {
continue; // Time went backwards. (NTP adjustment?)
}
if (!(result instanceof Date) && typeof(result) !== 'number') {
return false;
}
var time = result.valueOf();
return (startTime <= time) && (time <= endTime);
}
}

The sight of while (true) makes experienced programmers wary, as it can turn into an infinite loop under the right cirumstances. In some cases this will be catastrophic. Most test frameworks implement a timeout mechanism, so the consequence of an infinite loop will be a cryptic error message, and possibly slowing down other tests that are queued up to run on the same machine. Still, test failures are better than timeouts.

Revision 5: Time is Usually Monotonic

We can get rid of the while (true) if we assume that NTP updates are infrequent, and just bail if the time keeps moving backwards.

var test = function (code) {
for (var i = 0; i < 1000; ++i) {
var startTime = Date.now().valueOf();
var result = code();
var endTime = Date.now().valueOf();
if (startTime > endTime) {
continue; // Time went backwards. (NTP adjustment?)
}
if (!(result instanceof Date) && typeof(result) !== 'number') {
return false;
}
var time = result.valueOf();
return (startTime <= time) && (time <= endTime);
}
console.error("The clock is too broken to be used in tests.");
return false;
}

We still have to assume that when the clock is adjusted backwards, the jumps are relatively large. If there is a small jump backwards right after startTime is read, but code() takes long enough to run, endTime might still be greater than startTime, causing test() to return false even though code() might be correct.

I will stop here for now, but reserve the right to update the aricle if the code above proves insufficient.

Credits

All the clever bits in this article are lifted from code review comments contributed by various Chromium reviewers. The mistakes are all mine.

Running the Blink Layout Tests on Fedora Linux

I use Fedora instead of the recommended Ubuntu distribution. My main motivation is that Fedora generally ships newer packages than Ubuntu, so I get to try out newer versions of software earlier than most people. In the context of Chromium and Blink development, this means I’m more likely to catch bugs like this one.

This is a list of the steps I did to be able to run Blink’s layout tests on Fedora Linux. Layout tests are currently main method of testing in Blink, so being able to run them is a pretty hard prerequisite to working on patches.

Build Prerequisites

The Fedora Setup section of the Linux build prerequisites in the Chromium wiki has two commands that can be conveniently copy-pasted to get most of the packages needed for building Chromium. The optional packages are needed for running Blink’s layout tests, so make sure you run both commands.

The comments section also has a list of packages that should be installed, and somewhat makes up for the fact that the Chromium codebase evolves much faster than the Wiki page.

After installing all the packages, you should be able to build the blink_tests target, which contains the Content Shell and all the tools used to run the layout tests.

cd ~/chromium/src
ninja -C out/Debug blink_tests

Fonts for the Layout Tests

The layout tests refuse to run without the fonts used by the pixel tests. The font list and lookup details are in src/content/shell/app/webkit_test_platform_support_linux.cc. If the path above changes, try searching the codebase for kochi-gothic.ttf.

The easiest method for obtaining the needed fonts is:

  1. Download an Ubuntu desktop ISO.

  2. Use VirtualBox (or your favorite alternative) to set up an Ubuntu VM.

  3. Open up the Ubuntu Setup section in a browser in the VM.

  4. Install the font-related packages (look for font or ttf in the package name) in the VM.

  5. Copy the entire /usr/share/fonts/truetype directory to your Fedora installation.

    mkdir ~/Downloads/ChromiumFonts scp -r you@vmname.local:/usr/share/fonts/truetype ~/Downloads/ChromiumFonts sudo cp -r ~/Downloads/Chromiumfonts/truetype /usr/share/fonts

  6. Check that the layout tests are happy with your font setup.

    cd ~/chromium/src out/Debug/content_shell --dump-render-tree ~/chromium/src/third_party/WebKit/LayoutTests/fast/js/array-indexof.html

  7. Stash the fonts in your Dropbox or Google Drive, so you won’t have to do this again.

  8. Power off and remove the Ubuntu VM.

For whatever it’s worth, the Microsoft fonts can be easily obtained by installing the RPM from the mscorefonts2 project. However, a reasonably thorough search for koichi-gothic.ttf led to no results, so the Ubuntu VM method is the fastest method I know for getting all the fonts.

Story: Debugging a Crash in the Layout Tests Crash

On one occasion, all the layout tests that I ran resulted in crashes. The steps I took might be helpful for shedding light on a similar situation.

First, I first found an “easy” test that really shouldn’t crash the renderer, so I wouldn’t have to worry about whether the crash was introduced by a recent change. My test was fast/js/array-indexof.html, but other tests in that directory might be equally suitable.

Running the test with the extra --driver-logging option revealed the issue. (I was missing fonts.)

webkit/tools/layout_tests/run_webkit_tests.sh —debug —driver-logging fast/js/array-indexof.html

Had that not worked, I would have ran the test in the Content Shell directly. Note that unlike the full Chromium build, the Content Shell can’t convert relative paths into URLs, so I’m giving it a full path. I’m relying on bash to expand ~ to my home directory.

out/Debug/content_shell ~/chromium/src/third_party/WebKit/LayoutTests/fast/js/array-indexof.html

In my case, this command worked, and showed me that the problem was somewhere in the testing infrastructure, so I ran the test with the --dump-render-tree flag.

out/Debug/content_shell —dump-render-tree ~/chromium/src/third_party/WebKit/LayoutTests/fast/js/array-indexof.html

Searching the Chromium Codebase

After I started programming professionally, I was literally shocked when I asked my tech lead “where is X implemented?”, and his answer was “let’s grep for it”. My schoolwork consisted of writing a new piece of software, or completely understanding a piece of software, and then using or modifying it. At the same time, if I wanted to look up a piece of information, I would use Google instead of going to a library and trying to figure out what book might hold my answer.

Long story short, grep is a central piece to finding one’s way in a large codebase, just like Google is the only sensible way of finding specific bits of information in the large pool of human knowledge that we have on the Web.

Git Grep

git grep works for any repository, so it was go-to tool when I started hacking on the Chromium codebase. I still use it when I’m traveling and I don’t have Internet access. For most cases, Google’s code search (covered in the following section) is a better tool.

git grep is better than the plain vanilla grep because it searches over the entire repository. However, Chromium uses many repositories, so git grep does not cover the entire codebase.

For example, the search below does not cover the Blink repository.

cd ~/chromium/src
git grep -n -3 allowImage

Moving to Blink’s root directory will make git grep search the Blink repository. For this reason, I usually have at least one Terminal tab open in ~/chromium/src and one tab open in ~/chromium/src/third_party/WebKit.

cd ~/chromium/src/third_party/WebKit
git grep -n -3 allowImage

I found the command-line arguments above to be generally useful. -n shows line numbers, -3 provides 3 lines of context above and below the match. 3 is not a magic value, all digits work similarly.

Bash assigns special meaning to some characters. Remember to escape them or to quote the pattern when necessary.

cd ~/chromium/src/third_party/WebKit
git grep -n -3 "allowImage("
git grep -n -3 allowImage\(

If you find yourself using git grep a lot (presumably for other projects), consider upgrading to ack.

Code Search: Grep on Steroids

Google’s Code Search service was shut down in 2013, but it is still up and running for the Chromium code base, at cs.chromium.org.

The main advantage of Chromium Code Search is the search speed. For example, time git --no-pager grep -n -3 "allowImage(" took 1.9 seconds in the main Chromium repository and 2.7 seconds in the Blink repository, on a high-end mid-2012 Retina MacBook Pro. Chromium Code search answered the same query for the entire Chromium codebase in 0.2 seconds.

Here are the three features that I didn’t discover right away, but I used a lot once I figured them out.

  1. Hovering over a name turns it into a link to the definition for the respective variable, function, method or class. My Chromium setup opens a link in a new tab when I click on the middle mouse button, and I use this a lot when exploring Chromium code.

  2. Clicking over a name where it is defined opens up the XRefs (cross-references) pane, which is much better than a raw search for commonly used names.

  3. The file tree on the left can be replaced with an outline of the current file. Look for the Outline link above the tree.

If you’d like to use code search for other projects, you might be interested in the open-sourced Google Code Search back-end code. Unfortunately, this is not the full code for the Web application, but rather a starting point. Even if you don’t end up using the implementation there, the comments in the code are very insightful.

Active Exploration

This post covered passively exploring the Chromium codebase by searching. Sometimes, passive search is inefficient, and must be complemented by active exploration methods such as logging and getting stack traces (upcoming post).

References

Blog Setup

This article is a quick description of the software stack used to publish this blog.

Octopress

The blog uses the 2.5 branch of the Octopress static blog generator.

I have set up the Octopress repository as a remote called up (upstream), and my 2.5 branch mirrors the matching Octopress branch. This means that I can adopt updates with the following commands.

git checkout 2.5
git pull up 2.5
git checkout master
git rebase 2.5

I followed the official guides for Initial Setup, Configuration, and Blogging. Although they are written for Octopress 2.0, they still apply to 2.5.

Heroku

The blog is deployed on the Heroku platform, and using their generous free tier. I use one dyno and no plugins.

I did not follow the Octopress deployment guide for Heroku, because I did not agree with the approach of checking in the generated static pages.

Instead, I used an Octopress buildpack, which generates the static pages when I push to Heroku’s git repository.

This build pack is basically Heroku’s official Ruby buildpack, with the Octopress logic borrowed from jgarber’s Octopress buildpack. I’m betting that Heroku will update their buildpack much more frequently than I’ll have to update the Octopress bits, so I’m using a rebasing workflow for the buildpack.

I followed jgarber’s setup guide for the Octopress buildpack, with the exception of the Pygments setup, which is no longer required

Also, since I’m using my own buildpack, the command for setting it up is different from the command in the guide I followed.

Discussion

I really like that I have my contents in a portable form (Markdown), so I can migrate to a different blogging system, such as ghost, if I want to.

I don’t really like the Octopress theme. I like the styling in ghost and the typography in medium. Octopress won because of its simple setup (Ghost stores images in the filesystem, so it doesn’t work well on Heroku), because it’s not restrictive (medium requires draft reviewers to log in with Twitter). Good support for code highlighting doesn’t hurt.

Given an unlimited amount of time, I would have used the middleman static site generator with its support for blogs and syntax highlighting.

Blink: Chromium's Rendering Engine

Blink is Google’s fork of the WebKit browser engine, which in turn is Apple’s fork of KDE’s KHTML engine.

WebKit Legacy

WebKit (the precursor of Blink) was designed to be platform-independent, and was successfully ported to many platforms, such as Mac OS and iOS (using Cocoa), Windows, Linux (using GTK), and Android. Chromium was also designed to be ported to multiple platforms, and adds another platform-independent layer on top of the Blink rendering engine, called the content module. So, from the WebKit code perspective, Blink sits on top of a single platform, called “chromium”.

The WebKit project contains a rendering engine (WebCore), a JavaScript engine (JavaScriptCore), and an embedding layer. Blink adopted WebCore, but uses the V8 JavaScript Engine instead of JavaScriptCore, and dropped the embedding layer in favor of having its content layer talk directly to WebCore.

Code Layout

Blink’s source code is at third_party/WebKit in the Chromium source tree. The top-level directories are as follows.

  • public/ contains the header files makeing up the API between Blink and Chromium’s content layer.
  • Source contains the actual Blink source code.
  • LayoutTests has automated tests for the engine’s features.
  • Tools contains scripts for building and running the code. Tools/Scripts is heavily referenced in WebKit’s old

I have not explored the PerformanceTests or ManualTests directories. They appear to be described by the WebKit Wiki’s Performance Tests page and WebKit Wiki’s Manual Testing page.

At the time of this writing, WebKit’s source tree has the same top-level structure. Comparing the Source/ directories shows the top-level differences between Blink and WebKit.

The Public API

The headers in public/ are relatively well-commented, so they make a great starting point for understanding a part of Chromium.

The public APIs fall into two big categories.

  • public/web contains the API implemented by Blink and called by Chromium’s content layer
  • public/platform has the lower-level API used by Blink and implemented by the content layer

The classes in the public APIs usually have easy-to-guess corresponding classes in the implementation. For example, the WebView abstract class, defined in public/web/WebView.h, has one concrete implementation, WebViewImpl, which is defined in Source/web/WebViewImpl.h. Source/web/WebViewImpl.cpp contains both the WebViewImpl implementation, as well as the definitions for WebView’s static members.

The Source Code

Blink’s implementation is split into the following top-level directories.

The Layout Tests

Blink’s regression tests are Web pages that are loaded in a content layer that is hacked to output textual representations of the pages. This is really convenient, because a Web application developer can contribute a failing test for a bug or feature without having to learn C++ or Blink’s inner workings.

To get a taste of what tests look like, skim LayoutTests/fast/xmlhttprequest-get.xhtml and the corresponding LayoutTests/fast/xmlhttprequest-get-expected.txt.

Building and Testing

The all_webkit target contains the Blink-related deliverables.

ninja -C out/Debug blink_tests

Blink cannot run on its own, due to its dependency on Chromium’s content layer. The Content Shell offers a rudimentary UI on top of the content layer, and is a low-overhead way of getting Blink up and running.

# Linux
out/Debug/content_shell
# OS X
out/Debug/Content\ Shell.app/Contents/MacOS/Content\ Shell

Further Reading

At the time of this writing, the Technical Articles in the WebKit blog and many of the entries on the WebKit Wiki are still relevant and worth the time spent reading them.

References

Logging in Chromium

To me, logging is a fancy way of saying printf-debugging. In a large project, like Chromium, it comes in very handy for learning about the code. Strategically-placed logging statements can help you prove that a function is called in response to an action, or figure out the values that some parameters take.

Chromium puts together many different projects, and they all have their logging subsystems. I will cover logging for the projects that I worked with. That being said, I’m a people pleaser, so I am likely to add a project, if you ask nicely.

Chromium

The Chromium source code follows Google’s coding style, and the logging API resembles the API used in Google’s internal codebases. In particular, most logging statements are compiled in both the release and the debugging builds, and can be activated via command-line flags.

To enable Chromium logging, run your build with the following arguments.

path/to/chromium —enable-logging=stderr —v=1

Logging statemtents look like using streams in the C++ standard library.

#include "base/logging.h"
VLOG(1) << "SomeFunction(" << arg << ")";

Using DVLOG instead of VLOG causes a logging statement not to be compiled into release builds. You shouldn’t worry about this while you use logging for exploratoraty purposes. When you get in a position to commit a logging statement, your code reviewer can help you figure out the specific logging statement that you should use.

Blink

The Blink source code in /third_party/WebKit uses Apple-style logging. Most importantly, logging statements are only compiled in debugbuilds. Unfortunately, this means that if you need logging, you can’t use release builds, which are faster to produce.

To enable Blink logging, run your (debug) build with the following arguments.

path/to/chromium —webcore-log-channels=Loading,ResourceLoading

The valid channels are defined in Logging.cpp in the WebKit source tree. The file has moved in the recent past, so the command below is a good way of finding it.

cd ~/chromium/src/third_party/WebKit
find . | grep Logging.cpp

Logging statements look like printf calls.

#include "platform/Logging.h"
LOG(Loading, "ResourceFetcher::requestResource %s", url.latin1().data());

V8

V8 can run on its own, so you should try to do your development that way whenever possible.

To enable V8 logging, run your build with the following arguments.

path/to/chromium —js-flags="—log_all —logfile=–"

Running a tool such as tick-processor requires that V8’s logging output is neatly separated from Chromium’s. To have V8 output its log to a file, Chromium’s sandbox must be disabled. By default, V8 logs to v8.log.

path/to/chromium —no-sandbox —js-flags="—log_all"

I haven’t hacked on the V8 source code yet, so I don’t have an example of writing logging statements.

References

Build Your Own Chromium

I initially planned to start this series with some introductory material, such as giving a tour of the project, or explaining why I think contributing is a worthy endeavor.

At the same time, setting up Chromium will keep your development machine busy for a long time. Start this right away, so you won’t be blocked waiting for your computer later on.

Prerequisites

Chromium development requires a reasonably powerful machine. Your computer should have at least 4GB of RAM and at least 60GB of free disk space.

This article contains an easy way of getting a Chromium development environment. For the sake of simplicity, I made some decisions for you, and didn’t waste space documenting alternatives.

You can make different choices, and you’re responsible for figuring out how they impact my instructions. In general, you can get away with any change, as long as you can read the full Chromium guides, and use Google and StackOverflow to figure out error messages.

Use a UNIX Operating System

My articles assume you use a UNIX operating system. You’ll have the easiest time developing on OS X, but running it requires an Apple computer. For PCs, the best environment for Chromium development is the Ubuntu Linux distribution.

Mainstream Linux distributions, such as Fedora and Arch are also supported, but you’ll have to do a bit more research and debugging to if some of the commands don’t work right away.

Windows is not suitable for Chrome development, and I recommend against using it. The only good reason for running Windows on a machine is hardware support. If your machine needs Windows drivers, you should use a virtualization product, such as VirtualBox or VMWare Player, and get Ubuntu Linux. You should allocate at least 4GB of RAM and 64 GB of disk space to the VM that you use to run Ubuntu.

Install Git

You need git to be able to contribute to many popular open-source software. If you don’t know how to use git, worry about that later. Remember we’re trying to kick off all the downloads you need to hack on Chromium, so you can read up on stuff you need while your dev machine downloads code.

On OS X, installing Apple’s XCode will give you git and all the other tools you need to build Chromium.

On Ubuntu, run the command below in Terminal. Other Linux distributions require similar commands.

sudo apt-get install git

Install depot_tools

Chromium uses code from dozens (if not hundreds) of code repositories. The only sane way of getting all the code is to use depot_tools, which is their repository management software.

Copy-paste the commands below in Terminal.

Set Your Environment Variables

The Chromium tools use a few environment variables. To make your life easy, they should be always set on your development machine. The best way to achieve that is to add the lines below to ~/.bash_profile (on OS X) or to ~/.bashrc (on Linux).

export GYP_GENERATORS=ninja
export PATH=$PATH:$HOME/depot_tools
export CHROME_DEVEL_SANDBOX=$HOME/chromium/src/out/Debug/chrome_sandbox

On OS X, the easiest way of editing the file I mentioned is to run the commands below in Terminal, and adding the lines above in the TextEdit window that shows up.

touch ~/.bash_profile
open ~/.bash_profile

On Ubuntu Linux, use the commands below instead.

touch ~/.bashrc
gedit ~/.bashrc

Get the Chromium Source Code

You are now ready to download the all source code used to build Chromium. If you have a laptop, go to the place with the best Internet connection that you have access to, because you’ll be downloading a few gigabytes of code.

Run the commands below in Terminal.

cd ~
mkdir chromium
cd chromium
fetch —nohooks —no-history chromium —nosvn=True

Handling fetch Errors

The fetch command may fail if either your Internet connection or Google’s server goes down during the (very long) download. If that happens, remove everything and try again.

cd ~
rm -rf chromium

Get Libraries and Tools

On OS X, XCode contains all the tools you need to build Chromium.

On Ubuntu, run the commands below in Terminal.

cd ~/chromium/src
./build/install-build-deps.sh

For other Linux distributions, find the relevant section under Distribution-specific Notes in Chromium’s Linux Build Prerequisites wiki page, and follow the instructions there. The official instructions tend to be out of date, so read through the comments section on the page for updates.

Build Chromium

The Chromium build process takes up a few hours and all the CPU cycles that your machine has to spare, causing its fans to blow at full speed. Make sure this is appropriate in your environment.

Run the commands below in Terminal to build Chromium.

cd ~/chromium
gclient runhooks —force
cd src
ninja -C out/Debug chrome

Dealing with Build Errors

If you’re unlucky, you might get build errors. This usually happens when some tool or library (such as the compiler installed by Xcode) is updated, and Chromium hasn’t caught up with the changes. Before trying anything else, update the source code using the commands below, then run the build commands above again.

cd ~/chromium
git pull && gclient sync

If you’re particularly unlucky, you’ll have to figure out the error by yourself. It usually comes down to missing a library or tool.

On OSX, setting up homebrew is the easiest way to get missing libraries. On Ubuntu, auto-apt will help you figure out which package you need to install to get a missing file.

Set Up the Sandbox

Chromium uses sandboxing to reduce the amount of damage that a security vulnerability can cause. You don’t need to understand how it works right now, but you do need to run the commands below once to get your Chromium build running.

cd ~/chromium/src
ninja -C out/Debug chrome_sandbox
sudo chown root:root out/Debug/chrome_sandbox
sudo chmod 4755 out/Debug/chrome_sandbox

Run Chromium

If you made it this far, your development machine is all set up for hacking on Chromium. Check your work by running the Chromium binary that you have just built.

On OS X, run the commands below.

cd ~/chromium/src
out/Debug/Chromium.app/Contents/MacOS/Chromium

On Linux, use these commands instead.

cd ~/chromium/src
out/Debug/chrome

Congratulations! You are now ready to change the world!

References

I used information from the Chromium guides below.