XSS Auditor has been removed from Chromeß. The XSS Auditor can introduce cross-site information leaks and mechanisms to bypass the Auditor are widely known.
Intent to Remove | Chrome Platform Status | Chromium Bug
Stylus-based drawing applications built for the web have long suffered from latency issues because a web page has to synchronize graphics updates with the DOM. In any drawing application, latencies longer than 50 milliseconds can interfere with a user's hand-eye coordination, making applications difficult to use.
The desynchronized hint for canvas.getContext() invokes a different code
path that bypasses the usual DOM update
mechanism.
Instead the hint tells the underlying system to skip as much compositing as it
is able and in some cases, the canvas's underlying buffer is sent directly to
the screen's display controller. This eliminates the latency that would be
caused by using the renderer compositor queue.
If you want to get to the code, scroll ahead. To see it in action, you need a device with a touch screen, and preferably a stylus. (Fingers work too.) If you have one, try the 2d or webgl samples. For the rest of you check out this demo by Miguel Casas, one of the engineers who implemented this feature. Open the demo, press play, then move the slider back and forth randomly and quickly.
This example uses a one-minute, twenty-one second clip from the short film
Sintel by Durian, the Blender open movie
project. In this example, the movie is played in a <video> element whose
contents are simultaneously rendered to a <canvas> element. Many devices can
do this without tearing, though devices with front buffer rendering such as
Chrome OS for example may have tearing. (The movie is great, but heartbreaking.
I was useless for an hour after I saw it. Consider yourself warned.)
There's more to using low latency than adding desynchronized to
canvas.getContext(). I'll go over the issues one at a time.
On another API I'd discuss feature detection first. For the desynchronized
hint you must create the canvas first. Call canvas.getContext() and pass it
the new desynchronized hint with a value of true.
const canvas = document.querySelector('myCanvas');
const ctx = canvas.getContext('2d', {
desynchronized: true,
// Other options. See below.
});
Next, call getContextAttributes(). If the returned attributes object has a
desynchronized property, then test it.
if (ctx.getContextAttributes().desynchronized) {
console.log('Low latency canvas supported. Yay!');
} else {
console.log('Low latency canvas not supported. Boo!');
}
There are two instances where you can cause flicker if you don't code correctly.
Some browsers including Chrome clear WebGL canvases between frames. It's
possible for the display controller to read the buffer while it's empty causing
the image being drawn to flicker. To avoid this is to set
preserveDrawingBuffer to true.
const canvas = document.querySelector('myCanvas');
const ctx = canvas.getContext('webgl', {
desynchronized: true,
preserveDrawingBuffer: true
});
Flicker can also occur when you clear the screen context in your own drawing code. If you must clear, draw to an offscreen framebuffer then copy that to the screen.
A translucent canvas element, one where alpha is set to true, can still be desynchronized, but it must not have any other DOM elements above it.
You cannot change the context attributes after the first call to
canvas.getContext(). This has always been true, but repeating it might save
you some frustration if you're unaware or have forgotten .
For example, let's say that I get a context and specify alpha as false, then
somewhere later in my code I call canvas.getContext() a second time with alpha
set to true as shown below.
const canvas = document.querySelector('myCanvas');
const ctx1 = canvas.getContext('2d', {
alpha: false,
desynchronized: true,
});
//Some time later, in another corner of code.
const ctx2 = canvas.getContext('2d', {
alpha: true,
desynchronized: true,
});
It's not obvious that ctx1 and ctx2 are the same object. Alpha is still false and a
context with alpha equal to true is never created.
The first parameter passed to getContext() is the contextType. If you're
already familiar with getContext() you're no doubt wondering if anything
other than '2d' context types are supported. The table below shows the context
types that support desynchronized.
| contextType | Context type object |
|---|---|
|
|
|
|
|
|
If you want to see more of this, check out the samples. In addition to the video example already described there are examples showing both '2d' and 'webgl' contexts.
For a while now, Chrome has eagerly cleared the screen when transitioning to a new page to give users the reassurance that the page is loading. This "flash of white" is this brief moment during which the browser shows a white paint while loading a page. This can be distracting in-between navigations, especially when the page is reasonably fast in reaching a more interesting state.
But for pages that load lightning fast, this approach is actually detrimental to the user experience. In the following animation, you see an example of what this looks like today.
We are big fans of this website and it kills us that their quality experience has a flash of white, and we wanted to fix it. We did so with a new behavior that we're calling Paint Holding, where the browser waits briefly before starting to paint, especially if the page is fast enough. This ensures that the page renders as a whole delivering a truly instant experience.
The way this works is that we defer compositor commits until a given page load signal (PLS) (e.g. first contentful paint / fixed timeout) is reached. We distinguish between main-thread rendering work and commit to the impl thread (only the latter is deferred). Waiting until a PLS occurs reduces likelihood of flashes of white/solid-color.
Our goal with this work was for navigations in Chrome between two pages that are of the same origin to be seamless and thus deliver a fast default navigation experience with no flashes of white/solid-color background between old and new content.
Try Paint Holding in Chrome Canary (Chrome 76) and let us know what you think. Developers shouldn't have to worry about making any modifications to their pages to take advantage of it.
Lighthouse 2.8 is out! Highlights include:
See the 2.8 release notes for the full list of new features, changes, and bug fixes.
npm update lighthouse, or npm update lighthouse -g flag if you installed
Lighthouse globally.chrome://extensions.chrome://version. Chrome updates to a new version about every
6 weeks. You can run the latest Chrome code by downloading Chrome Canary.The Avoid Plugins audit lists plugins that you should remove, since plugins prevent the page from being mobile-friendly. Most mobile devices don't support plugins.
The Document Has A Valid rel=canonical audit in the SEO category checks for a
rel=canonical URL to make sure that a crawler knows which URL to show in search results.
rel=canonical audit
The Page Is Mobile-Friendly and Structured Data Is Valid manual audits can help further improve your SEO. "Manual" in this case means that Lighthouse can't automate these audits, so you need to test them yourself.
The Minify CSS and Minify JavaScript audits in the Performance category check for any CSS or Javascript that can be minified to reduce payload size and parse time.
Performance is now the first category you see in Lighthouse reports. Some users thought that Lighthouse was only for Progressive Web Apps, since that was the first category in reports. In reality, Lighthouse can help you understand how to improve any web page, whether or not it's a Progressive Web App.
If an accessibility audit is not applicable for a given page, that audit no longer counts towards the Accessibility score.
Note: This update is only visible when you run Lighthouse from the Audits panel of Chrome DevTools.
Check out the Release Guide For Maintainers for information on release timing, naming conventions, and more.
Sometimes you want to use a library that is only available as C or C++ code. Traditionally, this is where you give up. Well, not anymore, because now we have Emscripten and WebAssembly (or Wasm)!
Note: In this article, I will describe my journey of compiling libwebp to Wasm. To make use of this article as well as Wasm in general, you will need knowledge of C, especially pointers, memory management and compiler options.
I set myself the goal of working out how to compile some existing C code to Wasm. There's been some noise around LLVM's Wasm backend, so I started digging into that. While you can get simple programs to compile this way, the second you want to use C's standard library or even compile multiple files, you will probably run into problems. This led me to the major lesson I learned:
While Emscripten used to be a C-to-asm.js compiler, it has since matured to target Wasm and is in the process of switching to the official LLVM backend internally. Emscripten also provides a Wasm-compatible implementation of C's standard library. Use Emscripten. It carries a lot of hidden work, emulates a file system, provides memory management, wraps OpenGL with WebGL — a lot of things that you really don't need to experience developing for yourself.
While that might sound like you have to worry about bloat — I certainly worried — the Emscripten compiler removes everything that's not needed. In my experiments, the resulting Wasm modules are appropriately sized for the logic that they contain and the Emscripten and WebAssembly teams are working on making them even smaller in the future.
You can get Emscripten by following the instructions on their website or using Homebrew. If you are a fan of dockerized commands like me and don't want to install things on your system just to have a play with WebAssembly, there is a well-maintained Docker image that you can use instead:
$ docker pull trzeci/emscripten
$ docker run --rm -v $(pwd):/src trzeci/emscripten emcc <emcc options here>
Let's take the almost canonical example of writing a function in C that calculates the nth fibonacci number:
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
int fib(int n) {
if(n <= 0){
return 0;
}
int i, t, a = 0, b = 1;
for (i = 1; i < n; i++) {
t = a + b;
a = b;
b = t;
}
return b;
}
If you know C, the function itself shouldn't be too surprising. Even if you don't know C but know JavaScript, you will hopefully be able to understand what's going on here.
emscripten.h is a header file provided by Emscripten. We only need it so we
have access to the EMSCRIPTEN_KEEPALIVE macro, but it provides much more
functionality.
This macro tells the compiler to not remove a function even if it appears
unused. If we omitted that macro, the compiler would optimize the function away
— nobody is using it after all.
Let's save all that in a file called fib.c. To turn it into a .wasm file we
need to turn to Emscripten's compiler command emcc:
$ emcc -O3 -s WASM=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap"]' fib.c
Let's dissect this command. emcc is Emscripten's compiler. fib.c is our C
file. So far, so good. -s WASM=1 tells Emscripten to give us a Wasm file
instead of an asm.js file. -s
EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap"]' tells the compiler to leave the
cwrap() function available in the JavaScript file — more on this function
later. -O3 tells the compiler to optimize aggressively. You can choose lower
numbers to decrease build time, but that will also make the resulting bundles
bigger as the compiler might not remove unused code.
After running the command, you should end up with a JavaScript file called
a.out.js and a WebAssembly file called a.out.wasm. The Wasm file (or
"module") contains our compiled C code and should be fairly small. The
JavaScript file takes care of loading and initializing our Wasm module and
providing a nicer API. If needed, it will also take care of setting up the stack,
the heap, and other functionality usually expected to be provided by the
operating system when writing C code. As such, the JavaScript file is a bit
bigger, weighing in at 19KB (~5KB gzip'd).
The easiest way to load and run your module is to use the generated JavaScript
file. Once you load that file, you will have a Module
global
at your disposal. Use
cwrap
to create a JavaScript native function that takes care of converting parameters
to something C-friendly and invoking the wrapped function. cwrap takes the
function name, return type and argument types as
arguments, in that order:
<script src="a.out.js"></script>
<script>
Module.onRuntimeInitialized = _ => {
const fib = Module.cwrap('fib', 'number', ['number']);
console.log(fib(12));
};
</script>
If you run this code, you should see the "144" in the console, which is the 12th Fibonacci number.
Note: Emscripten offers a couple of options to handle loading multiple modules. More about that in their documentation.
Up until now, the C code we have written was written with Wasm in mind. A core use-case for WebAssembly, however, is to take the existing ecosystem of C libraries and allow developers to use them on the web. These libraries often rely on C's standard library, an operating system, a file system and other things. Emscripten provides most of these features, although there are some limitations.
Let's go back to my original goal: compiling an encoder for WebP to Wasm. The source for the WebP codec is written in C and available on GitHub as well as some extensive API documentation. That's a pretty good starting point.
$ git clone https://github.com/webmproject/libwebp
To start off simple, let's try to expose WebPGetEncoderVersion() from
encode.h to JavaScript by writing a C file called webp.c:
#include "emscripten.h"
#include "src/webp/encode.h"
EMSCRIPTEN_KEEPALIVE
int version() {
return WebPGetEncoderVersion();
}
This is a good simple program to test if we can get the source code of libwebp to compile, as we don't require any parameters or complex data structures to invoke this function.
To compile this program, we need to tell the compiler where it can find
libwebp's header files using the -I flag and also pass it all the C files of
libwebp that it needs. I'm going to be honest: I just gave it all the C
files I could find and relied on the compiler to strip out everything that was
unnecessary. It seemed to work brilliantly!
$ emcc -O3 -s WASM=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap"]' \
-I libwebp \
webp.c \
libwebp/src/{dec,dsp,demux,enc,mux,utils}/*.c
Note: This strategy will not work with every C project out there. Many projects
rely on autoconf/automake to generate system-specific code before compilation.
Emscripten provides emconfigure and emmake to wrap these commands and inject
the appropriate parameters. You can find more in the Emscripten
documentation.
Now we only need some HTML and JavaScript to load our shiny new module:
<script src="/a.out.js"></script>
<script>
Module.onRuntimeInitialized = async _ => {
const api = {
version: Module.cwrap('version', 'number', []),
};
console.log(api.version());
};
</script>
And we will see the correction version number in the output:
Note: libwebp returns the current version a.b.c as a hexadecimal number 0xabc. So v0.6.1 is encoded as 0x000601 = 1537.
Getting the encoder's version number is great and all, but encoding an actual image would be more impressive, right? Let's do that, then.
The first question we have to answer is: How do we get the image into Wasm land?
Looking at the encoding API of
libwebp,
it expects an array of bytes in RGB, RGBA, BGR or BGRA. Luckily, the Canvas API
has
getImageData(),
that gives us an
Uint8ClampedArray
containing the image data in RGBA:
async function loadImage(src) {
// Load image
const imgBlob = await fetch(src).then(resp => resp.blob());
const img = await createImageBitmap(imgBlob);
// Make canvas same size as image
const canvas = document.createElement('canvas');
canvas.width = img.width;
canvas.height = img.height;
// Draw image onto canvas
const ctx = canvas.getContext('2d');
ctx.drawImage(img, 0, 0);
return ctx.getImageData(0, 0, img.width, img.height);
}
Now it's "only" a matter of copying the data from JavaScript land into Wasm land. For that, we need to expose two additional functions. One that allocates memory for the image inside Wasm land and one that frees it up again:
EMSCRIPTEN_KEEPALIVE
uint8_t* create_buffer(int width, int height) {
return malloc(width * height * 4 * sizeof(uint8_t));
}
EMSCRIPTEN_KEEPALIVE
void destroy_buffer(uint8_t* p) {
free(p);
}
create_buffer allocates a buffer for the RGBA image — hence 4 bytes per
pixel. The pointer returned by malloc() is the address of the first memory
cell of that buffer. When the pointer is returned to JavaScript land, it is
treated as just a number. After exposing the function to JavaScript using
cwrap, we can use that number to find the start of our buffer and copy the
image data.
const api = {
version: Module.cwrap('version', 'number', []),
create_buffer: Module.cwrap('create_buffer', 'number', ['number', 'number']),
destroy_buffer: Module.cwrap('destroy_buffer', '', ['number']),
};
const image = await loadImage('/image.jpg');
const p = api.create_buffer(image.width, image.height);
Module.HEAP8.set(image.data, p);
// ... call encoder ...
api.destroy_buffer(p);
The image is now available in Wasm land. It is time to call the WebP encoder to
do its job! Looking at the WebP
documentation,
WebPEncodeRGBA seems like a perfect fit. The function takes a pointer to the
input image and its dimensions, as well as a quality option between 0 and 100.
It also allocates an output buffer for us, that we need to free using
WebPFree() once we are done with the WebP image.
The result of the encoding operation is an output buffer and its length. Because functions in C can't have arrays as return types (unless we allocate memory dynamically), I resorted to a static global array. I know, not clean C (in fact, it relies on the fact that Wasm pointers are 32bit wide), but to keep things simple I think this is a fair shortcut.
int result[2];
EMSCRIPTEN_KEEPALIVE
void encode(uint8_t* img_in, int width, int height, float quality) {
uint8_t* img_out;
size_t size;
size = WebPEncodeRGBA(img_in, width, height, width * 4, quality, &img_out);
result[0] = (int)img_out;
result[1] = size;
}
EMSCRIPTEN_KEEPALIVE
void free_result(uint8_t* result) {
WebPFree(result);
}
EMSCRIPTEN_KEEPALIVE
int get_result_pointer() {
return result[0];
}
EMSCRIPTEN_KEEPALIVE
int get_result_size() {
return result[1];
}
Now with all of that in place, we can call the encoding function, grab the pointer and image size, put it in a JavaScript-land buffer of our own, and release all the Wasm-land buffers we have allocated in the process.
api.encode(p, image.width, image.height, 100);
const resultPointer = api.get_result_pointer();
const resultSize = api.get_result_size();
const resultView = new Uint8Array(Module.HEAP8.buffer, resultPointer, resultSize);
const result = new Uint8Array(resultView);
api.free_result(resultPointer);
Note: new Uint8Array(someBuffer) will create a new view onto the same memory
chunk, while new Uint8Array(someTypedArray) will copy the data.
Depending on the size of your image, you might run into an error where Wasm can't grow the memory enough to accommodate both the input and the output image:
Luckily, the solution to this problem is in the error message! We just need to
add -s ALLOW_MEMORY_GROWTH=1 to our compilation command.
And there you have it! We compiled a WebP encoder and transcoded a JPEG image to
WebP. To prove that it worked, we can turn our result buffer into a blob and use
it on an <img> element:
const blob = new Blob([result], {type: 'image/webp'});
const blobURL = URL.createObjectURL(blob);
const img = document.createElement('img');
img.src = blobURL;
document.body.appendChild(img)
Behold, the glory of a new WebP image!
It's not a walk in the park to get a C library to work in the browser, but once you understand the overall process and how the data flow works, it becomes easier and the results can be mind-blowing.
WebAssembly opens many new possibilities on the web for processing, number crunching and gaming. Keep in mind that Wasm is not a silver bullet that should be applied to everything, but when you hit one of those bottlenecks, Wasm can be an incredibly helpful tool.
If you want to try and avoid the generated JavaScript file, you might be able to. Let's go back to the Fibonacci example. To load and run it ourselves, we can do the following:
<!doctype html>
<script>
(async function() {
const imports = {
env: {
memory: new WebAssembly.Memory({initial: 1}),
STACKTOP: 0,
}
};
const {instance} = await WebAssembly.instantiateStreaming(fetch('/a.out.wasm'), imports);
console.log(instance.exports._fib(12));
})();
</script>
Note: Make sure that your .wasm files have Content-Type: application/wasm.
Otherwise they will be rejected by WebAssembly.
WebAssembly modules that have been created by Emscripten have no memory to work
with unless you provide them with memory. The way you provide a Wasm module with
anything is by using the imports object — the second parameter of the
instantiateStreaming function. The Wasm module can access everything inside
the imports object, but nothing else outside of it. By convention, modules
compiled by Emscripting expect a couple of things from the loading JavaScript
environment:
env.memory. The Wasm module is unaware of the outside
world so to speak, so it needs to get some memory to work with. Enter
WebAssembly.Memory.
It represents a (optionally growable) piece of linear memory. The sizing
parameters are in "in units of WebAssembly pages", meaning the code above
allocates 1 page of memory, with each page having a size of 64
KiB. Without providing a maximum
option, the memory is theoretically unbounded in growth (Chrome currently has a
hard limit of 2GB). Most WebAssembly modules shouldn't need to set a maximum.env.STACKTOP defines where the stack is supposed to start growing. The stack
is needed to make function calls and to allocate memory for local variables.
Since we don't do any dynamic memory management shenanigans in our little
Fibonacci program, we can just use the entire memory as a stack, hence STACKTOP
= 0.The immersive web means virtual world experiences hosted through the browser. This covers entire virtual reality (VR) experiences surfaced in the browser or in VR enabled headsets like Google's Daydream, Oculus Rift, Samsung Gear VR, HTC Vive, and Windows Mixed Reality Headsets, as well as augmented reality experiences developed for AR-enabled mobile devices.
Though we use two terms to describe immersive experiences, they should be thought of as a spectrum from complete reality to a completely immersive VR environment, with various levels of AR in between.
Examples of immersive experiences include:
The immersive web has been available for nearly a year now in embryonic form. This was done through the WebVR 1.1 API, which has been available in an origin trial since Chrome 62. That API is also supported by Firefox and Edge as well as a polyfill for Safari.
But it's time to move on.
The origin trial ended on July 24, 2018, and the spec has been superseded by the WebXR Device API and a new origin trial.
Note: If you're participating in the WebVR origin trial, you need a separate registration for the WebXR Origin Trial (explainer, sign-up form).
We learned a lot from WebVR 1.1, but over time, it became clear that some major changes were needed to support the types of applications developers want to build. The full list of lessons learned is too long to go into here, but includes issues like the API being explicitly tied to the main JavaScript thread, too many opportunities for developers to set up obviously wrong configurations, and common uses like magic window being a side effect rather than an intentional feature. (Magic window is a technique for viewing immersive content without a headset wherein the app renders a single view based on the device's orientation sensor.)
The new design facilitates simpler implementations and large performance improvements. At the same time, AR and other use cases were emerging and it became important that the API be extensible to support those in the future.
The WebXR Device API was designed and named with these expanded use cases in mind and provides a better path forward. Implementors of WebVR have committed to migrating to the WebXR Device API.
Like the WebVR spec before it, the WebXR Device API is a product of the Immersive Web Community Group which has contributors from Google, Microsoft, Mozilla, and others. The 'X in XR is intended as a sort of algebraic variable that stands for anything in the spectrum of immersive experiences. It's available in the previously mentioned origin trial as well as through a polyfill.
When this article was originally published during the Chrome 67 beta period, only VR capabilities were enabled. Augmented reality arrived in Chrome 69. Read about it in Augmented reality for the web.
There's more to this new API than I can go to in an article like this. I want to give you enough to start making sense of the WebXR samples. You can find more information in both the original explainer and our Immersive Web Early Adopters Guide. I'll be expanding the latter as the origin trial progresses. Feel free to open issues or submit pull requests.
Note: The Early Adopters Guide is updated when spec changes land in Chrome. To be notified of those updates watch its repo on GitHub.
For this article, I'm going to discuss starting, stopping and running an XR session, plus a few basics about processing input.
What I'm not going to cover is how to draw AR/VR content to the screen. The WebXR Device API does not provide image rendering features. That's up to you. Drawing is done using WebGL APIs. You can do that if you're really ambitious. Though, we recommend using a framework. The immersive web samples use one created just for the demos called Cottontail. Three.js has supported WebXR since May. I've heard nothing about A-Frame.
The basic process is this:
Let's look at this in a little more detail and include some code. You won't be able to run an app from what I'm about to show you. But again, this is just to give a sense of it.
Here, you'll recognize the standard feature detection code. You could wrap this
in a function called something like checkForXR().
If you're not using an immersive session you can skip advertising the functionality and getting a user gesture and go straight to requesting a session. An immersive session is one that requires a headset. A non-immersive session simply shows content on the device screen. The former is what most people think of when you refer to virtual reality or augmented reality. The latter is sometimes called a 'magic window'.
if (navigator.xr) {
navigator.xr.requestDevice()
.then(xrDevice => {
// Advertise the AR/VR functionality to get a user gesture.
})
.catch(err => {
if (err.name === 'NotFoundError') {
// No XRDevices available.
console.error('No XR devices available:', err);
} else {
// An error occurred while requesting an XRDevice.
console.error('Requesting XR device failed:', err);
}
})
} else{
console.log("This browser does not support the WebXR API.");
}
Now that we have our device and our user gesture, it's time to get a session. To create a session, the browser needs a canvas on which to draw.
xrPresentationContext = htmlCanvasElement.getContext('xrpresent');
let sessionOptions = {
// The immersive option is optional for non-immersive sessions; the value
// defaults to false.
immersive: false,
outputContext: xrPresentationContext
}
xrDevice.requestSession(sessionOptions)
.then(xrSession => {
// Use a WebGL context as a base layer.
xrSession.baseLayer = new XRWebGLLayer(session, gl);
// Start the render loop
})
The code for this step takes a bit of untangling. To untangle it, I'm about to throw a bunch of words at you. If you want a peek at the final code, jump ahead to have a quick look then come back for the full explanation. There's quite a bit that you may not be able to infer.
The basic process for a render loop is this:
The word 'frame' has several meanings in a Web XR context. The first is the frame of reference which defines where the origin of the coordinate system is calculated from, and what happens to that origin when the device moves. (Does the view stay the same when the user moves or does it shift as it would in real life?)
The second type of frame is the presentation frame, represented by an
XRFrame object. This object contains the information needed to
render a single frame of an AR/VR scene to the device. This is a bit confusing
because a presentation frame is retrieved by calling requestAnimationFrame().
This makes it compatible with window.requestAnimationFrame().
Before I give you any more to digest, I'll offer some code. The sample below
shows how the render loop is started and maintained. Notice the dual use of the
word frame. And notice the recursive call to requestAnimationFrame(). This
function will be called 60 times a second.
xrSession.requestFrameOfReference('eye-level')
.then(xrFrameOfRef => {
xrSession.requestAnimationFrame(onFrame(time, xrFrame) {
// The time argument is for future use and not implemented at this time.
// Process the frame.
xrFrame.session.requestAnimationFrame(onFrame);
}
});
Before drawing anything to the screen, you need to know where the display
device is pointing and you need access to the screen. In general, the position
and orientation of a thing in AR/VR is called a pose. Both viewers and input
devices have a pose. (I cover input devices later.) Both viewer and input
device poses are defined as a 4 by 4 matrix stored in a Float32Array in column
major order. You get the viewer's pose by calling
XRFrame.getDevicePose() on the current animation frame object.
Always test to see if you got a pose back. If something went wrong you don't
want to draw to the screen.
let pose = xrFrame.getDevicePose(xrFrameOfRef);
if (pose) {
// Draw something to the screen.
}
After checking the pose, it's time to draw something. The object you draw to is
called a view (XRView). This is where the session type becomes important. Views
are retrieved from the XRFrame object as an array. If you're in a
non-immersive session the array has one view. If you're in an immersive
session, the array has two, one for each eye.
for (let view of xrFrame.views) {
// Draw something to the screen.
}
This is an important difference between WebXR and other immersive systems. Though it may seem pointless to iterate through one view, doing so allows you to have a single rendering path for a variety of devices.
If I put all this together, I get the code below. I've left a placeholder for the input devices, which I'll cover in a later section.
xrSession.requestFrameOfReference('eye-level')
.then(xrFrameOfRef => {
xrSession.requestAnimationFrame(onFrame(time, xrFrame) {
// The time argument is for future use and not implemented at this time.
let pose = xrFrame.getDevicePose(xrFrameOfRef);
if (pose) {
for (let view of xrFrame.views) {
// Draw something to the screen.
}
}
// Input device code will go here.
frame.session.requestAnimationFrame(onFrame);
}
}
An XR session may end for several reasons, including ending by your own code
through a call to XRSession.end(). Other causes include the headset being
disconnected or another application taking control of it. This is why a
well-behaved application should monitor the end event and when it occurs,
discard the session and renderer objects. An XR session once ended cannot be
resumed.
xrDevice.requestSession(sessionOptions)
.then(xrSession => {
// Create a WebGL layer and initialize the render loop.
xrSession.addEventListener('end', onSessionEnd);
});
// Restore the page to normal after immersive access has been released.
function onSessionEnd() {
xrSession = null;
// Ending the session stops executing callbacks passed to the XRSession's
// requestAnimationFrame(). To continue rendering, use the window's
// requestAnimationFrame() function.
window.requestAnimationFrame(onDrawFrame);
}
As with the application lifetime, I'm just going to give you a taste for how to interact with objects in AR or VR.
The WebXR Device API adopts a "point and click" approach to user input. With
this approach every input source has a defined pointer ray to indicate where
an input device is pointing and events to indicate when something was selected.
Your app draws the pointer ray and shows where it's pointed. When the user
clicks the input device, events are fired—select, selectStart, and
selectEnd, specifically. Your app determines what was clicked and responds
appropriately.
To users, the pointer ray is just a faint line between the controller and whatever they're pointing at. But your app has to draw it. That means getting the pose of the input device and drawing a line from its location to an object in AR/VR space. That process looks roughly like this:
let inputSources = xrSession.getInputSources();
for (let xrInputSource of inputSources) {
let inputPose = frame.getInputPose(inputSource, xrFrameOfRef);
if (!inputPose) {
continue;
}
if (inputPose.gripMatrix) {
// Render a virtual version of the input device
// at the correct position and orientation.
}
if (inputPose.pointerMatrix) {
// Draw a ray from the gripMatrix to the pointerMatrix.
}
}
This is a stripped down version of the Input Tracking sample from the Immersive Web Community Group. As with frame rendering, drawing the pointer ray and the device is up to you. As alluded to earlier, this code must be run as part of the render loop.
Merely pointing at things in AR/VR is pretty useless. To do anything useful,
users need the ability to select things. The WebXR Device API provides three
events for responding to user interactions: select, selectStart, and
selectEnd. They have a quirk I didn't expect: they only tell you that an
input device was clicked. They don't tell you what item in the environment was
clicked. Event handlers are added to the XRSession object and should be added
as soon as its available.
xrDevice.requestSession(sessionOptions)
.then(xrSession => {
// Create a WebGL layer and initialize the render loop.
xrSession.addEventListener('selectstart', onSelectStart);
xrSession.addEventListener('selectend', onSelectEnd);
xrSession.addEventListener('select', onSelect);
});
This code is based on an Input Selection example , in case you want more context.
To figure out what was clicked you use a pose. (Are you surprised? I didn't think so.) The details of that are specific to your app or whatever framework you're using, and hence beyond the scope of this article. Cottontail's approach is in the Input Selection example.
function onSelect(ev) {
let inputPose = ev.frame.getInputPose(ev.inputSource, xrFrameOfRef);
if (!inputPose) {
return;
}
if (inputPose.pointerMatrix) {
// Figure out what was clicked and respond.
}
}
As I said earlier, augmented reality is expected in Chrome 69 (Canary some time in June 2018). Nevertheless, I encourage you try what we've got so far. We need feedback to make it better. Follow it's progress by watching ChromeStatus.com for WebXR Hit Test. You can also follow WebXR Anchors which will improve pose tracking.
Chrome 67 on desktop has a new feature called Site Isolation enabled by default. This article explains what Site Isolation is all about, why it’s necessary, and why web developers should be aware of it.
The internet is for watching cat videos and managing cryptocurrency wallets, amongst other things —
but you wouldn’t want fluffycats.example to have access to your precious cryptocoins! Luckily,
websites typically cannot access each other’s data inside the browser thanks to the Same-Origin
Policy. Still, malicious websites may try to bypass this policy to attack other websites, and
occasionally, security bugs are found in the browser code that enforces the Same-Origin Policy. The
Chrome team aims to fix such bugs as quickly as possible.
Site Isolation is a security feature in Chrome that offers an additional line of defense to make such attacks less likely to succeed. It ensures that pages from different websites are always put into different processes, each running in a sandbox that limits what the process is allowed to do. It also blocks the process from receiving certain types of sensitive data from other sites. As a result, with Site Isolation it’s much more difficult for a malicious website to use speculative side-channel attacks like Spectre to steal data from other sites. As the Chrome team finishes additional enforcements, Site Isolation will also help even when an attacker’s page can break some of the rules in its own process.
Site Isolation effectively makes it harder for untrusted websites to access or steal information from your accounts on other websites. It offers additional protection against various types of security bugs, such as the recent Meltdown and Spectre side-channel attacks.
For more details on Site Isolation, see our article on the Google Security blog.
Even when all cross-site pages are put into separate processes, pages can still legitimately request
some cross-site subresources, such as images and JavaScript. A malicious web page could use an
<img> element to load a JSON file with sensitive data, like your bank balance:
<img src="https://your-bank.example/balance.json">
<!-- Note: the attacker refused to add an `alt` attribute, for extra evil points. -->
Without Site Isolation, the contents of the JSON file would make it to the memory of the renderer process, at which point the renderer notices that it’s not a valid image format and doesn’t render an image. But, the attacker could then exploit a vulnerability like Spectre to potentially read that chunk of memory.
Instead of using <img>, the attacker could also use <script> to commit the sensitive data to
memory:
<script src="https://your-bank.example/balance.json"></script>
Cross-Origin Read Blocking, or CORB, is a new security feature that prevents the contents of
balance.json from ever entering the memory of the renderer process memory based on its MIME type.
Let’s break down how CORB works. A website can request two types of resources from a server:
A website is able to receive data resources from its own origin or from other origins with
permissive CORS headers such as
Access-Control-Allow-Origin: *. On the other hand, media resources can be included from any
origin, even without permissive CORS headers.
CORB prevents the renderer process from receiving a cross-origin data resource (i.e. HTML, XML, or JSON) if:
X-Content-Type-Options: nosniff headerIf the cross-origin data resource doesn’t have the X-Content-Type-Options: nosniff header set,
CORB attempts to sniff the response body to determine whether it’s HTML, XML, or JSON. This is
necessary because some web servers are misconfigured and serve images as text/html, for example.
Data resources that are blocked by the CORB policy are presented to the process as empty, although the request does still happen in the background. As a result, a malicious web page has a hard time pulling cross-site data into its process to steal.
For optimal security and to benefit from CORB, we recommend the following:
Content-Type header. (For example, HTML resources should be
served as text/html, JSON resources with
a JSON MIME type and XML resources with
an XML MIME type).X-Content-Type-Options: nosniff header. Without this header,
Chrome does do a quick content analysis to try to confirm that the type is correct, but since this
errs on the side of allowing responses through to avoid blocking things like JavaScript files,
you’re better off affirmatively doing the right thing yourself.For more details, refer to the CORB for web developers article or our in-depth CORB explainer.
For the most part, Site Isolation is a behind-the-scenes browser feature that is not directly exposed to web developers. There is no new web-exposed API to learn, for example. In general, web pages shouldn’t be able to tell the difference when running with or without Site Isolation.
However, there are some exceptions to this rule. Enabling Site Isolation comes with a few subtle side-effects that might affect your website. We maintain a list of known Site Isolation issues, and we elaborate on the most important ones below.
With Site Isolation, full-page layout is no longer guaranteed to be synchronous, since the frames of a page may now be spread across multiple processes. This might affect pages if they assume that a layout change immediately propagates to all frames on the page.
As an example, let’s consider a website named fluffykittens.example that communicates with a
social widget hosted on social-widget.example:
<!-- https://fluffykittens.example/ -->
<iframe src="https://social-widget.example/" width="123"></iframe>
<script>
const iframe = document.querySelector('iframe');
iframe.width = 456;
iframe.contentWindow.postMessage(
// The message to send:
'Meow!',
// The target origin:
'https://social-widget.example'
);
</script>
At first, the social widget’s <iframe>’s width is 123 pixels. But then, the FluffyKittens page
changes the width to 456 pixels (triggering layout) and sends a message to the social widget,
which has the following code:
<!-- https://social-widget.example/ -->
<script>
self.onmessage = () => {
console.log(document.documentElement.clientWidth);
};
</script>
Whenever the social widget receives a message through the postMessage API, it logs the width of
its root <html> element.
Which width value gets logged? Before Chrome enabled Site Isolation, the answer was 456. Accessing
document.documentElement.clientWidth forces layout, which used to be synchronous before Chrome
enabled Site Isolation. However, with Site Isolation enabled, the cross-origin social widget
re-layout now happens asynchronously in a separate process. As such, the answer can now also be
123, i.e. the old width value.
If a page changes the size of a cross-origin <iframe> and then sends a postMessage to it, with
Site Isolation the receiving frame may not yet know its new size when receiving the message. More
generally, this might break pages if they assume that a layout change immediately propagates to all
frames on the page.
In this particular example, a more robust solution would set the width in the parent frame, and
detect that change in the <iframe> by listening for a resize event.
Key Point: In general, avoid making implicit assumptions about browser layout behavior. Full-page
layout involving cross-origin <iframe>s was never explicitly specified to be synchronous, so it’s
best to not write code that relies on this.
When a frame navigates or closes, the old document as well as any subframe documents embedded in it
all run their unload handler. If the new navigation happens in the same renderer process (e.g. for
a same-origin navigation), the unload handlers of the old document and its subframes can run for
an arbitrarily long time before allowing the new navigation to commit.
addEventListener('unload', () => {
doSomethingThatMightTakeALongTime();
});
In this situation, the unload handlers in all frames are very reliable.
However, even without Site Isolation some main frame navigations are cross-process, which impacts
unload handler behavior. For example, if you navigate from old.example to new.example by typing
the URL in the address bar, the new.example navigation happens in a new process. The unload
handlers for old.example and its subframes run in the old.example process in the background,
after the new.example page is shown, and the old unload handlers are terminated if they don’t
finish within a certain timeout. Because the unload handlers may not finish before the timeout,
the unload behavior is less reliable.
Note: Currently, DevTools support for unload handlers is largely missing. For example, breakpoints
inside of unload handlers don’t work, any requests made during unload handlers don’t show up in the
Network pane, any console.log calls made during unload handlers may not show up, etc. Star
Chromium issue #851882 to
receive updates.
With Site Isolation, all cross-site navigations become cross-process, so that documents from
different sites don’t share a process with each other. As a result, the above situation applies in
more cases, and unload handlers in <iframe>s often have the background and timeout behaviors
described above.
Another difference resulting from Site Isolation is the new parallel ordering of unload handlers: without Site Isolation, unload handlers run in a strict top-down order across frames. But with Site Isolation, unload handlers run in parallel across different processes.
These are fundamental consequences of enabling Site Isolation. The Chrome team is working on improving the reliability of unload handlers for common use cases, where feasible. We’re also aware of bugs where subframe unload handlers aren’t yet able to utilize certain features and are working to resolve them.
An important case for unload handlers is to send end-of-session pings. This is commonly done as follows:
addEventListener('pagehide', () => {
const image = new Image();
img.src = '/end-of-session';
});
Note: We
recommend to
use the pagehide event over beforeunload or unload, for reasons unrelated to Site Isolation.
A better approach that is more robust in light of this change is to use navigator.sendBeacon
instead:
addEventListener('pagehide', () => {
navigator.sendBeacon('/end-of-session');
});
If you need more control over the request, you can use the Fetch API’s keepalive option:
addEventListener('pagehide', () => {
fetch('/end-of-session', { keepalive: true });
});
Site Isolation makes it harder for untrusted websites to access or steal information from your accounts on other websites by isolating each site into its own process. As part of that, CORB tries to keep sensitive data resources out of the renderer process. Our recommendations above ensure you get the most out of these new security features.
Thanks to Alex Moshchuk, Charlie Reis, Jason Miller, Nasko Oskov, Philip Walton, Shubhie Panicker, and Thomas Steiner for reading a draft version of this article and giving their feedback.
Developers using service workers and the Cache Storage API should be on the lookout for two small changes rolling out in Chrome 71. Both changes bring Chrome's implementation more in line with specifications and other browsers.
importScripts()
tells your main service worker script to pause its current execution, download additional code from
a given URL, and run it to completion in the current global
scope. Once that's done,
the main service worker script resumes execution. importScripts() comes in handy when
you want to break your main service worker script into smaller pieces for organizational reasons, or
pull in third-party code to add functionality to your service worker.
Browsers attempt to mitigate the possible performance gotchas of "download and run some synchronous
code" by automatically caching anything pulled in via importScripts(), meaning that after the
initial download, there's very little overhead involved in executing the imported code.
For that to work, though, the browser needs to know that there won't be any "surprise" code imported
into the service worker after the initial
installation.
As per the service worker specification,
calling importScripts() is supposed to only work during the synchronous execution of the top-level
service worker script, or if needed, asynchronously inside of the install handler.
Prior to Chrome 71, calling importScripts() asynchronously outside of the install handler would
work. Starting with Chrome 71, those calls
throw a runtime exception (unless the same URL was previously imported in an install handler),
matching the behavior in other browsers.
Instead of code like this:
// This only works in Chrome 70 and below.
self.addEventListener('fetch', event => {
importScripts('my-fetch-logic.js');
event.respondWith(self.customFetchLogic(event));
});
Your service worker code should look like:
// Move the importScripts() to the top-level scope.
// (Alternatively, import the same URL in the install handler.)
importScripts('my-fetch-logic.js');
self.addEventListener('fetch', event => {
event.respondWith(self.customFetchLogic(event));
});
Note: Some users of the Workbox library might be
implicitly relying on asynchronous calls to importScripts() without realizing it. Please see this
guidance to
make sure you don't run into issues in Chrome 71.
If you're using the Cache Storage API alongside of a service worker, there's another small change in
Chrome 71 to align with the relevant
specification. When the same URL is
passed in multiple times to a single call to
cache.addAll(), the
specification says that the promise returned by the call should reject.
Prior to Chrome 71, that was not detected, and the duplicate URLs would effectively be ignored.
This logging is a prelude to Chrome 72, where instead of just a logged warning, duplicate URLs will
lead to cache.addAll() rejecting. If you're calling cache.addAll() as part of a promise chain
passed to
InstallEvent.waitUntil(),
as is common practice, that rejection might cause your service worker to fail to install.
Here's how you might run into trouble:
const urlsToCache = [
'/index.html',
'/main.css',
'/app.js',
'/index.html' // Oops! This is listed twice and should be removed.
];
self.addEventListener('install', event => {
event.waitUntil(
caches.open('my-cache')
.then(cache => cache.addAll(urlsToCache))
);
});
This restriction only applies to the actual URLs being passed to cache.addAll(), and caching what
ends up being two equivalent responses that have different URLs—like '/' and '/index.html'—will
not trigger a rejection.
Service workers are widely implemented across all major "evergreen" browsers at this point. If you regularly test your progressive web app against a number of browsers, or if you have a significant number of users who don't use Chrome, then chances are you've already detected the inconsistency and updated your code. But on the off chance that you haven't noticed this behavior in other browsers, we wanted to call out the change before switching Chrome's behavior.
The web is an amazing platform, it reaches users all around the world - on essentially any device. It’s easy to use, and easy to share. There’s nothing to install. But most importantly, it’s an open-ecosystem that anyone can use or build on.
There are some apps that are not possible to build and deliver on the open web today. We call this, the app gap. The gap between what’s possible on the web and what’s possible on native. We want to close that gap. We believe web apps should be able to do anything native apps can.
Through our capabilities project, we want to make it possible for web apps to do anything native apps can, by exposing the capabilities of native platforms to the web platform, while maintaining user security, privacy, trust, and other core tenets of the web.
We developed this process to make it possible to design and develop new web platform capabilities that meet the needs of developers quickly, in the open, and most importantly, work within the existing standards process. It’s no different than how we develop every other web platform feature, but it puts an emphasis on developer feedback.
Developer feedback is critical to help us ensure we’re shipping the right features, but when it comes in late in the process, it can be hard to change course. That’s why we’re starting to ask for feedback earlier. When actionable technical and use-case feedback comes in early, it’s easier to course correct or even stop development, without having shipped poorly thought out or badly implemented features. Features being developed at WICG are not set in stone, and your input can make a big difference in how they evolve.
It’s worth noting that many ideas never make it past the explainer or origin trial stage. The goal of the process is to ship the right feature. That means we need to learn and iterate quickly. Not shipping a feature because it doesn’t solve the developer need is OK. To enable this learning, we have come to employ the following process (although there is frequently some re-ordering of later steps due to feedback):
The first step is to identify and understand the developer need. What is the developer trying to accomplish? Who would use it? How are they doing it today? And what problems or frustrations are fixed by this new capability. Typically, these come in as feature request from developers, frequently through bugs filed on bugs.chromium.org.
After identifying the need for a new capability, create an explainer, essentially a design doc that is meant to explain the problem, along with some sample code showing how the API might work. The explainer is a living design document that will go through heavy iteration as the new capability evolves.
Once the explainer has a reasonable level of clarity, it’s time to publicize it, to solicit feedback, and iterate on the design. This is an opportunity to verify the new capability meets the needs of developers and works in a way that they expect. This is also an opportunity to gather public support and verify that there really is a need for this capability.
Once the explainer is in a good state, the design work transitions into a formal specification, working with developers and other browser vendors to iterate and improve on the design.
Then, once the design starts to stabilize, we typically use an origin trial to experiment with the implementation. Origin trials allow you to try new features with real users, and give feedback on the implementation. This real world feedback helps shape and validate the design, ensuring we get it right, before it becomes a standard.
Finally, once the origin trial is complete, the spec has been finalized, and all of the other launch steps have been completed, it’s time to ship it to stable.
Some of these features may seem scary at first, especially in light of how they’re implemented on native. But the web is inherently safer than native, opening a web page shouldn’t be scary.
Nothing should ever be granted access by default, but instead rely on a permission model that puts the user in total control, and is easily revoke-able. It needs to be crystal clear when, and how these APIs are being used. We've outlined some of our thought process in Controlling Access to Powerful Web Platform Features.
As developers, we are often faced with decisions that will affect the entire architecture of our applications. One of the core decisions web developers must make is where to implement logic and rendering in their application. This can be a difficult, since there are a number of different ways to build a website.
Our understanding of this space is informed by our work in Chrome talking to large sites over the past few years. Broadly speaking, we would encourage developers to consider server rendering or static rendering over a full rehydration approach.
In order to better understand the architectures we’re choosing from when we make this decision, we need to have a solid understanding of each approach and consistent terminology to use when speaking about them. The differences between these approaches help illustrate the trade-offs of rendering on the web through the lens of performance.
Rendering
Performance
Server rendering generates the full HTML for a page on the server in response to navigation. This avoids additional round-trips for data fetching and templating on the client, since it’s handled before the browser gets a response.
Server rendering generally produces a fast First Paint (FP) and First Contentful Paint (FCP). Running page logic and rendering on the server makes it possible to avoid sending lots of JavaScript to the client, which helps achieve a fast Time to Interactive (TTI). This makes sense, since with server rendering you’re really just sending text and links to the user’s browser. This approach can work well for a large spectrum of device and network conditions, and opens up interesting browser optimizations like streaming document parsing.
With server rendering, users are unlikely to be left waiting for CPU-bound JavaScript to process before they can use your site. Even when third-party JS can’t be avoided, using server rendering to reduce your own first-party JS costs can give you more "budget" for the rest. However, there is one primary drawback to this approach: generating pages on the server takes time, which can often result in a slower Time to First Byte (TTFB).
Whether server rendering is enough for your application largely depends on what type of experience you are building. There is a longstanding debate over the correct applications of server rendering versus client-side rendering, but it’s important to remember that you can opt to use server rendering for some pages and not others. Some sites have adopted hybrid rendering techniques with success. Netflix server-renders its relatively static landing pages, while prefetching the JS for interaction-heavy pages, giving these heavier client-rendered pages a better chance of loading quickly.
Many modern frameworks, libraries and architectures make it possible to render the same application on both the client and the server. These techniques can be used for Server Rendering, however it’s important to note that architectures where rendering happens both on the server and on the client are their own class of solution with very different performance characteristics and tradeoffs. React users can use renderToString() or solutions built atop it like Next.js for server rendering. Vue users can look at Vue’s server rendering guide or Nuxt. Angular has Universal. Most popular solutions employ some form of hydration though, so be aware of the approach in use before selecting a tool.
Static rendering happens at build-time and offers a fast First Paint, First Contentful Paint and Time To Interactive - assuming the amount of client-side JS is limited. Unlike Server Rendering, it also manages to achieve a consistently fast Time To First Byte, since the HTML for a page doesn’t have to be generated on the fly. Generally, static rendering means producing a separate HTML file for each URL ahead of time. With HTML responses being generated in advance, static renders can be deployed to multiple CDNs to take advantage of edge-caching.
Solutions for static rendering come in all shapes and sizes. Tools like Gatsby are designed to make developers feel like their application is being rendered dynamically rather than generated as a build step. Others like Jekyll and Metalsmith embrace their static nature, providing a more template-driven approach.
One of the downsides to static rendering is that individual HTML files must be generated for every possible URL. This can be challenging or even infeasible when you can't predict what those URLs will be ahead of time, or for sites with a large number of unique pages.
React users may be familiar with Gatsby, Next.js static export or Navi - all of these make it convenient to author using components. However, it’s important to understand the difference between static rendering and prerendering: static rendered pages are interactive without the need to execute much client-side JS, whereas prerendering improves the First Paint or First Contentful Paint of a Single Page Application that must be booted on the client in order for pages to be truly interactive.
If you’re unsure whether a given solution is static rendering or prerendering, try this test: disable JavaScript and load the created web pages. For statically rendered pages, most of the functionality will still exist without JavaScript enabled. For prerendered pages, there may still be some basic functionality like links, but most of the page will be inert.
Another useful test is to slow your network down using Chrome DevTools, and observe how much JavaScript has been downloaded before a page becomes interactive. Prerendering generally requires more JavaScript to get interactive, and that JavaScript tends to be more complex than the Progressive Enhancement approach used by static rendering.
Server rendering is not a silver bullet - its dynamic nature can come with significant compute overhead costs. Many server rendering solutions don't flush early, can delay TTFB or double the data being sent (e.g. inlined state used by JS on the client). In React, renderToString() can be slow as it's synchronous and single-threaded. Getting server rendering "right" can involve finding or building a solution for component caching, managing memory consumption, applying memoization techniques, and many other concerns. You're generally processing/rebuilding the same application multiple times - once on the client and once in the server. Just because server rendering can make something show up sooner doesn't suddenly mean you have less work to do.
Server rendering produces HTML on-demand for each URL but can be slower than just serving static rendered content. If you can put in the additional leg-work, server rendering + HTML caching can massively reduce server render time. The upside to server rendering is the ability to pull more "live" data and respond to a more complete set of requests than is possible with static rendering. Pages requiring personalization are a concrete example of the type of request that would not work well with static rendering.
Server rendering can also present interesting decisions when building a PWA. Is it better to use full-page service worker caching, or just server-render individual pieces of content?
Client-side rendering (CSR) means rendering pages directly in the browser using JavaScript. All logic, data fetching, templating and routing are handled on the client rather than the server.
Client-side rendering can be difficult to get and keep fast for mobile. It can
approach the performance of pure server-rendering if doing minimal work, keeping
a tight JavaScript budget and delivering value in as few RTTs as possible.
Critical scripts and data can be delivered sooner using HTTP/2 Server Push or
<link rel=preload>, which gets the parser working for you sooner. Patterns
like PRPL are worth evaluating in order to ensure initial and subsequent
navigations feel instant.
The primary downside to Client-Side Rendering is that the amount of JavaScript required tends to grow as an application grows. This becomes especially difficult with the addition of new JavaScript libraries, polyfills and third-party code, which compete for processing power and must often be processed before a page’s content can be rendered. Experiences built with CSR that rely on large JavaScript bundles should consider aggressive code-splitting, and be sure to lazy-load JavaScript - "serve only what you need, when you need it". For experiences with little or no interactivity, server rendering can represent a more scalable solution to these issues.
For folks building a Single Page Application, identifying core parts of the User Interface shared by most pages means you can apply the Application Shell caching technique. Combined with service workers, this can dramatically improve perceived performance on repeat visits.
Often referred to as Universal Rendering or simply “SSR”, this approach attempts to smooth over the trade-offs between Client-Side Rendering and Server Rendering by doing both. Navigation requests like full page loads or reloads are handled by a server that renders the application to HTML, then the JavaScript and data used for rendering is embedded into the resulting document. When implemented carefully, this achieves a fast First Contentful Paint just like Server Rendering, then “picks up” by rendering again on the client using a technique called (re)hydration. This is a novel solution, but it can have some considerable performance drawbacks.
The primary downside of SSR with rehydration is that it can have a significant negative impact on Time To Interactive, even if it improves First Paint. SSR’d pages often look deceptively loaded and interactive, but can’t actually respond to input until the client-side JS is executed and event handlers have been attached. This can take seconds or even minutes on mobile.
Perhaps you’ve experienced this yourself - for a period of time after it looks like a page has loaded, clicking or tapping does nothing. This quickly becoming frustrating... “Why is nothing happening? Why can’t I scroll?”
Rehydration issues can often be worse than delayed interactivity due to JS. In order for the client-side JavaScript to be able to accurately “pick up” where the server left off without having to re-request all of the data the server used to render its HTML, current SSR solutions generally serialize the response from a UI’s data dependencies into the document as script tags. The resulting HTML document contains a high level of duplication:
As you can see, the server is returning a description of the application’s UI in response to a navigation request, but it’s also returning the source data used to compose that UI, and a complete copy of the UI’s implementation which then boots up on the client. Only after bundle.js has finished loading and executing does this UI become interactive.
Performance metrics collected from real websites using SSR rehydration indicate its use should be heavily discouraged. Ultimately, the reason comes down to User Experience: it's extremely easy to end up leaving users in an “uncanny valley”.
There’s hope for SSR with rehydration, though. In the short term, only using SSR for highly cacheable content can reduce the TTFB delay, producing similar results to prerendering. Rehydrating incrementally, progressively, or partially may be the key to making this technique more viable in the future.
Server rendering has had a number of developments over the last few years.
Streaming server rendering allows you to send HTML in chunks that the browser can progressively render as it's received. This can provide a fast First Paint and First Contentful Paint as markup arrives to users faster. In React, streams being asynchronous in renderToNodeStream() - compared to synchronous renderToString - means backpressure is handled well.
Progressive rehydration is also worth keeping an eye on, and something React has been exploring. With this approach, individual pieces of a server-rendered application are “booted up” over time, rather than the current common approach of initializing the entire application at once. This can help reduce the amount of JavaScript required to make pages interactive, since client-side upgrading of low priority parts of the page can be deferred to prevent blocking the main thread. It can also help avoid one of the most common SSR Rehydration pitfalls, where a server-rendered DOM tree gets destroyed and then immediately rebuilt - most often because the initial synchronous client-side render required data that wasn’t quite ready, perhaps awaiting Promise resolution.
Partial rehydration has proven difficult to implement. This approach is an extension of the idea of progressive rehydration, where the individual pieces (components / views / trees) to be progressively rehydrated are analyzed and those with little interactivity or no reactivity are identified. For each of these mostly-static parts, the corresponding JavaScript code is then transformed into inert references and decorative functionality, reducing their client-side footprint to near-zero. The partial hydration approach comes with its own issues and compromises. It poses some interesting challenges for caching, and client-side navigation means we can't assume server-rendered HTML for inert parts of the application will be available without a full page load.
If service workers are an option for you, “trisomorphic” rendering may also be of interest. It's a technique where you can use streaming server rendering for initial/non-JS navigations, and then have your service worker take on rendering of HTML for navigations after it has been installed. This can keep cached components and templates up to date and enables SPA-style navigations for rendering new views in the same session. This approach works best when you can share the same templating and routing code between the server, client page, and service worker.
Teams often factor in the impact of SEO when choosing a strategy for rendering on the web. Server-rendering is often chosen for delivering a "complete looking" experience crawlers can interpret with ease. Crawlers may understand JavaScript, but there are often limitations worth being aware of in how they render. Client-side rendering can work but often not without additional testing and leg-work. More recently dynamic rendering has also become an option worth considering if your architecture is heavily driven by client-side JavaScript.
When in doubt, the Mobile Friendly Test tool is invaluable for testing that your chosen approach does what you're hoping for. It shows a visual preview of how any page appears to Google's crawler, the serialized HTML content found (after JavaScript executed), and any errors encountered during rendering.
When deciding on an approach to rendering, measure and understand what your bottlenecks are. Consider whether static rendering or server rendering can get you 90% of the way there. It's perfectly okay to mostly ship HTML with minimal JS to get an experience interactive. Here’s a handy infographic showing the server-client spectrum:
Thanks to everyone for their reviews and inspiration:
Jeffrey Posnick, Houssein Djirdeh, Shubhie Panicker, Chris Harrelson, and Sebastian Markbåge
Note: We're always updating and improving
<model-viewer>. Check out the <model-viewer> homepage
to explore what it can do.
Adding 3D models to a website can be tricky. 3D models ideally will be shown in a viewer that can work responsively on all browsers - from smartphones, to desktop, to new head-mounted displays. The viewer should support progressive enhancement for performance, rendering quality and use cases on all devices ranging from older, lower-powered smartphones to newer devices that support augmented reality. It should stay up to date with current technologies. It should be performant and accessible. However, building such a viewer requires specialty 3D programming skills, and can be a challenge for web developers that want to host their own models instead of using a third-party hosting service.
To help with that, we're introducing the <model-viewer> web component which
lets you declaratively add a 3D model to a web page, while hosting the model on
your own site. The web component supports responsive design and use cases like
augmented reality on some devices, and we're adding features for accessibility,
rendering quality, and interactivity. The goal of the component is making it
easy to add 3D models to your website without being on top of the latest changes
in the underlying technology and platforms.
A web component is a custom HTML element built from standard web platform
features. A web component behaves for all intents and purposes like a standard
element. It has a unique tag, it can have properties and methods, and it can
fire and respond to events. In short, you don't need to know anything special to
use it. In this article, I will show you some things that are particular to
<model-viewer>.
More specifically, what can it do now? I'll show you its current capabilities.
You'll get a great experience today, and <model-viewer> will get better
over time as we add new features and improve rendering quality. The examples
I've provided are just to give you a sense of what it does. If you want to try
them there are installation and usage instructions in its GitHub
repo.
Embedding a 3D model is as simple as the markup below. By using gltf files, we've ensured that this component will work on any major browser.
<model-viewer src="assets/Astronaut.gltf" alt="A 3D model of an astronaut">
To see
With the auto-rotate and controls attributes I can provide motion and user
control. The examples show a complete list of attributes.
<model-viewer src="assets/Astronaut.gltf" **controls auto-rotate**>
Some 3D models can be very large, so you might want to hold off loading them until the user has requested the model. For this, the component has a built-in means of delaying loading until the user wants it.
<model-viewer src="assets/Astronaut.gltf" controls auto-rotate
poster="assets/poster2.png">
To show your users that it's a 3D model, and not just an image, you can provide some preload animation by using script to switch between multiple posters.
<model-viewer id="toggle-poster" src="assets/Astronaut.gltf" controls
auto-rotate poster="assets/poster2.png"></model-viewer>
<script>
const posters = ['poster.png', 'poster2.png'];
let i = 0;
setInterval(() =>
$('#toggle-poster').setAttribute('poster', `assets/${posters[i++ %
2]}`), 2000);
</script>
The component handles some types of responsive design, scaling for both mobile and desktop. It can also manage multiple instances on a page and uses Intersection Observer to conserve battery power and GPU cycles when a model isn't visible.
Install <model-viewer> and give it a try
We want <model-viewer> to be useful to you, and we want your input on its
future. That's not to say we don't have ideas, which we have on
our project roadmap.
So give it a try and let us know what you think by
filing an issue in GitHub.
What if I told you, there's more than one viewport.
BRRRRAAAAAAAMMMMMMMMMM
And the viewport you're using right now, is actually a viewport within a viewport.
BRRRRAAAAAAAMMMMMMMMMM
And sometimes, the data the DOM gives you, refers to one of those viewport and not the other.
BRRRRAAAAM… wait what?
It's true, take a look:
The video above shows a web page being scrolled and pinch-zoomed, along with a mini-map on the right showing the position of viewports within the page.
Things are pretty straight forward during regular scrolling. The green area
represents the layout viewport, which position: fixed items stick to.
Things get weird when pinch-zooming is introduced. The red box represents the
visual viewport, which is the part of the page we can actually see. This
viewport can move around while position: fixed elements remain where they
were, attached to the layout viewport. If we pan at a boundary of the layout
viewport, it drags the layout viewport along with it.
Unfortunately web APIs are inconsistent in terms of which viewport they refer to, and they're also inconsistent across browsers.
For instance, element.getBoundingClientRect().y returns the offset within the
layout viewport. That's cool, but we often want the position within the page,
so we write:
element.getBoundingClientRect().y + window.scrollY
However, many browsers use the visual viewport for window.scrollY, meaning
the above code breaks when the user pinch-zooms.
Chrome 61 changes window.scrollY to refer to the layout viewport instead,
meaning the above code works even when pinch-zoomed. In fact, browsers are
slowly changing all positional properties to refer to the layout viewport.
With the exception of one new property…
A new API exposes the visual viewport as window.visualViewport. It's a draft
spec, with cross-browser
approval, and it's
landing in Chrome 61.
console.log(window.visualViewport.width);
Here's what window.visualViewport gives us:
visualViewport properties |
|
|---|---|
offsetLeft
|
Distance between the left edge of the visual viewport, and the layout viewport, in CSS pixels. |
offsetTop
|
Distance between the top edge of the visual viewport, and the layout viewport, in CSS pixels. |
pageLeft
|
Distance between the left edge of the visual viewport, and the left boundary of the document, in CSS pixels. |
pageTop
|
Distance between the top edge of the visual viewport, and the top boundary of the document, in CSS pixels. |
width
|
Width of the visual viewport in CSS pixels. |
height
|
Height of the visual viewport in CSS pixels. |
scale
|
The scale applied by pinch-zooming. If content is twice the size due to
zooming, this would return 2. This is not affected by
devicePixelRatio.
|
There are also a couple of events:
window.visualViewport.addEventListener('resize', listener);
visualViewport events |
|
|---|---|
resize
|
Fired when width, height, or
scale changes.
|
scroll
|
Fired when offsetLeft or offsetTop changes.
|
The video at the start of this article was created using visualViewport,
check it out in Chrome 61+. It uses
visualViewport to make the mini-map stick to the top-right of the visual
viewport, and applies an inverse scale so it always appears the same size,
despite pinch-zooming.
It feels like an obvious thing to state, but it caught me out when I first
played with visualViewport.
If the layout viewport resizes but the visual viewport doesn't, you don't get a
resize event. However, it's unusual for the layout viewport to resize without
the visual viewport also changing width/height.
The real gotcha is scrolling. If scrolling occurs, but the visual viewport
remains static relative to the layout viewport, you don't get a scroll event
on visualViewport, and this is really common. During regular document
scrolling, the visual viewport stays locked to the top-left of the layout
viewport, so scroll does not fire on visualViewport.
If you're wanting to hear about all changes to the visual viewport, including
pageTop and pageLeft, you'll have to listen to the window's scroll event
too:
visualViewport.addEventListener('scroll', update);
visualViewport.addEventListener('resize', update);
window.addEventListener('scroll', update);
Similar to listening to scroll & resize on the window, you're likely to call
some kind of "update" function as a result. However, it's common for many of
these events to happen at the same time. If the user resizes the window, it'll
trigger resize, but quite often scroll too. To improve performance, avoid
handling the change multiple times:
// Add listeners
visualViewport.addEventListener('scroll', update);
visualViewport.addEventListener('resize', update);
addEventListener('scroll', update);
let pendingUpdate = false;
function update() {
// If we're already going to handle an update, return
if (pendingUpdate) return;
pendingUpdate = true;
// Use requestAnimationFrame so the update happens before next render
requestAnimationFrame(() => {
pendingUpdate = false;
// Handle update here
});
}
I've filed a spec issue for
this, as I think there may be a
better way, such as a single update event.
Due to a Chrome bug, this does not work:
Buggy – uses an event handler
visualViewport.onscroll = () => console.log('scroll!');
Instead:
Works – uses an event listener
visualViewport.addEventListener('scroll', () => console.log('scroll'));
I think (well, I hope) this is another Chrome bug.
offsetLeft and offsetTop are rounded, which is pretty inaccurate once the
user has zoomed in. You can see the issues with this during the
demo – if the user zooms in and pans
slowly, the mini-map snaps between unzoomed
pixels.
Like other resize and scroll events, these no not fire every frame,
especially on mobile. You can see this during the
demo – once you pinch zoom, the mini-map
has trouble staying locked to the viewport.
In the demo I used visualViewport to
counteract the user's pinch-zoom. It makes sense for this particular demo, but
you should think carefully before doing anything that overrides the user's
desire to zoom in.
visualViewport can be used to improve accessibility. For instance, if the user
is zooming in, you may choose to hide decorative position: fixed items, to get
them out of the user's way. But again, be careful you're not hiding something
the user is trying to get a closer look at.
You could consider posting to an analytics service when the user zooms in. This could help you identify pages that users are having difficulty with at the default zoom level.
visualViewport.addEventListener('resize', () => {
if (visualViewport.scale > 1) {
// Post data to analytics service
}
});
And that's it! visualViewport is a nice little API which solves compatibility
issues along the way.
The current WebVR origin trial is ending on November 14, 2017, shortly after the stable release of Chrome 62. We have begun a new trial with the WebVR 1.1 API in Chrome 62 that will continue through Chrome 64.
The new trial includes some API behavior updates that are consistent with the direction of the forthcoming WebVR 2.0 spec:
allow="vr" to the iframe tag, or use a
Feature-Policy header (spec discussion, bug).getFrameData() and submitFrame() to VRDisplay.requestAnimationFrame()
(spec discussion, bug).window.requestAnimationFrame() does not fire if the page is not visible, meaning it will not
fire on Android while WebVR is presenting (spec discussion, bug).vrdisplayactivate event is now considered a user gesture,
and may be used to request presentation and begin media playback, without relying on the click
event.
Code that was previously relying on click event handlers for input should be converted to check
for gamepad button presses. (Example implementation)Your current WebVR Origin Trial tokens will not be recognized by Chrome 62. To participate in this new trial please use the sign up form.
Blurring is a great way to redirect a user's focus. Making some visual elements appear blurred while keeping other elements in focus naturally directs the user's focus. Users ignore the blurred content and instead focus on the content they can read. One example would be a list of icons that display details about the individual items when hovered over. During that time, the remaining choices could be blurred to redirect the user to the newly displayed information.
Animating a blur is not really an option as it is very slow. Instead, pre-compute a series of increasingly blurred versions and cross-fade between them. My colleague Yi Gu wrote a library to take care of everything for you! Take a look at our demo.
However, this technique can be quite jarring when applied without any transitional period. Animating a blur — transitioning from unblurred to blurred — seems like a reasonable choice, but if you've ever tried doing this on the web, you probably found that the animations are anything but smooth, as this demo shows if you don't have a powerful machine. Can we do better?
Note: Always test your web apps on mobile devices. Desktop machines tend to have deceptively powerful GPUs.
As of now, we can't make animating a blur work efficiently. We can, however,
find a work-around that looks good enough, but is, technically speaking, not
an animated blur. To get started, let's first understand why the animated blur
is slow. To blur elements on the web, there are two techniques: The CSS filter
property and SVG filters. Thanks to increased support and ease of use, CSS
filters are typically used. Unfortunately, if you are required to support Internet
Explorer, you have no choice but to use SVG filters as IE 10 and 11 support
those but not CSS filters. The good news is that our workaround for animating a
blur works with both techniques. So let's try to find the bottleneck by looking
at DevTools.
If you enable "Paint Flashing" in DevTools, you won’t see any flashes at all. It looks like no repaints are happening. And that's technically correct as a "repaint" refers to the CPU having to repaint the texture of a promoted element. Whenever an element is both promoted and blurred, the blur is applied by the GPU using a shader.
Both SVG filters and CSS filters use convolution filters) to apply a blur. Convolution filters are fairly expensive as for every output pixel a number of input pixels have to be considered. The bigger the image or the bigger the blur radius, the more costly the effect is.
And that's where the problem lies, we are running a rather expensive GPU operation every frame, blowing our frame budget of 16ms and therefore ending up well below 60fps.
So what can we do to make this run smoothly? We can use sleight of hand! Instead
of animating the actual blur value (the radius of the blur), we pre-compute a
couple of blurred copies where the blur value increases exponentially, then
cross-fade between them using opacity.
The cross-fade is a series of overlapping opacity fade-ins and fade-outs. If we have four blur stages for example, we fade out the first stage while fading in the second stage at the same time. Once the second stage reaches 100% opacity and the first one has reached 0%, we fade out the second stage while fading in the third stage. Once that is done, we finally fade out the third stage and fade in the fourth and final version. In this scenario, each stage would take ¼ of the total desired duration. Visually, this looks very similar to a real, animated blur.
In our experiments, increasing the blur radius exponentially per stage yielded
the best visual results. Example: If we have four blur stages, we'd apply
filter: blur(2^n) to each stage, i.e. stage 0: 1px, stage 1: 2px, stage 2: 4px
and stage 3: 8px. If we force each of these blurred copies onto their own layer
(called "promoting") using will-change: transform, changing opacity on these
elements should be super-duper fast. In theory, this would allow us to
front-load the expensive work of blurring. Turns out, the logic is flawed. If
you run this
demo,
you'll see that framerate is still below 60fps, and the blurring is actually
worse than before.
A quick look into DevTools reveals that the GPU is still extremely busy and stretches each frame to ~90ms. But why? We are not changing the blur value anymore, only the opacity, so what's happening? The problem lies, once again, in the nature of the blur effect: As explained before, if the element is both promoted and blurred, the effect is applied by the GPU. So even though we are not animating the blur value anymore, the texture itself is still unblurred and needs to be re-blurred every frame by the GPU. The reason for the frame rate being even worse than before stems from the fact that compared to the naïve implementation, the GPU actually has more work than before, as most of the time two textures are visible that need to be blurred independently.
What we came up with is not pretty, but it makes the animation blazingly fast. We go back to not promoting the to-be-blurred element, but instead promote a parent wrapper. If an element is both blurred and promoted, the effect is applied by the GPU. This is what made our demo slow. If the element is blurred but not promoted, the blur is rasterized to the nearest parent texture instead. In our case that's the promoted parent wrapper element. The blurred image is now the texture of the parent element and can be re-used for all future frames. This only works because we know that the blurred elements are not animated and caching them is actually beneficial. Here's a demo that implements this technique. I wonder what the Moto G4 thinks of this approach? Spoiler alert: it thinks it's great:
Now we've got lots of headroom on the GPU and a silky-smooth 60fps. We did it!
In our demo, we duplicated a DOM structure multiple times to have copies of the content to blur at different strengths. You might be wondering how this would work in a production environment as that might have some unintended side-effects with the author's CSS styles or even their JavaScript. You are right. Enter Shadow DOM!
While most people think about Shadow
DOM as a way to attach "internal"
elements to their Custom
Elements, it is also an
isolation and performance primitive! JavaScript and CSS cannot pierce Shadow DOM
boundaries which allows us to duplicate content without interfering with the
developer's styles or application logic. We already have a <div> element for
each copy to rasterize onto and now use these <div>s as shadow hosts. We
create a ShadowRoot using attachShadow({mode: 'closed'}) and attach a copy of
the content to the ShadowRoot instead of the <div> itself. We have to make
sure to also copy all stylesheets into the ShadowRoot to guarantee that our
copies are styled the same way as the original.
Note: In most cases — especially when writing custom elements — we advise
against using closed ShadowRoots. Find out more in Eric's
article.
Some browsers do not support Shadow DOM v1, and for those, we fall back to just duplicating the content and hoping for the best that nothing breaks. We could use the Shadow DOM polyfill with ShadyCSS, but we did not implement this in our library.
And there you go. After our journey down Chrome's rendering pipeline, we figured out how we can animate blurs efficiently across browsers!
This kind of effect is not to be used lightly. Due to the fact that we copy DOM
elements and force them onto their own layer, we can push the limits of lower-end
devices. Copying all stylesheets into each ShadowRoot is a potential
performance risk as well, so you should decide whether you would rather adjust
your logic and styles to not be affected by copies in the LightDOM or use our
ShadowDOM technique. But sometimes our technique might be a worthwhile
investment. Take a look at the code in our GitHub
repository
as well as the
demo and
hit me up on Twitter if you have any questions!
Lighthouse 2.5 is now released! Highlights include:
See the release notes for the full list of new features, changes, and bug fixes coming to Lighthouse in version 2.5.
chrome-launcher is now a standalone Node module, making it easier to launch Google Chrome from your own Node applications.
Category: Best Practices
The Does not use images with appropriate aspect ratios audit alerts you when an image's rendered aspect ratio is significantly different than the image's actual dimensions. The aspect ratio is the ratio between width and height. If the ratio is significantly different when rendered, then the image probably looks distorted.
Category: Best Practices
The Includes front-end JavaScript libraries with known security vulnerabilities audit warns you about how many vulnerabilities a library has, as well as the highest severity level among those vulnerabilities.
Category: Performance
The Unused JavaScript audit breaks down how much JavaScript a page loads but does not use during startup.
Note: This audit is only available when running Lighthouse from Node or the command line in full-config mode.
Category: Performance
The Keep server response times low (TTFB) audit measures how long it takes the client to receive the first byte of the main document. If Time To First Byte (TTFB) is long, then the request is taking a long time traveling through the network, or the server is slow.
Category: Best Practices
The Browser errors were logged to the console audit alerts you to any errors that are logged to the console as the page loads.
Check out the new Throttling Guide to learn how to conduct high-quality, packet-level throttling. This guide is intended for advanced audiences.
If you're working with Media Source Extensions (MSE), one thing you will
eventually need to deal with is an over-full buffer. When this occurs, you'll
get what's called a QuotaExceededError. In this article, I'll cover some of
the ways to deal with it.
Basically, QuotaExceededError is what you get if you try to add too much data
to your SourceBuffer object. (Adding more SourceBuffer objects to a parent
MediaSource element can also throw this error. That's outside the scope of
this article.) If SourceBuffer has too much data in it, calling
SourceBuffer.appendBuffer() will trigger the following message in the Chrome
console window.
There are a few things to note about this. First, notice that the name
QuotaExceededError appears nowhere in the message. To see that, set a
breakpoint at a location where you can catch the error and examine it in your
watch or scope window. I've shown this below.
Second, there's no definitive way to find out how much data the SourceBuffer
can handle.
At the time of writing, Safari does not throw a QuotaExceededError in many of
its builds. Instead it removes frames using a two step algorithm, stopping if
there is enough room to handle the appendBuffer(). First, it frees frames from
between 0 and 30 seconds before the current time in 30 second chunks. Next, it
frees frames in 30 second chunks from duration backwards to as close as 30
seconds after currentTime. You can read more about this in a Webkit
changeset from 2014.
Fortunately, along with Chrome, Edge and Firefox do throw this error. If you're using another browser, you'll need to do your own testing. Though probably not what you'd build for a real-life media player, François Beaufort's source buffer limit test at least lets you observe the behavior.
The exact number varies from browser to browser. Since you can't query for the amount currently appended data, you'll have to keep track of how much you're appending yourself. As for what to watch, here's the best data I can gather at the time of writing. For Chrome these numbers are upper limits meaning they can be smaller when the system encounters memory pressure.
| Chrome | Chromecast* | Firefox | Safari | Edge | |
|---|---|---|---|---|---|
| Video | 150MB | 30MB | 100MB | 290MB | Unknown |
| Audio | 12MB | 2MB | 15MB | 14MB | Unknown |
Since the amount of supported data varies so widely and you can't find the
amount of data in a SourceBuffer, you must get it indirectly by handling the
QuotaExceededError. Now let's look at a few ways to do that.
There are several approaches to dealing with QuotaExceededError. In reality a
combination of one or more approaches is best. Your approach should be to base
the work on how much you're fetching and attempting to append beyond
HTMLMediaElement.currentTime and adjusting that size based on the
QuotaExceededError. Also using a manifest of some kind such as an mpd
file
(MPEG-DASH) or an m3u8
file
(HLS) can help you keep track of the data you're appending to the buffer.
Now, let's look at several approaches to dealing with the
QuotaExceededError.
Though they can be used in combination, I'll cover them one at a time.
Really this one should be called, "Remove least-likely-to-be-used-soon data, and then retry append of data likely-to-be-used-soon." That was too long of a title. You'll just need to remember what I really mean.
Removing recent data is not as simple as calling SourceBuffer.remove(). To
remove data from the SourceBuffer, it's updating flag must be false. If it is
not, call SourceBuffer.abort() before removing any data.
There are a few things to keep in mind when calling SourceBuffer.remove().
currentTime because doing so could cause playback stall. Such information
may need to be parsed out of the bytestream by the web app if it is not
available in the manifest. A media manifest or app knowledge of keyframe
intervals in the media can help guide your app's choice of removal ranges to
prevent removing the currently playing media. Whatever you remove, don't
remove the currently playing group of pictures or even the first few beyond
that. Generally, don't remove beyond the current time unless you're certain
that the media is not needed any longer. If you remove close to the
playhead you may cause a stall.SourceBuffer.abort().
In fact, they throw errors that will halt playback. Fortunately there are
open bug trackers here
and here. In the
meantime, you'll have to work around this somehow. Shaka Player does
it
by stubbing out an empty abort() function on those versions of Safari.I've shown the procedure below. This may not work in every case, but it has the advantage that the size of the smaller chunks can be adjusted to suit your needs. It also doesn't require going back to the network which might incur additional data costs for some users.
const pieces = new Uint8Array([data]);
(function appendFragments(pieces) {
if (sourceBuffer.updating) {
return;
}
pieces.forEach(piece => {
try {
sourceBuffer.appendBuffer(piece);
}
catch e {
if (e.name !== 'QuotaExceededError') {
throw e;
}
// Reduction schedule: 80%, 60%, 40%, 20%, 16%, 12%, 8%, 4%, fail.
const reduction = pieces[0].byteLength * 0.8;
if (reduction / data.byteLength < 0.04) {
throw new Error('MediaSource threw QuotaExceededError too many times');
}
const newPieces = [
pieces[0].slice(0, reduction),
pieces[0].slice(reduction, pieces[0].byteLength)
];
pieces.splice(0, 1, newPieces[0], newPieces[1]);
appendBuffer(pieces);
}
});
})(pieces);
This is similar to removing recent data and re-appending. In fact, the two may be done together, though the example below only shows lowering the resolution.
There are a few things to keep in mind when using this technique:
Starting in Chrome 63, you cannot use the shadow-piercing selectors ::shadow
and /deep/ to style content inside of a shadow root.
/deep/ combinator will act as a descendant selector. x-foo /deep/ div
will work like x-foo div.::shadow pseudo-element will not match any elements.Note: If your site uses Polymer, the team has put together a thorough guide
walking through steps to migrate off of ::shadow and /deep/.
The ::shadow and /deep/ were deprecated in Chrome version 45. This was
decided by all of the participants at the April 2015 Web Components
meetup.
The primary concern with shadow-piercing selectors is that they violate encapsulation and create situations where a component can no longer change its internal implementation.
Note: For the moment, ::shadow and /deep/ will continue to work with
JavaScript APIs like querySelector() and querySelectorAll(). Ongoing support
for these APIs is being discussed on
GitHub.
The CSS Shadow Parts spec is being advanced as an alternative to shadow piercing selectors. Shadow Parts will allow a component author to expose named elements in a way that preserves encapsulation and still allows page authors the ability to style multiple properties at once.
The ::shadow and /deep/ selectors only affect legacy Shadow DOM v0
components. If you're using Shadow DOM v1, you should not need to change
anything on your site.
You can use Chrome Canary
to verify your site does not break with these new changes. If you notice issues,
try and remove any usage of ::shadow and /deep/. If it's too difficult to
remove usage of these selectors, consider switching from native shadow DOM over
to the shady DOM polyfill. You should only need to make this change if your site
relies on native shadow DOM v0.
CSS Paint API (also known as “CSS Custom Paint” or “Houdini’s paint worklet”) is enabled by default starting in Chrome 65. What is it? What can you do with it? And how does it work? Well, read on, will ya’…
CSS Paint API allows you to programmatically generate an image whenever a CSS
property expects an image. Properties like background-image or border-image
are usually used with url() to load an image file or with CSS built-in
functions like linear-gradient(). Instead of using those, you can now use
paint(myPainter) to reference a paint worklet.
To define a paint worklet called myPainter, we need to load a CSS paint
worklet file using CSS.paintWorklet.addModule('my-paint-worklet.js'). In that
file, we can use the registerPaint function to register a paint worklet class:
class MyPainter {
paint(ctx, geometry, properties) {
// ...
}
}
registerPaint('myPainter', MyPainter);
Inside the paint() callback, we can use ctx the same way we would a
CanvasRenderingContext2D as we know it from <canvas>. If you know how to
draw in a <canvas>, you can draw in a paint worklet! geometry tells us the
width and the height of the canvas that is at our disposal. properties I will
explain later in this article.
Note: A paint worklet’s context is not 100% the same as a <canvas> context. As
of now, text rendering methods are missing and for security reasons you cannot
read back pixels from the canvas.
As an introductory example, let’s write a checkerboard paint worklet and use it
as a background image of a <textarea>. (I am using a textarea because it’s
resizable by default.):
<!-- index.html -->
<!doctype html>
<style>
textarea {
background-image: paint(checkerboard);
}
</style>
<textarea></textarea>
<script>
CSS.paintWorklet.addModule('checkerboard.js');
</script>
// checkerboard.js
class CheckerboardPainter {
paint(ctx, geom, properties) {
// Use `ctx` as if it was a normal canvas
const colors = ['red', 'green', 'blue'];
const size = 32;
for(let y = 0; y < geom.height/size; y++) {
for(let x = 0; x < geom.width/size; x++) {
const color = colors[(x + y) % colors.length];
ctx.beginPath();
ctx.fillStyle = color;
ctx.rect(x * size, y * size, size, size);
ctx.fill();
}
}
}
}
// Register our class under a specific name
registerPaint('checkerboard', CheckerboardPainter);
If you’ve used <canvas> in the past, this code should look familiar. See
the live
demo
here.
Note: As with almost all new APIs, CSS Paint API is only available over HTTPS
(or localhost).
The difference from using a common background image here is that the pattern will be re-drawn on demand, whenever the user resizes the textarea. This means the background image is always exactly as big as it needs to be, including the compensation for high-density displays.
That’s pretty cool, but it’s also quite static. Would we want to write a new worklet every time we wanted the same pattern but with differently sized squares? The answer is no!
Luckily, the paint worklet can access other CSS properties, which is where the
additional parameter properties comes into play. By giving the class a static
inputProperties attribute, you can subscribe to changes to any CSS property,
including custom properties. The values will be given to you through the
properties parameter.
<!-- index.html -->
<!doctype html>
<style>
textarea {
/* The paint worklet subscribes to changes of these custom properties. */
--checkerboard-spacing: 10;
--checkerboard-size: 32;
background-image: paint(checkerboard);
}
</style>
<textarea></textarea>
<script>
CSS.paintWorklet.addModule('checkerboard.js');
</script>
// checkerboard.js
class CheckerboardPainter {
// inputProperties returns a list of CSS properties that this paint function gets access to
static get inputProperties() { return ['--checkerboard-spacing', '--checkerboard-size']; }
paint(ctx, geom, properties) {
// Paint worklet uses CSS Typed OM to model the input values.
// As of now, they are mostly wrappers around strings,
// but will be augmented to hold more accessible data over time.
const size = parseInt(properties.get('--checkerboard-size').toString());
const spacing = parseInt(properties.get('--checkerboard-spacing').toString());
const colors = ['red', 'green', 'blue'];
for(let y = 0; y < geom.height/size; y++) {
for(let x = 0; x < geom.width/size; x++) {
ctx.fillStyle = colors[(x + y) % colors.length];
ctx.beginPath();
ctx.rect(x*(size + spacing), y*(size + spacing), size, size);
ctx.fill();
}
}
}
}
registerPaint('checkerboard', CheckerboardPainter);
Now we can use the same code for all different kind of checkerboards. But even better, we can now go into DevTools and fiddle with the values until we find the right look.
Note: It would be great to parameterize the colors, too, wouldn’t it? The spec
allows for the paint() function to take a list of arguments. This feature is
not implemented in Chrome yet, as it heavily relies on Houdini’s Properties and
Values API, which still needs some work before it can ship.
At the time of writing, only Chrome has paint worklet implemented. While there are positive signals from all other browser vendors, there isn’t much progress. To keep up to date, check Is Houdini Ready Yet? regularly. In the meantime, be sure to use progressive enhancement to keep your code running even if there’s no support for paint worklet. To make sure things work as expected, you have to adjust your code in two places: The CSS and the JS.
Detecting support for paint worklet in JS can be done by checking the CSS object:
if ('paintWorklet' in CSS) {
CSS.paintWorklet.addModule('mystuff.js');
}
For the CSS side, you have two options. You can use @supports:
@supports (background: paint(id)) {
/* ... */
}
A more compact trick is to use the fact that CSS invalidates and subsequently ignores an entire property declaration if there is an unknown function in it. If you specify a property twice — first without paint worklet, and then with the paint worklet — you get progressive enhancement:
textarea {
background-image: linear-gradient(0, red, blue);
background-image: paint(myGradient, red, blue);
}
In browsers with support for paint worklet, the second declaration of
background-image will overwrite the first one. In browsers without support
for paint worklet, the second declaration is invalid and will be discarded,
leaving the first declaration in effect.
For many uses, it's also possible to use the CSS Paint Polyfill, which adds CSS Custom Paint and Paint Worklets support to modern browsers.
There are many use cases for paint worklets, some of them more obvious than
others. One of the more obvious ones is using paint worklet to reduce the size
of your DOM. Oftentimes, elements are added purely to create embellishments
using CSS. For example, in Material Design Lite the button
with the ripple effect contains 2 additional <span> elements to implement the
ripple itself. If you have a lot of buttons, this can add up to quite a number
of DOM elements and can lead to degraded performance on mobile. If you
implement the ripple effect using paint
worklet
instead, you end up with 0 additional elements and just one paint worklet.
Additionally, you have something that is much easier to customize and
parameterize.
Another upside of using paint worklet is that — in most scenarios — a solution using paint worklet is small in terms of bytes. Of course, there is a trade-off: your paint code will run whenever the canvas’s size or any of the parameters change. So if your code is complex and takes long it might introduce jank. Chrome is working on moving paint worklets off the main thread so that even long-running paint worklets don’t affect the responsiveness of the main thread.
To me, the most exciting prospect is that paint worklet allows an efficient polyfilling of CSS features that a browser doesn’t have yet. One example would be to polyfill conic gradients until they land in Chrome natively. Another example: in a CSS meeting, it was decided that you can now have multiple border colors. While this meeting was still going on, my colleague Ian Kilpatrick wrote a polyfill for this new CSS behavior using paint worklet.
Most people start to think about background images and border images when they
learn about paint worklet. One less intuitive use case for paint worklet is
mask-image to make DOM elements have arbitrary shapes. For example a
diamond:
mask-image takes an image that is the size of the element. Areas where the
mask image is transparent, the element is transparent. Areas where the mask
image is opaque, the element opaque.
Paint worklet has been in Chrome Canary for a while. With Chrome 65, it is enabled by default. Go ahead and try out the new possibilities that paint worklet opens up and show us what you built! For more inspiration, take a look at Vincent De Oliveira’s collection.
Note: Breakpoints are currently not supported in CSS Paint API, but will be enabled in a later release of Chrome.