Building better in the world of build tools!

00:18 - All right, so one question: when you are starting a new project or you are given brand new repo and then, “Hey, Surma, please start this project,” where do you start, what tools do you choose? Tell me about your setup.

00:33 - This has evolved so much over time. I take a long time to decide, and I usually start out without a bundler.

00:41 - My preferred starting place is I just want to write latest JS, latest CSS, and the rest of it is out of my way until I’m going to solve it.

00:50 - It really depends on what the project is. I’ve been building a lot of static sites lately and 11ty is a recently– a relatively new tool for static site generation, and I’ve been converting some Jekyll sites into that, and it’s just felt so nice and not super robust, which is what you need for a static site, but then, if I am building a more dynamic project, I’ll either go with Next. js or Gatsby if I want to react with a static site because I like how it parses out to just HTML, CSS and JavaScript.

01:22 - but there’s a thousand tools and it really just depends.

01:26 - That’s the answer: it depends. So I was a Webpack user, and I use Webpack because, at the time, it was the only one that supported code splitting because I knew I just needed to add this tag in the head, but like the HTML plugin went, “No this is mine now, you may not touch. ” And then other things like in the earlier days of Service Worker, I just wanted a list of the output files.

01:49 - I want– let me know what the hash is going to be for these files.

01:53 - But reading the Webpack plugin Docs, I was getting so frustrated, I couldn’t figure it out.

01:59 - I used to use Gulp or Grunt a lot because at least I understood what’s happening, and then with Webpack, I really didn’t understand what’s happening.

02:09 - Now, over time, I have fallen in love with Rollup.

02:14 - So I have my own build system that I’ve been managing.

02:17 - it’s probably my sixth build system I’ve made.

02:20 - If you go look at my GitHub history, I have a Grunt one, I have a Make one, I have Gulp ones. [laughs] I’ve been following all these build tools for a long time.

02:28 - Here’s how I feel about Web projects: its complexity is inevitable.

02:32 - There’s no way to get around it, and you’re either injecting complexity from the beginning and making it worse or you’re waiting until your complexity confronts you.

02:41 - [Mariko] Yes, even if you start simple, complexity is inevitable.

02:47 - The idea of making a website, in principle, is straightforward.

02:50 - You make HTML document, add style to it, and add some functionality to it too.

02:56 - But, in practice, web development gets a lot more complex.

03:01 - Your application code may depend on outside libraries or different modules.

03:06 - You might be importing web fonts. Or you might be pre-rendering a portion of a page as a static site so that it can get delivered faster to the users.

03:14 - And chances are you’re probably using build tools to manage all of these complexities.

03:21 - But because tools expect a certain setup, sometimes seemingly simple tasks, like add a line to HTML, gets harder to accomplish.

03:31 - There are many challenges like this in web development.

03:33 - So let’s look at how we manage JavaScript. In the past, we’ve written everything in independent file or different script tags and carefully combine them or add it to HTML by ourselves.

03:47 - The way we write JavaScript, we used to have this massive file, the humans needed to know which one goes first and which one goes after.

03:56 - Yeah, and even you’re making me remember where I started with bundling, which was a PHP script that concatenated all my JavaScript files together, which now feels wrong, but at the time felt very powerful. [laughs] [Mariko] But now we have modules, which means dependency of each module are specified in the code, which means a build tool can analyze a file and create bundle for us.

04:21 - Even better, some tools, like Webpack, analyze which part of the code is actually being used and extract it to make smaller bundles.

04:32 - When we started writing JavaScript this way, I feel Webpack came in the scene as a tool of choice, with things that have a lot of bells and whistles and do things like tree shaking and scope hoisting.

04:45 - So could you explain how Webpack handles this module JavaScript field.

04:51 - Yeah, so Webpack supports a huge number of module formats.

04:56 - Some of the common ones, obviously, ES modules, CommonJS, everybody knows about, but it actually supports parsing and understanding the structure of SystemJS modules and AMD modules and even Wasm imports.

05:13 - So it takes all that information, and in its in-memory graph representation, it attaches that and can use it to…

05:22 - if you only imported one thing from a module, it can essentially delete the export from that module that you didn’t use, and that way that code path won’t end up in your bundle.

05:34 - And when you take that and you fragment it out, maybe that export was using another import from another module, and now that’s unused– you can see how flowing that information through the graph.

05:45 - eventually, you could end up removing a fair bit of code.

05:49 - So Webpack doesn’t actually convert modules to an internal source format.

05:54 - It is more focused on understanding them and their structure as they exist on disk.

06:01 - [Mariko] Our JavaScript file does not have to be a single file, so tools like Rollup will split them up into smaller chunks.

06:08 - Can you explain why Rollup is really good at it? Yeah so Webpack has its own loader and so does Parcel, whereas Rollup by default is ECMAScript modules.

06:23 - That’s where it lives; it lives in that world.

06:25 - So the output it generates is way simpler than the other tools.

06:29 - In terms of code splitting, Rollup’s implementation is very pure, I would say.

06:36 - It will create the smallest number of chunks that it can, but it will create a small chunk, like maybe your chunk just containing one function, if that’s the only bit shared by two entry points.

06:48 - And that’s something that Webpack and Parcel don’t do.

06:52 - They will duplicate that module in both of their bundles, whereas Rollup will always just create a separate chunk.

07:00 - It’s very pure, it’ll never duplicate code.

07:03 - [Mariko] And different script runs in different thread.

07:06 - Ideally, common dependencies are exported as one chunk.

07:10 - But subtools doesn’t understand it, so it creates duplicates.

07:15 - What was interesting to me was that the Parcel supports worker and main thread splitting out of the box.

07:22 - Is there any backstory for that? There is, and I think I can take a tiny bit of credit for Parcel supporting that because it was, I think, February 2018, you, me, and some others were working on Squoosh, and Squoosh made heavy use of WebAssembly, and put the WebAssembly in the Web Worker, and then used Comlink to use those Web Workers.

07:45 - And we built Squoosh using Webpack, and we figured out over time that actually the way Webpack built that project, it put a copy of Comlink into the worker and into the main thread.

07:57 - So the user ended up loading that code twice.

08:00 - Now, Comlink isn’t that big, so it’s not that big of a deal, but with bigger dependencies, that could actually become a significant problem.

08:08 - So I filed a fairly long bug on Webpack, with a graph and everything, explaining why that should change, and it hasn’t been fixed yet, but it’s been discussed a lot, and I think Sean Larkin told me that Webpack 5 will finally be able to address that problem.

08:24 - But shortly after I opened that bug on Webpack, Devon Govett, who is the maintainer of Parcel, actually opened an issue himself on Parcel for himself, saying, “I think we can do this. I think Parcel can fix this problem. ” I don’t know how long it took him.

08:40 - I guess a couple months later, the bug was closed, and suddenly Parcel now supported this code splitting across worker boundaries, and that was just really cool to see.

08:50 - [Mariko] Beyond making JavaScript bundles, build tools help us manage assets too- sometimes separately and sometimes through JavaScript.

08:58 - So I think that as CSS has evolved, the tooling around it has grown as well.

09:03 - And I think the first really big instance of CSS tooling came about with Sass, LESS, and Stylus, and all of those preprocessor tools.

09:12 - That was when Ruby was really big, and they were written in Ruby, and it was clunky and slow because then you had to wait for your CSS to process before it was spit out from Sass to CSS, for example.

09:24 - And then Node came around, and that became a lot faster, and Sass was rewritten, and people were still using a lot of the benefits from that language, the Sass language, and then PostCSS replaced a lot of that need because it allowed for you to do some of these same things, but instead of pre-processing and waiting for the developer to see their CSS file be exported, you could then run through the CSS file after you had already written it and then apply transformations and changes with PostCSS.

09:54 - That also enabled you to have pluggable very small bits of code that was way less robust and just large in terms of developer fingerprint in your dev files, in your architecture, by enabling these small plugins, like an autoprefixer plugin, or something for size, where if you wanted to use a size keyword, you could use that.

10:19 - You could even write your own. So that was really cool.

10:22 - And I think that that has continued to the state people use PostCSS all the time.

10:25 - And now you’re seeing a lot of additional CSS tools that allow for things like tree shaking and some of these optimizations that are beyond just minification.

10:34 - And that is really coincided with the switch to framework-based JavaScript as being so prevalent in the way we architect our projects now.

10:46 - Naturally, I think CSS in JS happens, or maybe it’s JS in your CSS, but, inevitably, what will happen is you’ll find a moment where you needed something that was just so richly dynamic that some declarative static styles might not work for.

11:02 - So I’ve been putting CSS in my JS for a long time, but if we’re talking about the new CSS in JS libraries, take your object notation or let you do extending and abstracting in different ways at the client side, those are great too.

11:16 - All options for writing styles still have foot guns.

11:20 - So you just need to be careful, and you’ll learn your tool the more you use it.

11:24 - [Mariko] Okay, so what’s the quirks when dealing with assets? I mean Webpack has a super long history with various techniques for doing this.

11:36 - At its simplest, there are tools for– essentially, you can take a loader which is a transform you apply to a module, and you can apply that to something that isn’t JavaScript to turn it into a string in a JavaScript file.

11:51 - So, historically, the way that assets have worked in Webpack has tended to center around turning them into a JavaScript thing.

11:59 - Even for assets where the asset itself might have dependencies, either on JavaScript or on other assets, it will turn that into a JavaScript module with a string and turn the dependencies, the CSS import statements, into JavaScript require calls.

12:18 - So it will inject them into the graph by converting them to the equivalent JavaScript code.

12:23 - [Mariko] Now, this is changing in Webpack 5, but for other tools, like Parcel, assets has been a center of it from the beginning.

12:32 - One thing that sets Parcel apart for many others is that they actually don’t use JavaScript as the entry point, but HTML.

12:39 - So what is considered an asset in many other build tools is their main entry points in Parcel end.

12:45 - And that makes sense because on the web, that is the thing we go to.

12:48 - We go to HTML pages, and from there on, we reference assets.

12:52 - So whether you reference an image from JavaScript or from within HTML, Parcel will understand that, and that’s actually really cool.

13:00 - And it does this to many layers. So if you reference a CSS file from HTML, and that CSS file references some images, all of these things are tracked by Parcel, and they will be hashed, and they will get a version number and whatnot, so it actually builds an entire asset graph.

13:18 - [Mariko] What Surma just explained is called asset hash cascading.

13:22 - If one file in a graph is updated, then the file hash changes.

13:27 - And because of that, hash of the file that used the updated asset should also change.

13:33 - This is important so that we can control cache for better performance.

13:38 - Let’s see how Rollup handles it. Yeah. So right now, hashing is Rollup’s weakness, and it’s something they know about, and it’s something they are going to be working on fixing very soon.

13:51 - So when you generate the hash for a file, that little bit of the file names go at the end of the letters and numbers, you need to do that as late in the process as possible.

14:04 - You want it to just be based on the contents of the file, not the directory it’s in, not your config settings, anything like that.

14:10 - If you’ve got JavaScript which imports JavaScript which imports JavaScript, it does the right thing with the hashing.

14:16 - You change a leaf one, and it changes all of the other ones in the chain, because all of the URLs have updated.

14:22 - Whereas, with assets, you use a magic string in Rollup, and Rollup will change that to be the assets URL, but it does that after hashing.

14:31 - So you update your asset, it changes hash– it updates the JavaScript file fine, but it doesn’t update the hash of that JavaScript.

14:39 - It’s a weakness, but they are fixing it. [Mariko] These differences and gotchas in our build tools are a constant source of frustration.

14:47 - As Una said, the best tool for the job really depends on your project.

14:53 - So, we wanted to make it easier for you to navigate this landscape.

14:57 - Tooling. report is a new website that gives developers like you an overview of various features supported by different build tools.

15:06 - We built this website so you can evaluate and choose the right tools for your next project.

15:11 - Or maybe you are in the middle of migrating from an infrastructure and hitting a roadblock.

15:17 - Tooling. report should help you answer your questions.

15:21 - We’ve essentially written a test suite for different tools based on common web development practices, so you can read why those tests are relevant, and see how each tool handles it.

15:33 - And when you are ready to implement it yourself, you can look at our test code to see how you might integrate certain features into your build setup.

15:43 - We also welcome your contributions, so if you think certain features should get tested, please raise an issue on the repo.

15:51 - Thank you, and please reach out if you have any questions.

15:54 - ♪ [music] ♪.