rossta.netRoss Kaffenbergerhttps://rossta.net/2020-09-08T00:00:00+00:00These Rails apps are overpacking their JavaScript bundles/blog/rails-apps-overpacking-with-webpacker.html2020-09-08T00:00:00+00:002020-09-08T00:00:00+00:00Ross Kaffenberger<p>You might think dividing your JavaScript into multiple bundles will help improve page load performance. When done incorrectly with Webpacker, it's possible to make things worse.</p>
<p>This mistake appears relatively common. As I'll share in this post, I...</p><p>You might think dividing your JavaScript into multiple bundles will help improve page load performance. When done incorrectly with Webpacker, it's possible to make things worse.</p>
<p>This mistake appears relatively common. As I'll share in this post, I've discovered several of my favorite Rails applications are making browsers download and parse <em>more</em> JavaScript than necessary even while attempting to send less.</p>
<p>I believe Rails developers may think the mechanics of packaging JavaScript for the browsers work similarly in Webpacker as it does with the Rails asset pipeline. This assumption is fraught with peril!</p>
<p><img src="/assets/images/blog/overpacking-case-studies/crash.gif" alt="Crash bang warehouse accident" loading="lazy" /></p>
<p>As we'll see, Webpacker is a very different beast than the Rails asset pipeline. We need a different mental model to understand how it works. We should also follow a few basic guidelines to deliver JavaScript correctly and avoid falling victim to "bundle bloat."</p>
<p>First, let's take a little safari and see what we can do to help a few companies correct their Webpacker usage out in the wild.</p>
<blockquote>
<p><strong>Help me help them</strong>: <em>If you know anyone who works at any of the following companies, please share this post or ask them to reach out to me at ross at rossta dot net.</em></p>
</blockquote>
<h2 id="case-study-podia" class="title title-h2">
<a name="case-study-podia" class="anchor" href="#case-study-podia"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Case study: Podia
</h2>
<p>Podia is a fantastic service that provides content creators with a platform to sell digital content, including e-books, online courses, and webinars.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/podia-homepage.png" alt="Podia homepage" loading="lazy" /></p>
<p>We can tell Podia uses Webpacker to bundle assets because it renders a Webpacker manifest at <code>https://app.podia.com/packs/manifest.json</code>:</p>
<pre><code class="json">{
"admin/ui.css": "/packs/css/admin/ui-59291053.css",
"admin/ui.js": "/packs/js/admin/ui-931ad01f76a9c8b4c1af.js",
"admin/ui.js.map": "/packs/js/admin/ui-931ad01f76a9c8b4c1af.js.map",
"application.js": "/packs/js/application-42b89cd8ec22763d95ae.js",
"application.js.map": "/packs/js/application-42b89cd8ec22763d95ae.js.map",
//...
</code></pre>
<p>The manifest contains URLs to many Webpacker "packs," also described as <a href="https://webpack.js.org/concepts/#entry" target="_blank" rel="noopener noreferrer"><em>entry points</em> in the webpack docs</a>.</p>
<p>When I visit the public-facing storefront, my browser downloads the following "packs" from the Podia CDN:</p>
<pre><code class="sh">/packs/js/storefront/index-1d9e9c36f5f9ab7056d1.js
/packs/js/storefront/current_time_ago-0c5c682d173647ef3199.js
/packs/js/storefront/messaging-60ddf6567eb7b74a1083.js
</code></pre>
<p>By splitting JavaScript up across multiple files on this page, I believe Podia intends to deliver only required JavaScript to the client's browser. For example, there's no need to send JavaScript for the CMS UI to the public-facing storefront page.</p>
<p>As we said earlier, the intent here is good.</p>
<h2 id="the-problem" class="title title-h2">
<a name="the-problem" class="anchor" href="#the-problem"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
The problem
</h2>
<p>On closer look, though, something doesn't seem quite right. The payloads for these individual packs are, in fact, rather large.</p>
<p>Take the "storefront/current_time_ago.js" bundle. It transfers as 73KB gzipped and comes out to 396KB of parsed JavaScript.</p>
<p><em>Does Podia's "storefront/current_time_ago" functionality need to be nearly 400KB?</em></p>
<p>If so, I'd be shocked. I imagine this pack's primary responsibility is similar to the tiny jQuery plugin <a href="https://timeago.org/" target="_blank" rel="noopener noreferrer">timeago</a>, which claims a 2KB size. As a comparison, a bundled version of <code>react-dom</code> module parses at around ~150KB.</p>
<p>Something's not right here.</p>
<h3 id="exploring-source-maps" class="title title-h3">
<a name="exploring-source-maps" class="anchor" href="#exploring-source-maps"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Exploring source maps
</h3>
<p>I don't work at Podia, so I can't use my favorite tool, <a href="/blog/webpacker-output-analysis-with-webpack-bundle-analyzer.html">the webpack-bundle-analyzer</a>, to peek inside the bundled JavaScript; this requires access to source code.</p>
<p>But, there's another trick we can use. We can find out what's happening inside these bundles from Podia's source maps.</p>
<p><em>It's like magic.</em></p>
<p><img src="/assets/images/blog/overpacking-case-studies/magic.gif" alt="Corny magician gif" loading="lazy" /></p>
<p>Source maps are included in production by default with Webpacker. You can find the URLs to source maps in the Webpacker manifest file, as shown above.</p>
<blockquote>
<p>If you're curious about why source maps in production are enabled by default in Webpacker, you may be interested in <a href="https://github.com/rails/webpacker/issues/769" target="_blank" rel="noopener noreferrer">this GitHub issue thread</a>.</p>
</blockquote>
<p>Another place to find the URL to a source map is in the corresponding source file's last line:</p>
<pre><code class="javascript">//# sourceMappingURL=application-42b89cd8ec22763d95ae.js.map
</code></pre>
<p>We can analyze Podia's publicly available source maps using <a href="https://github.com/danvk/source-map-explorer" target="_blank" rel="noopener noreferrer">source-map-explorer</a>. It can provide a visualization of all the modules bundled on this page. Here's an example:</p>
<p><img src="/assets/images/blog/overpacking-case-studies/podia-source-map-explorer-bare.png" alt="Treemap image example of webpack JavaScript bundles" loading="lazy" /></p>
<p><strong>Podia storefront editor</strong></p>
<p>Here's a screenshot of the source-map-explorer treemap for the three Webpacker packs rendered on the storefront editor page, with my annotations for emphasis:</p>
<p><img src="/assets/images/blog/overpacking-case-studies/podia-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Podia's storefront editor" loading="lazy" /></p>
<p>You can see the three JavaScript bundles in purple, blue, and orange, and with each, you can see included modules such as <code>actioncable</code>, <code>stimulus</code>, <code>moment.js</code>, <code>core-js</code>, and <code>cableready</code>.</p>
<p>Here's the problem: <strong>some modules appear twice on the same page!</strong></p>
<p><img src="/assets/images/blog/overpacking-case-studies/mrbean.gif" alt="Mr. Bean is shocked" loading="lazy" /></p>
<p>Two bundles include both moment.js and all the 100+ moment-locale modules. That means the browser has to download and parse moment.js (52KB) and moment-locales (326KB) twice on the same page!</p>
<p>Same for actioncable, cableready, stimulus, and core-js.</p>
<p>In an attempt to deliver less JavaScript to the browser with page-specific bundles, they've ended up with even bigger payloads. Podia is <a href="/blog/overpacking-a-common-webpacker-mistake.html">"overpacking"</a>, and it's resulting in the <strong>redundant module problem</strong>.</p>
<h2 id="more-case-studies" class="title title-h2">
<a name="more-case-studies" class="anchor" href="#more-case-studies"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
More case studies
</h2>
<p>It's not just Podia. I've recently discovered several other Rails applications with the same problem.</p>
<p><strong>Funny or Die</strong>
<img src="/assets/images/blog/overpacking-case-studies/page-funnyordie.png" alt="Funny or Die home page" loading="lazy" /></p>
<p>I'm always up for a laugh, but you know what's <em>not</em> funny? Duplicate <code>jquery</code> on the <a href="https://www.funnyordie.com" target="_blank" rel="noopener noreferrer">Funny or Die</a> home page.</p>
<p>That's an extra 80KB and, I would presume, a potential source of bugs for jquery plugins that assume only one instance of <code>$</code> in the page scope.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/funnyordie-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Funny or Die's home page" loading="lazy" /></p>
<p><strong>Dribbble</strong>
<img src="/assets/images/blog/overpacking-case-studies/page-dribbble.png" alt="Dribbble profile page" loading="lazy" /></p>
<p>I'm whistling the <a href="https://dribbble.com/dribbble" target="_blank" rel="noopener noreferrer">Dribbble profile page</a> for multiple infractions, including duplicate instances of <code>vue</code> and <code>axios</code>. They could reduce their total payload size by up to 150KB.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/dribbble-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Dribbble's profile page" loading="lazy" /></p>
<p><strong>Teachable</strong>
<img src="/assets/images/blog/overpacking-case-studies/page-teachable.png" alt="Teachable course page" loading="lazy" /></p>
<p>The <a href="https://www.jessicasprague.com/p/digibeginnerbundle" target="_blank" rel="noopener noreferrer">course page on Teachable</a> must love <code>jquery</code> and <code>lodash</code>. They're both bundled twice across the three Webpacker packs rendered on this page.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/teachable-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Teachable's student course page" loading="lazy" /></p>
<p><strong>Drizly</strong>
<img src="/assets/images/blog/overpacking-case-studies/page-drizly.png" alt="Drizly search page" loading="lazy" /></p>
<p>It's raining JavaScript at <a href="https://drizly.com/beer/c2" target="_blank" rel="noopener noreferrer">Drizly</a>! The product search page renders three packs, each of which includes instances <code>material-ui</code>, <code>react</code>, and <code>lodash</code>, among others. If Drizly were to introduce React hooks, I am relatively sure <a href="https://reactjs.org/warnings/invalid-hook-call-warning.html#duplicate-react" target="_blank" rel="noopener noreferrer">multiple instances of React will cause issues</a> if they haven't already.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/drizly-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Drizly's profile page" loading="lazy" /></p>
<p><strong>Strava's activity feed</strong>
<img src="/assets/images/blog/overpacking-case-studies/page-strava.png" alt="Strava activity feed" loading="lazy" /></p>
<p>As an endurance athlete in my spare time, I use Strava almost daily, where the activity feed forces me to render four instances of <code>react</code>! Strava could reduce their activity feed payloads by a whopping 500KB by getting rid of their duplicated modules.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/strava-source-map-explorer-annotated.png" alt="Treemap image of duplicated JavaScript bundles loaded on Strava's activity feed" loading="lazy" /></p>
<h3 id="analyzing-javascript-usage" class="title title-h3">
<a name="analyzing-javascript-usage" class="anchor" href="#analyzing-javascript-usage"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Analyzing JavaScript usage
</h3>
<p>Another tool I recommend is <a href="https://github.com/aholachek/bundle-wizard" target="_blank" rel="noopener noreferrer">bundle-wizard</a>, which can be used to find unused JavaScript modules on page load.</p>
<pre><code class="sh">$ npx -p puppeteer -p bundle-wizard bundle-wizard --interact
</code></pre>
<p>This tool turns the source-map-explorer into a heatmap representing code coverage across the bundled modules from high (green) to low (red).</p>
<p>Here are the source maps from the Strava activity feed visualized again with bundle-wizard coverage heatmap:
<img src="/assets/images/blog/overpacking-case-studies/bundle-wizard-strava-optimized.png" alt="Bundle wizard treemap with heatmap overlay" loading="lazy" /></p>
<p>See all that red? Those extra React modules are unused on page load.</p>
<h3 id="measuring-end-user-performance" class="title title-h3">
<a name="measuring-end-user-performance" class="anchor" href="#measuring-end-user-performance"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Measuring end user performance
</h3>
<p>We can also see whether Google's <a href="https://developers.google.com/web/tools/lighthouse" target="_blank" rel="noopener noreferrer">Lighthouse</a> performance auditing tool would back these findings.</p>
<blockquote>
<p>Lighthouse is an open-source, automated tool that can perform web page audits for performance, accessibility, among other quality indicators. <a href="https://developers.google.com/web/tools/lighthouse" target="_blank" rel="noopener noreferrer">You can generate a Lighthouse report for almost any page you can access on the web through Chrome DevTools or a Firefox extension</a>.</p>
</blockquote>
<p>I generated <a href="https://googlechrome.github.io/lighthouse/viewer/?gist=2a3785da1cfa4922190f3924d02edf39" target="_blank" rel="noopener noreferrer">this Lighthouse report for my Strava dashboard</a>:</p>
<p><a href="https://googlechrome.github.io/lighthouse/viewer/?gist=2a3785da1cfa4922190f3924d02edf39" target="_blank" rel="noopener noreferrer"><img src="/assets/images/blog/overpacking-case-studies/lighthouse-strava-optimized.png" alt="Lighthouse report screenshot for strava.com/dashboard" loading="lazy" /></a></p>
<p>The page scores 23/100 based on <a href="https://web.dev/performance-scoring/" target="_blank" rel="noopener noreferrer">Lighthouse's performance metric scoring rubric</a>, and, by far, the <em>biggest opportunity</em> for improving page load performance is to remove unused JavaScript.</p>
<p>This much is clear: JavaScript bloat is hampering the performance of these Rails apps.</p>
<h2 id="why-the-redundant-modules-" class="title title-h2">
<a name="why-the-redundant-modules-" class="anchor" href="#why-the-redundant-modules-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why the redundant modules?
</h2>
<p>It should be clear by now some Rails apps using Webpacker are bundling some modules unnecessarily across multiple bundles on a single page. As a result:</p>
<ol>
<li>JavaScript payloads are larger—not smaller—causing increased download and parse times for the end user</li>
<li>Logic may assume "singleton" behavior or touch global concerns leading to confounding bugs</li>
</ol>
<p>So why is this happening?</p>
<p>These Rails applications aren't intentionally bundling all this extra JavaScript. The fact that they are splitting up their bundles indicates they are attempting to be selective about what JavaScript is delivered on a given page.</p>
<p><em>Wait, so we can't split code into multiple bundles without duplicating modules in Webpacker?</em></p>
<p>Let's be clear that the practice of code-splitting isn't wrong; <a href="https://web.dev/reduce-javascript-payloads-with-code-splitting/" target="_blank" rel="noopener noreferrer">it's a recommended best practice</a> for improving page load performance.</p>
<p>The problem with these examples is in the execution; it's not happening <em>the way webpack expects</em>.</p>
<p>Consider <a href="https://cookpad.com" target="_blank" rel="noopener noreferrer">Cookpad.com</a>. It's a Rails app that renders numerous Webpacker bundles on its home page, yet no modules are duplicated:</p>
<p><img src="/assets/images/blog/overpacking-case-studies/cookpad-source-map-explorer-annotated-optimized.png" alt="Treemap of Cookpad source mapsl" loading="lazy" /></p>
<p>When it comes to Webpacker, the Cookpad recipe is top-notch.</p>
<h2 id="a-new-mental-model" class="title title-h2">
<a name="a-new-mental-model" class="anchor" href="#a-new-mental-model"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
A new mental model
</h2>
<p>The redundant module problem highlights that although the Rails asset pipeline and webpack solve the same general problem, they do so in <em>fundamentally different ways</em>.</p>
<blockquote>
<p>Webpack is a module bundler.</p>
<p>The asset pipeline is a file concatenator.</p>
</blockquote>
<p>The asset pipeline builds a list of what the developer explicitly requires. Think of it as a stack. "What you see is what you get."</p>
<p><img src="/assets/images/blog/overpacking-case-studies/analog-sprockets-stack.png" alt="St."ck" loading="lazy" /></p>
<p>Webpack, on the other hand, recursively parses the import statements in all the dependencies within a single pack, such as <code>app/javascript/packs/application.js</code>, like a directed graph.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/analog-webpack-graph.png" alt="Directed graph" loading="lazy" /></p>
<p>Webpack will include all imported modules in the output, ensuring that no import is included in the same bundle twice.</p>
<p><em>If that's true, why are there multiple instances modules in Podia's output, for example?</em></p>
<p>The reason: <strong>each pack is a separate dependency graph.</strong></p>
<p>Consider this illustration of an imaginary project with multiple packs. One pack imports <code>moment</code> explicitly, and the other pack imports a made-up <code>timeago</code> plugin that depends on <code>moment</code>.</p>
<p><img src="/assets/images/blog/overpacking-case-studies/imports-webpacker.png" alt="Image of Webpacker bundling assets with multiple packs" loading="lazy" /></p>
<p>See that the <code>moment</code> package is imported in both packs. There is an explicit import in the first pack, and an implicit import via <code>timeago</code> in the other.</p>
<p>So splitting your code into multiple packs can lead to this problem <em>if</em> you don't configure webpack properly.</p>
<p>What we want is a way to split code up into smaller pieces without all the overhead and potential bugs. It turns out, webpack was initially created to address precisely this: code-splitting.</p>
<p>It's just done differently than you expect.</p>
<h2 id="the-webpacker-packing-checklist" class="title title-h2">
<a name="the-webpacker-packing-checklist" class="anchor" href="#the-webpacker-packing-checklist"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
The Webpacker Packing Checklist
</h2>
<p>Now that we know what the problem is and how to diagnose it, what can we do about it?</p>
<p>The key to addressing this kind of Webpacker code bloat is to keep <em>all dependencies in the same dependency graph</em>.</p>
<p>Below, I summarize the steps I would take to help these companies, which you can apply in your applications. These steps are iterative; you need not complete all these actions to begin seeing benefits.</p>
<h3 id="step-1-start-with-one-entry-point-per-page" class="title title-h3">
<a name="step-1-start-with-one-entry-point-per-page" class="anchor" href="#step-1-start-with-one-entry-point-per-page"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Step 1: Start with one entry point per page
</h3>
<p>Webpack recommends one entry point per page. <a href="https://webpack.js.org/concepts/entry-points" target="_blank" rel="noopener noreferrer">From the webpack docs</a>:</p>
<blockquote>
<p>As a rule of thumb: Use precisely one entry point for each HTML document.</p>
</blockquote>
<p>That's how webpack assumes your application will work out-of-the-box. Practically speaking, that means there would be only one usage of <code>javascript_pack_tag</code> per page:</p>
<pre><code class="html"><%= javascript_pack_tag "application" %>
</code></pre>
<p>For the companies described in this post, that would mean consolidating those separate packs into one on a page. Rendering multiple entry points on a single page <em>correctly</em> requires additional configuration. We'll get to that, but "one pack per page" is how I recommend starting.</p>
<p>Does this mean you have to put <strong>all</strong> your JavaScript in one pack? No, but:</p>
<h3 id="step-2-keep-the-number-of-packs-small" class="title title-h3">
<a name="step-2-keep-the-number-of-packs-small" class="anchor" href="#step-2-keep-the-number-of-packs-small"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Step 2: Keep the number of packs small
</h3>
<p>Don't split your JavaScript into a ton of little packs/entry points unless you understand the tradeoffs, and you're comfortable with webpack.</p>
<p>For smaller applications, just an "application.js" may be well worth the tradeoff of having an application that's easier to maintain over the added cost of learning how to best split up JS code with webpack with little performance gain.</p>
<p>Think of packs as the entry points to <em>distinct</em> experiences rather than page-specific bundles.</p>
<p>For Podia, this could be one pack for the public-facing storefront, one for the storefront editor, one for the customer dashboard. Maybe an employee admin area pack. That's it.</p>
<p><em>Render one pack per page?... Keep the number of packs small? ... these bundles could get huge!</em></p>
<p>Ok, now we've come to webpack's sweet spot:</p>
<h3 id="step-3-use-dynamic-imports" class="title title-h3">
<a name="step-3-use-dynamic-imports" class="anchor" href="#step-3-use-dynamic-imports"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Step 3: Use dynamic imports
</h3>
<p>Webpack has several automated features for code-splitting that will never be supported in the Rails asset pipeline. The primary example of this is <a href="https://webpack.js.org/guides/code-splitting/#dynamic-imports" target="_blank" rel="noopener noreferrer">dynamic imports</a>.</p>
<p>Dynamic imports allow you to define split points <em>in code</em> rather than by configuration or multiple entry points. Note the <code>import()</code> function syntax:</p>
<pre><code class="javascript">// Contrived examples
// Import page-specific chunks
if (currentPage === 'storefront') {
import('../src/pages/storefront')
}
// Import progressive enhancement chunks
if (document.querySelector('[data-timeago]').length) {
import('../src/initializer/current_time_ago')
}
// Import bigger on-demand chunks following user interaction
document.addEventListener('[data-open-trix-editor]', 'click', () => {
import('../src/components/trix_editor')
})
</code></pre>
<p>In the example above, the imported modules are not separate packs. They are modules included in the same dependency graph but compiled as <em>separate files</em>. Webpack will load dynamic imports asynchronously at runtime.</p>
<p><em>Dynamic imports allow you to divide your "packs" into smaller pieces while avoiding the redundant module problem.</em></p>
<p>Does this mean import every little module in little dynamic chunks? No. Measure, experiment. Consider the tradeoffs with handling asynchronous code loading. Timebox your efforts</p>
<h3 id="step-4-go-further-with-splitchunks-but-only-when-you-39-re-ready" class="title title-h3">
<a name="step-4-go-further-with-splitchunks-but-only-when-you-39-re-ready" class="anchor" href="#step-4-go-further-with-splitchunks-but-only-when-you-39-re-ready"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Step 4: Go further with splitChunks, but only when you're ready
</h3>
<p>For a more powerful combination, use page-specific dynamic imports combined with <a href="https://help.github.com/en/github/managing-files-in-a-repository/getting-permanent-links-to-files" target="_blank" rel="noopener noreferrer">the splitChunks configuration API</a> to split out bundles for vendor code that can be shared across packs. In other words, browsers wouldn't have to pay the cost of re-downloading bundles containing moment.js, lodash.js, etc., across multiple pages with a warm cache.</p>
<p>Beware, though; this technique is a bit more advanced. It requires the use of separate Rails helpers, <code>javascript_packs_with_chunks_tag</code> and <code>stylesheet_packs_with_chunks_tag</code>, which will output multiple bundles produced from a single pack and these helpers should only be used once during the page render. It may take some <a href="https://webpack.js.org/plugins/split-chunks-plugin/" target="_blank" rel="noopener noreferrer">reading up on the webpack docs</a> and some experimentation with the chunking logic to achieve optimal results.</p>
<p>Check out the open-source <a href="https://github.com/forem/forem" target="_blank" rel="noopener noreferrer">Forem</a> application (formerly dev.to) for an excellent example of how to do "splitChunks."</p>
<h2 id="summing-up" class="title title-h2">
<a name="summing-up" class="anchor" href="#summing-up"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Summing up
</h2>
<p>Webpack can be a bit confusing to understand at first. Webpacker goes a long way toward providing that "conceptual compression" to get developers up-and-running on Rails. Unfortunately, Webpacker doesn't yet offer <em>all</em> the guard rails required to avoid problems like overpacking. As we've seen, some Rails apps are using Webpacker with an asset-pipeline mindset.</p>
<p>Embracing new tools may mean a little more investment, along with letting go of the way we used to do things.</p>
<p>Apply the Webpacker Packing Checklist to ensure a good experience for clients who desire faster webpages and developers who want fewer bugs.</p>
A visual guide to Webpacker/blog/visual-guide-to-webpacker.html2020-08-02T00:00:00+00:002020-08-02T00:00:00+00:00Ross Kaffenberger<p>Confused about how Webpacker works in Rails? Let's unpack it with some diagrams.</p>
<p>First, we'll take a look at the "classic" way of bundling assets with Rails: the Rails asset pipeline. Let's see how things look when requests are served with the Rails...</p><p>Confused about how Webpacker works in Rails? Let's unpack it with some diagrams.</p>
<p>First, we'll take a look at the "classic" way of bundling assets with Rails: the Rails asset pipeline. Let's see how things look when requests are served with the Rails asset pipeline in development.</p>
<blockquote>
<p>Struggling with webpack and Webpacker?</p>
<p><a href="https://buttondown.email/joyofrails" target="_blank" rel="noopener noreferrer"><strong>Subscribe to my newsletter for occasional emails with new content.</strong></a>.</p>
</blockquote>
<h4 id="html-request-with-the-asset-pipeline" class="title title-h4">
<a name="html-request-with-the-asset-pipeline" class="anchor" href="#html-request-with-the-asset-pipeline"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
HTML request with the asset pipeline
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-asset-pipeline-1.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>With the asset pipeline, when an HTML request is processed (1), bundling takes place within your Rails server (2). The asset pipeline is responsible for creating the bundled assets in the <code>public/assets/</code> directory and providing URLs to those assets to be rendered in the View.</p>
<pre><code><script src="/assets/application-3b2c...a7e.js">
</code></pre>
<blockquote>
<p>Note: Please consider the diagrams in this post as approximations of how the pieces fit together. Much of the detail has been omitted for simplicity and to draw attention to the salient aspects.</p>
</blockquote>
<h4 id="asset-request-with-the-asset-pipeline" class="title title-h4">
<a name="asset-request-with-the-asset-pipeline" class="anchor" href="#asset-request-with-the-asset-pipeline"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Asset request with the asset pipeline
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-asset-pipeline-2.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>As the browser parses the HTML response and finds that script tag, it must make an additional request to fetch the linked JavaScript resource from your Rails server. This time, the request is processed by (1) the <code>ActionDispatch::Static</code> middleware, which recognizes the asset lives on disk in the <code>public/assets/</code> directory (2).</p>
<p>Good so far?</p>
<h3 id="understanding-webpacker" class="title title-h3">
<a name="understanding-webpacker" class="anchor" href="#understanding-webpacker"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Understanding Webpacker
</h3>
<p>With Webpacker, things work differently.</p>
<p>While the Rails asset pipeline lives <em>within</em> the Rails server process, webpack, as you may know, is written in JavaScript and executes within the Node.js runtime. Therefore, webpack must run in a separate process.</p>
<p>Rails needs some "glue" to help it communicate with webpack. Webpacker provides the glue, which includes:</p>
<ul>
<li>webpack configuration,
<ul>
<li>i.e., <code>config/webpacker.yml</code> and <code>config/webpack/{production,development,test}.js</code></li>
</ul></li>
<li>helpers,
<ul>
<li>e.g. <code>javascript_pack_tag</code> and <code>stylesheet_link_tag</code></li>
</ul></li>
<li>executables,
<ul>
<li>i.e. <code>bin/webpack</code> and <code>bin/webpack-dev-server</code></li>
</ul></li>
<li>and middleware for development
<ul>
<li>i.e., <code>Webpacker::DevServerProxy</code></li>
</ul></li>
</ul>
<p>For the purpose of illustration, we'll focus on how Webpacker works in development. Webpacker supports two development "modes":</p>
<ul>
<li>compile on demand</li>
<li>dev server</li>
</ul>
<h3 id="compile-on-demand" class="title title-h3">
<a name="compile-on-demand" class="anchor" href="#compile-on-demand"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Compile on demand
</h3>
<p>Let's look at "compile on demand" first.</p>
<p>This mode is enabled when the <code>compile</code> option is set to <code>true</code> in <code>config/webpacker.yml</code>.</p>
<pre><code class="yaml">development:
compile: true
</code></pre>
<p>Here's how webpack fits into the picture.</p>
<h4 id="html-request-with-webpacker-compile" class="title title-h4">
<a name="html-request-with-webpacker-compile" class="anchor" href="#html-request-with-webpacker-compile"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
HTML request with Webpacker compile
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-compile-1.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>As it processes an HTML request (1), Rails will render the View, including "pack" tags such as the following:</p>
<pre><code class="erb"><%= stylesheet_pack_tag 'application', media: 'all' %>
<%= javascript_pack_tag 'application' %>
<%= image_pack_tag('media/images/apple-touch-icon.png') %>
</code></pre>
<p>Rails must be able to determine the URL for webpack JS, CSS, and image assets.</p>
<p>To do so, the Rails server will run webpack in a child process if the compiled assets are missing or stale (2).</p>
<p>The webpack config used to invoke webpack, <code>config/webpack/development.js</code> merges settings from <code>config/webpacker.yml</code> and the config provided by the <code>@rails/webpacker</code> NPM package (3).</p>
<p>Executing webpack will generate assets in the <code>public/packs/</code> directory along with the important <code>manifest.json</code> (4).</p>
<p>The <code>manifest.json</code> file is the key communication point from webpack to Rails: it provides a simple mapping to look up asset paths from their canonical names.</p>
<pre><code class="json">{
"application.js": "/packs/js/application-7efac57c4de0539bb941.js",
"application.css": "/packs/css/application-7526fd011c9ea132a45f.css",
"media/images/apple-touch-icon.png": "/packs/media/images/apple-touch-icon-b8d7025d5da762a9c1dd30980f412c92.png"
}
</code></pre>
<p>Rails will read this file via the Webpacker view helpers to render the correct URL to the assets.</p>
<h4 id="asset-request-with-webpacker-compile" class="title title-h4">
<a name="asset-request-with-webpacker-compile" class="anchor" href="#asset-request-with-webpacker-compile"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Asset request with Webpacker compile
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-compile-2.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>Once the browser has received the HTML response, it must make additional request to retrieve the assets. In "compile-on-demand" mode, this is identical to the asset pipeline:</p>
<p>The <code>ActionDispatch::Static</code> middleware (1) locates the asset on the file system in the <code>public/</code> directory (2) which is returned to the browser, short-circuiting the rest of the Rails process.</p>
<p>In production, of course, the role of the middleware may likely be played instead by a web server, such as Nginx, and/or a CDN.</p>
<h3 id="dev-server" class="title title-h3">
<a name="dev-server" class="anchor" href="#dev-server"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Dev server
</h3>
<p>The "compile on-demand" mode is similar to using the Rails asset pipeline in that you need only run the Rails server explicitly to get started. In development, I prefer "dev server" mode. This approach involves using the <code>webpack-dev-server</code> as a standalone HTTP server to compile assets and serve asset requests. Advantages include auto-recompilation, live-reloading, and <a href="https://webpack.js.org/concepts/hot-module-replacement/" target="_blank" rel="noopener noreferrer">hot module replacement</a>.</p>
<p>Using a tool like <a href="https://github.com/ddollar/foreman" target="_blank" rel="noopener noreferrer">foreman</a> or <a href="https://github.com/DarthSim/overmind" target="_blank" rel="noopener noreferrer">overmind</a>, we can boot up both the Rails server and the webpack-dev-server with a single command using a Procfile such as:</p>
<pre><code class="yaml"># Procfile.dev
rails: bin/rails server
webpack: bin/webpack-dev-server
</code></pre>
<p>We'll also set the <code>compile</code> option to <code>false</code> in <code>config/webpacker.yml</code>.</p>
<pre><code class="yaml">development:
compile: false
</code></pre>
<p>This way, Rails won't shell out to the webpack executable if the dev-server is running. The dev-server will be responsible for detecting changes to assets and recompiling automatically.</p>
<p>Let's see how requests are processed in "dev server" mode.</p>
<h4 id="html-request-with-dev-server" class="title title-h4">
<a name="html-request-with-dev-server" class="anchor" href="#html-request-with-dev-server"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
HTML request with dev server
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-dev-server-1.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>When the dev-server boots up (1), it "pre-compiles" assets in memory, instead of writing them to disk, for performance. It does still write the <code>manifest.json</code> file to disk in the <code>public/packs/</code> directory (2), where Rails can find it.</p>
<p>As with the other examples, as the request (3) is processed, the View will attempt to render javascript, stylesheet, and image tags. Rails will determine asset URLs from <code>manifest.json</code> (4). If the webpack-dev-server is not running or hasn't finished compiling as the request comes in, you may see the infamous <a href="https://github.com/rails/webpacker/issues/1730" target="_blank" rel="noopener noreferrer">Webpacker::Manifest::MissingEntryError</a>.</p>
<h4 id="asset-request-with-dev-server" class="title title-h4">
<a name="asset-request-with-dev-server" class="anchor" href="#asset-request-with-dev-server"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Asset request with dev server
</h4>
<p><img src="/assets/images/blog/visual-guide-to-webpacker/webpacker-dev-server-2.png" alt="Rails request with the asset pipeline" loading="lazy" /></p>
<p>As before, Rails middleware (1) locates the asset, this time, by proxying to the webpack-dev-server (2).</p>
<p>Recall that with the asset pipeline or webpack "compile on demand", the <code>ActionDispatch::Static</code> middleware located the assets on the file system in the <code>public/</code> directory. However, when using the webpack-dev-server, assets on not written to disk, they're created in memory; <code>ActionDispatch::Static</code> doesn't work in this case.</p>
<p>To solve this problem, Webpacker provides its own middleware: <code>Webpacker::DevServerProxy</code> (<a href="https://github.com/rails/webpacker/blob/bf278f9787704ed0f78038ad7d36c008abc2edfd/lib/webpacker/dev_server_proxy.rb#L3" target="_blank" rel="noopener noreferrer">source</a>). This allows Rails to act as an intermediary between the browser and the webpack-dev-server in development.</p>
<blockquote>
<p>As a side note, I'm not entirely sure why the proxy middleware would be necessary. Rails could point asset urls directly at webpack-dev-server by prepending the host, i.e., <code>http://localhost:3035/...</code>. Instead, by default in development, asset paths are produced so that the browser will direct its assets requests to the Rails server. I'm not entirely sure why the maintainers went this direction if only to make it seem more like the asset pipeline from the browser's perspective.</p>
</blockquote>
<h3 id="wrapping-up" class="title title-h3">
<a name="wrapping-up" class="anchor" href="#wrapping-up"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Wrapping up
</h3>
<p>At times, Rails can feel like a black box. I hope these mental maps help make it easier to navigate the world of Webpacker and webpack on Rails with confidence.</p>
Why doesn't Webpacker use my test config when I run Rails tests?/blog/why-doesnt-webpacker-use-my-test-config-when-i-run-rails-tests.html2020-07-20T00:00:00+00:002020-07-20T00:00:00+00:00Ross Kaffenberger<p>Here's something you might not expect: when running Rails tests, Webpacker will load the <em>development</em> webpack config instead of the test config by default.</p>
<p>To demonstrate, I'll use some "puts" debugging. Here's a <code>console.log</code> statement in the development...</p><p>Here's something you might not expect: when running Rails tests, Webpacker will load the <em>development</em> webpack config instead of the test config by default.</p>
<p>To demonstrate, I'll use some "puts" debugging. Here's a <code>console.log</code> statement in the development config.</p>
<pre><code class="javascript">// config/webpack/development.js
// ...
console.log('Loading config/webpack/development.js...')
</code></pre>
<p>When I run my RSpec tests while logging to STDOUT and RAILS_ENV set to test, the log line is displayed.</p>
<pre><code class="sh">$ RAILS_LOG_TO_STDOUT=true RAILS_ENV=test bin/rspec
# ...
[Webpacker] Compiling...
# ...
[Webpacker] Loading config/webpack/development.js...
# ...
</code></pre>
<p>If you've noticed this before, there's nothing wrong with your setup; this is the way Webpacker is setup to work out-of-the-box. Nevertheless, this is quite confusing and has lead to <a href="https://github.com/rails/webpacker/issues/2654" target="_blank" rel="noopener noreferrer">a recently reported issue</a> on the Webpacker GitHub repository.</p>
<p>It turns out, even though RAILS_ENV is set to 'test', NODE_ENV is set to 'development' (<a href="https://github.com/rails/webpacker/blob/bf278f9787704ed0f78038ad7d36c008abc2edfd/lib/install/bin/webpack#L4" target="_blank" rel="noopener noreferrer">source</a>) and the webpack config is determined directly from NODE_ENV.</p>
<h3 id="what-gives-" class="title title-h3">
<a name="what-gives-" class="anchor" href="#what-gives-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
What gives?
</h3>
<p>If the "test" NODE_ENV isn't used when I run my Rails tests, what is it good for?</p>
<p>First, let's make it clear that NODE_ENV has no explicit relationship to RAILS_ENV. Setting one of the ENV variables will have no effect on the other.</p>
<p>This is useful for debugging; for example, you can compile your production webpack build locally:</p>
<pre><code class="sh">NODE_ENV=production RAILS_ENV=development bin/rails s
</code></pre>
<p>Speaking of production, when running <code>rails assets:precompile</code> to compile your build, you don't have to set NODE_ENV to production explicitly because <a href="https://github.com/rails/webpacker/blob/bf278f9787704ed0f78038ad7d36c008abc2edfd/lib/tasks/webpacker/compile.rake#L21" target="_blank" rel="noopener noreferrer">Webpacker does this for you</a>. Otherwise, development is the default.</p>
<p>Another key point the production and development configurations are designed for compiling your JS for a real browser. Though they have different optimization characteristics, <a href="https://github.com/rails/webpacker/blob/bf278f9787704ed0f78038ad7d36c008abc2edfd/lib/install/config/babel.config.js#L28-L38" target="_blank" rel="noopener noreferrer">they share the same browser-focused Babel config</a> which will transform your nice ES6+ syntax into JavaScript your supported browsers will understand.</p>
<p>To recap so far: with Webpacker, NODE_ENV determines which webpack config it will use, i.e., <code>config/webpack/{development,test,production}.js</code> and will impact behavior in your Babel config, i.e., <code>babel.config.js</code>.</p>
<h3 id="testing-1-2-3" class="title title-h3">
<a name="testing-1-2-3" class="anchor" href="#testing-1-2-3"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Testing 1-2-3
</h3>
<p>This finally brings us to the use case for <code>NODE_ENV=test</code>:</p>
<p><em>JavaScript unit testing</em></p>
<p>By this I mean executing tests, written in JavaScript, against your application JavaScript code <em>within a Node.js process</em>.</p>
<p>We're talking <a href="https://github.com/mochajs/mocha" target="_blank" rel="noopener noreferrer">Mocha</a>, <a href="https://github.com/facebook/jest" target="_blank" rel="noopener noreferrer">Jest</a>, <a href="https://karma-runner.github.io/latest/index.html" target="_blank" rel="noopener noreferrer">Karma</a>, and more.</p>
<p>For some applications, JavaScript unit tests may not add much value, say if you're just doing a little DOM-manipulation with jQuery here and there. However, there may be some benefit to structuring your JavaScript utilities and components into discrete units which can be tested in isolation. And where there are discrete, testable units, there is room for unit testing. That's where JavaScript test runners come in.</p>
<p>The Rails unit testing for asset-pipeline compiled JavaScript is a bit cumbersome; it typically requires a gem, like <a href="https://github.com/jejacks0n/teaspoon" target="_blank" rel="noopener noreferrer">teaspoon</a> or <a href="https://github.com/searls/jasmine-rails" target="_blank" rel="noopener noreferrer">jasmine-rails</a>, that integrates tightly with the Rails asset pipeline by booting up both Rails and a browser to compile JavaScript and execute tests.</p>
<p>Webpacker opens the door to JavaScript unit test runners that can run in a Node.js process instead of a real browser (typically for speed). Jest, for example, <a href="https://jestjs.io/docs/en/configuration#testenvironment-string" target="_blank" rel="noopener noreferrer">executes tests against a "browser-like" environment called jsdom by default</a>. To support these node.js test runners, Webpacker's <a href="https://github.com/rails/webpacker/blob/bf278f9787704ed0f78038ad7d36c008abc2edfd/lib/install/config/babel.config.js#L20-L27" target="_blank" rel="noopener noreferrer">default Babel config targets the node.js runtime instead of a browser when <code>NODE_ENV=test</code></a>; this means Babel will transform your nice ES6+ syntax into JavaScript your current node version will understand assuming you set <code>NODE_ENV=test</code> for running your JavaScript unit tests.</p>
<h3 id="what-this-means-for-your-application" class="title title-h3">
<a name="what-this-means-for-your-application" class="anchor" href="#what-this-means-for-your-application"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
What this means for your application
</h3>
<p>You can see the potential problem then if you explicitly set <code>NODE_ENV=test</code> for your Rails system and integration tests without considering your Babel config; compiling your JavaScript for the Node.js runtime and loading in the browser may lead to some surprising issues. You can, of course, override this behavior if you really want; at least with this introduction provides some awareness of what you'd be getting yourself into.</p>
<p>You can <a href="https://jestjs.io/docs/en/webpack" target="_blank" rel="noopener noreferrer">setup Jest to compile your JavaScript through your webpack configuration</a>. If you follow the <a href="https://jestjs.io/docs/en/getting-started" target="_blank" rel="noopener noreferrer">general setup instructions for Jest</a>, it's possible to integrate to run your unit tests without webpack at all. Other test runners like <a href="https://karma-runner.github.io/latest/index.html" target="_blank" rel="noopener noreferrer">karma</a> offer similar options for running with or without webpack.</p>
<p>All this to say: your webpack test config, i.e. <code>config/webpack/test.js</code>, is essentially useless unless your application:</p>
<ol>
<li>uses a Node.js test runner for JavaScript unit test AND configure it to use your webpack config</li>
<li>overrides the defaults so that the test config is loaded in your Rails tests (just be sure to change Babel behavior)</li>
</ol>
<h3 id="what-about-webpacker-yml-" class="title title-h3">
<a name="what-about-webpacker-yml-" class="anchor" href="#what-about-webpacker-yml-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
What about webpacker.yml?
</h3>
<p>Also, as I've described previously in <a href="/blog/how-to-use-webpacker-yml.html">Understanding webpacker.yml</a>, Webpacker provides a webpack configuration while merging settings declared in <code>config/webpacker.yml</code> from YAML to JavaScript. This file contains settings for production, development, and test environments as do most Rails-y YAML files. Unlike the webpack config, webpacker.yml settings are determined by the current RAILS_ENV.</p>
<p>This means, webpacker.yml test settings are merged into the development webpack config when running your Rails tests.</p>
<h3 id="wrapping-up" class="title title-h3">
<a name="wrapping-up" class="anchor" href="#wrapping-up"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Wrapping up
</h3>
<p>Does all of this seem a little confusing? I agree. Here's a breakdown of how webpack configuration maps to RAILS_ENV and NODE_ENV in various contexts.</p>
<table style="font-size:85%; margin-bottom: 2em;">
<thead>
<tr style="border-bottom: 1px solid #CCC">
<th>Context</th>
<th>RAILS_ENV</th>
<th>webpacker.yml</th>
<th>NODE_ENV</th>
<th>webpack config</th>
</tr>
</thead>
<tbody>
<tr style="border-bottom: 1px solid #CCC">
<td>Deployed app</td>
<td>production</td>
<td>production</td>
<td>production</td>
<td>config/webpack/production.js</td>
</tr>
<tr style="border-bottom: 1px solid #CCC">
<td>Local server</td>
<td>development</td>
<td>development</td>
<td>development</td>
<td>config/webpack/development.js</td>
</tr>
<tr style="border-bottom: 1px solid #CCC">
<td>Rails tests</td>
<td>test</td>
<td>test</td>
<td>development</td>
<td>config/webpack/development.js</td>
</tr>
<tr style="border-bottom: 1px solid #CCC">
<td>JS unit tests</td>
<td>n/a</td>
<td>n/a</td>
<td>test</td>
<td>config/webpack/test.js <i>if</i> test runner is configured to use webpack</td>
</tr>
</tbody>
</table>
<p><em>In summary</em>:</p>
<blockquote>
<p>RAILS_ENV determines which Webpacker YAML settings are used and NODE_ENV determines which webpack configuration is used.</p>
</blockquote>
<p>Whether or not you find the use case for JavaScript unit tests compelling, it helps to know that Webpacker does not make any distinction between your development and test environments beyond the settings in your webpacker.yml; both are local concerns that target the same runtime, i.e., the browser.</p>
<hr>
<p>If you found this helpful, please consider subscribing to my newsletter to stay tuned for more on upping your "JavaScript on Rails" game.</p>
The webpack plugin I can't live without/blog/webpacker-output-analysis-with-webpack-bundle-analyzer.html2020-05-17T00:00:00+00:002020-05-17T00:00:00+00:00Ross Kaffenberger<blockquote>
<p>tl;dr Install the <a href="https://github.com/webpack-contrib/webpack-bundle-analyzer" target="_blank" rel="noopener noreferrer"><code>webpack-bundle-analyzer</code></a> to visualize what's included in your webpack bundles and debug common problems.</p>
</blockquote>
<p>Does webpack feel still a bit scary? Maybe a bit too magical? Too much of <em>WTF is going on here?</em></p>
<p>It felt that way for me once...</p><blockquote>
<p>tl;dr Install the <a href="https://github.com/webpack-contrib/webpack-bundle-analyzer" target="_blank" rel="noopener noreferrer"><code>webpack-bundle-analyzer</code></a> to visualize what's included in your webpack bundles and debug common problems.</p>
</blockquote>
<p>Does webpack feel still a bit scary? Maybe a bit too magical? Too much of <em>WTF is going on here?</em></p>
<p>It felt that way for me once. I was struggling to <a href="https://rossta.net/blog/from-sprockets-to-webpack.html" target="_blank" rel="noopener noreferrer">switch from Sprockets to Webpacker in a large Rails app</a>. With Sprockets, I could require a jQuery plugin through a magic comment (the require directive), and it would "Just Work."</p>
<p>Such was not the case when I first started using webpack; ever seen an error like on the console?</p>
<pre><code class="js">'Uncaught TypeError: $(...).fancybox is not a function'
</code></pre>
<p>Yeah, you and me both.</p>
<p>Then one day, it all clicked for me.</p>
<p>My main problem was <em>I didn't have a good mental model how webpack worked.</em> To form that mental model, I researched dozens of articles, watched numerous screencasts, and read a lot of source code. One thing helped "flip the switch" more than anything else: understanding the product of a webpack build, the output.</p>
<p><em>It was right there in front of me the whole time.</em></p>
<p>Now you might call me crazy to say, "you should read the source of your bundled output," even assuming we're talking about the unminified/unobfuscated development build, so I'm not going to tell you to go do that. (Not without some guidance; let's save that for a future project).</p>
<p>But you can use a tool <em>right now</em> to <strong>visualize</strong> what's in your bundle. And that could be enough to make all the difference in helping you understand, at least at a high level, how webpack does its thing.</p>
<h3 id="introducing-the-webpack-bundle-analyzer" class="title title-h3">
<a name="introducing-the-webpack-bundle-analyzer" class="anchor" href="#introducing-the-webpack-bundle-analyzer"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Introducing the webpack-bundle-analyzer
</h3>
<p>But, there is something else you can do that requires a lot less work: you can use the <code>webpack-bundle-analyzer</code>. You can probably get it up-and-running in less time than it takes to read this article.</p>
<blockquote>
<p>Curious about or need help with webpack? I may be able to help! I'm developing a course for webpack on Rails and I frequently write about it on this blog.</p>
<p><a href="https://buttondown.email/joyofrails" target="_blank" rel="noopener noreferrer"><strong>Subscribe to my newsletter to get updates</strong></a>.</p>
</blockquote>
<p>The webpack-bundle-analyzer is a tool that you can use to visualize the contents of a webpack build. It parses the "stats" output of a webpack build and constructs an interactive <a href="https://www.jasondavies.com/voronoi-treemap/" target="_blank" rel="noopener noreferrer">Voronoi treemap</a> using the <a href="https://carrotsearch.com/foamtree/" target="_blank" rel="noopener noreferrer">FoamTree</a> library.</p>
<p>It might look a little something like this:</p>
<p><img src="/assets/images/blog/webpack/analyzer-single-bundle-1.png" alt="An example of a Voronoi treemap output by the webpack-bundle-analyzer" loading="lazy" /></p>
<blockquote>
<p>Funny story, this wasn't the first time I've come across Voronoi diagrams. The hands-down best Computer Science class I took at NYU was <a href="https://cs.nyu.edu/courses/fall16/CSCI-GA.2965-001/" target="_blank" rel="noopener noreferrer">Heuristics</a> with Dennis Shasha in which we learned algorithms for approximating solutions to NP-hard problems and applied them to compete in automated 2-player competitive battles including a <a href="https://cs.nyu.edu/courses/fall16/CSCI-GA.2965-001/voronoi_gravitational.html" target="_blank" rel="noopener noreferrer">gravitation Voronoi game</a>. My source code is up on GitHub somewhere useful to no one else, serving mostly as a reminder I can accomplish big things under challenging constraints.</p>
</blockquote>
<p>The analyzer will represent multiple bundles as distinct colors with relative sizes:
<img src="/assets/images/blog/webpack/analyzer-multiple-bundles.png" alt="webpack-bundle-analyzer multiple bundles" loading="lazy" /></p>
<p>Individual modules are displayed in their relative sizes. Hover over bundles and modules to view statistics. Click or scroll to zoom in:
<img src="/assets/images/blog/webpack/analyzer-module-closeup.png" alt="webpack-bundle-analyzer module closeup" loading="lazy" /></p>
<p>Use the slide-out menu on the left to toggle the gzipped and parsed ("un"-gzipped) bundles:
<img src="/assets/images/blog/webpack/analyzer-menu-closeup.png" alt="webpack-bundle-analyzer close up of menu" loading="lazy" /></p>
<p>Highlight modules that match a search phrase, like "react":
<img src="/assets/images/blog/webpack/analyzer-module-highlight.png" alt="webpack-bundle-analyzer module highlight" loading="lazy" /></p>
<p>Are you using Moment.js? It might be including translations for all its locales by default at enormous cost. <a href="https://github.com/jmblog/how-to-optimize-momentjs-with-webpack" target="_blank" rel="noopener noreferrer">Consider using only the locales you need</a>.
<img src="/assets/images/blog/webpack/analyzer-moment-locales.png" alt="webpack-bundle-analyzer moment locales" loading="lazy" /></p>
<h4 id="key-questions" class="title title-h4">
<a name="key-questions" class="anchor" href="#key-questions"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Key questions
</h4>
<p>Here are just some examples of questions the webpack-bundle-analyzer can help answer:</p>
<blockquote>
<ol>
<li>Why is this bundle so large?</li>
<li>What are the relative sizes of each <em>bundle</em> in the webpack build?</li>
<li>What are the relative sizes of each <em>module</em> in the webpack build?</li>
<li>Where is my business logic bundled?</li>
<li>Are the modules I expect included?</li>
<li>Are any modules included more than once?</li>
<li>Are there modules I expect to be excluded?</li>
<li>Which third-party libraries are bundled?</li>
<li>Which bundle contains $MODULE_NAME?</li>
<li>Is <a href="https://webpack.js.org/guides/tree-shaking/" target="_blank" rel="noopener noreferrer">tree-shaking</a>* working?</li>
<li>WTF is in this bundle?</li>
</ol>
<p><strong>Glossary alert</strong> "Tree shaking" is jargon for dead code elimination: the process of removing unreferenced code from your build. Webpack will perform tree shaking when running in "production" mode which is enabled when building assets using <code>rake assets:precompile</code> or via <code>./bin/webpack</code> with <code>RAILS_ENV=production</code> and <code>NODE_ENV=production</code>. I'll share more about how to take full advantage of tree shaking in future posts.</p>
</blockquote>
<p>In short, webpack-bundle-analyzer graphs what is happening in your build. It can help you debug unexpected behavior or optimize your build output to reduce bundle size. All that, in service of better user experience!</p>
<h3 id="installation" class="title title-h3">
<a name="installation" class="anchor" href="#installation"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Installation
</h3>
<p>The <code>webpack-bundle-analyzer</code> is distributed as an NPM package. To install via yarn:</p>
<pre><code class="sh">yarn add --dev webpack-bundle-analyzer
</code></pre>
<p>Since this tool is typically only useful for local development, we add it to <code>devDependencies</code> using the <code>--dev</code> flag.</p>
<h3 id="usage" class="title title-h3">
<a name="usage" class="anchor" href="#usage"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Usage
</h3>
<p>To use the webpack-bundler-analyzer, you can either integrate it as a plugin to your Webpacker configuration or you use a two-step command line process.</p>
<p>Typically, it makes the most sense to analyze the output of production builds since they will be what's delivered to the client and may contain several optimizations that will make the output differ significantly from the development build. Analyzing the development build can still be useful for additional context when debugging.</p>
<p>Though the instructions are tailored to a Rails project using <a href="https://github.com/rails/webpacker" target="_blank" rel="noopener noreferrer">Webpacker</a>, you could adapt them to any webpack project.</p>
<p>When the analyzer is run, it will launch a local webserver; visit <a href="http://locahost:8888" target="_blank" rel="noopener noreferrer">http://locahost:8888</a> to view the treemap. The <a href="https://github.com/webpack-contrib/webpack-bundle-analyzer#options-for-plugin" target="_blank" rel="noopener noreferrer">port is configurable</a>, and you'll need to hit Ctrl+C to stop the server.</p>
<h4 id="option-1-analyze-json-from-command-line" class="title title-h4">
<a name="option-1-analyze-json-from-command-line" class="anchor" href="#option-1-analyze-json-from-command-line"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Option 1: Analyze JSON from command line
</h4>
<p>The <code>webpack-bundle-analyzer</code> package ships with a command-line interface (CLI) that can ingest a webpack JSON stats file. In other words, it's a two-step process in which we generate a webpack build that's outputs build stats to a JSON file and then run the <code>webpack-bundle-analyzer</code> CLI to analyze the build stats and the output bundles generated in the build:</p>
<p>In a Rails project, we might typically first run the webpack build:</p>
<pre><code class="sh">bin/webpack --profile --json > tmp/webpack-stats.json
</code></pre>
<p>Then we would analyze the output with the command <code>webpack-bundle-analyzer [stats file] [output directory]</code>:</p>
<pre><code class="sh">npx webpack-bundle-analyzer tmp/webpack-stats.json public/packs
</code></pre>
<blockquote>
<p><code>npx</code> is a separate command-line interface that is installed along with <code>node</code>. It will look for the command you specify in your locally installed <code>node_modules</code>. In other words, this replaces <code>./bin/node_modules/webpack-bundle-analyzer ...</code>.
Get this: with <code>npx</code>, the package script you're trying to run <em>doesn't even need to be installed</em>! Yes, that's right: if you want, you can skip <code>yarn add webpack-bundle-analyzer</code>. Use <code>npx webpack-bundler-analyzer</code> as if it's installed globally. <code>npx</code> will search your locally installed packages and will look up the package on the remote npm registry when not found locally. Pretty cool!</p>
</blockquote>
<p>Since I don't want to type all that out every time, I put those commands in the <code>scripts</code> section of my <code>package.json</code>:</p>
<pre><code class="json">// package.json
{
// ...
"scripts": {
"webpack:analyze": "yarn webpack:build_json && yarn webpack:analyze_json",
"webpack:build_json": "RAILS_ENV=${RAILS_ENV:-production} NODE_ENV=${NODE_ENV:-production} bin/webpack --profile --json > tmp/webpack-stats.json",
"webpack:analyze_json": "webpack-bundle-analyzer tmp/webpack-stats.json public/packs"
}
}
</code></pre>
<p>To analyze the build using these npm scripts, run:</p>
<pre><code class="sh">yarn webpack:analyze
</code></pre>
<p>You could instead write this as a rake tasks as follows:</p>
<pre><code class="ruby">namespace :webpack do
desc "Analyze the webpack build"
task :analyze => [:build_json, :analyze_json]
task :build_json do
system "RAILS_ENV=#{ENV.fetch('RAILS_ENV', 'production')} \
NODE_ENV=#{ENV.fetch('NODE_ENV', 'production')} \
bin/webpack --profile --json > tmp/webpack-stats.json"
end
task :analyze_json do
system "npx webpack-bundle-analyzer tmp/webpack-stats.json public/packs"
rescue Interrupt
end
end
</code></pre>
<p>To analyze the build using these rake tasks, run:</p>
<pre><code class="sh">rake webpack:analyze
</code></pre>
<h4 id="option-2-integrated-setup" class="title title-h4">
<a name="option-2-integrated-setup" class="anchor" href="#option-2-integrated-setup"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Option 2: Integrated setup
</h4>
<p>Instead of using separate scripts to trigger the bundle analyzer, you can instead incorporate the webpack-bundle-analyzer into your webpack configuration. Doing so runs the webpack-bundle-analyzer localhost server as a side effect of running the build command.</p>
<p>Below, we'll look at how you can integrate the analyzer into a Rails using <a href="https://github.com/rails/webpacker" target="_blank" rel="noopener noreferrer">Webpacker</a>.</p>
<pre><code class="javascript">// config/webpack/environment.js
const { environment } = require('@rails/webpacker')
if (process.env.WEBPACK_ANALYZE === 'true') {
const BundleAnalyzerPlugin =
require('webpack-bundle-analyzer').BundleAnalyzerPlugin
environment.plugins.append(
'BundleAnalyzerPlugin',
new BundleAnalyzerPlugin(),
)
}
module.exports = environment
</code></pre>
<p>Note that the plugin is incorporated into the webpack config only with the environment variable <code>WEBPACK_ANALYZE=true</code>, so it is only added to the configuration as an opt-in feature.</p>
<p>To visualize the production build, run this command instead:</p>
<pre><code class="sh">WEBPACK_ANALYZE=true RAILS_ENV=production NODE_ENV=production ./bin/webpack
</code></pre>
<p>You could even run the analyzer server alongside your webpack-dev server with <code>WEBPACK_ANALYZE=true ./bin/webpack-dev-server</code> to get instant feedback. Keep in mind that the bundle analysis while in development mode will yield different results from the production build.</p>
<h4 id="rails-template" class="title title-h4">
<a name="rails-template" class="anchor" href="#rails-template"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Rails template
</h4>
<p>For your convenience, I packaged <a href="https://railsbytes.com/public/templates/Xo5sYr" target="_blank" rel="noopener noreferrer">this changeset as a Rails template</a> on <a href="https://railsbytes.com" target="_blank" rel="noopener noreferrer">railsbytes.com</a>.</p>
<p>You can preview this template at <a href="https://railsbytes.com/public/templates/Xo5sYr" target="_blank" rel="noopener noreferrer">https://railsbytes.com/public/templates/Xo5sYr</a>. To use the template, skip the steps above and run the following command:</p>
<pre><code class="sh">rails app:template LOCATION="https://railsbytes.com/script/Xo5sYr"
</code></pre>
<h3 id="what-39-s-next-" class="title title-h3">
<a name="what-39-s-next-" class="anchor" href="#what-39-s-next-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
What's next?
</h3>
<p>So you've set up the webpack-bundle-analyzer and started understanding what's happening in your webpack bundles, what now? You may have noticed some things you don't like. In future posts, I'll be examining how you can deal with the excesses, including:</p>
<ul>
<li>Replacing libraries with built-in browser functionality or smaller packages</li>
<li>Taking full advantage of tree-shaking with imports</li>
<li>Using webpack to filter out unnecessary imports</li>
<li>The "right way" to split bundles for multi-page applications</li>
<li>Code-splitting with dynamic imports</li>
</ul>
<p>Until then, here are some more resources you can use:</p>
<ul>
<li><a href="https://developers.google.com/web/fundamentals/performance/webpack/monitor-and-analyze" target="_blank" rel="noopener noreferrer">Google: Monitor and analyze the app</a></li>
<li><a href="https://www.youtube.com/watch?v=ltlxjq4YEKU" target="_blank" rel="noopener noreferrer">Video: How to use the webpack bundle analyzer</a></li>
<li><a href="https://github.com/jmblog/how-to-optimize-momentjs-with-webpack" target="_blank" rel="noopener noreferrer">How to optimize momentjs with webpack</a></li>
<li><a href="https://www.blazemeter.com/blog/the-correct-way-to-import-lodash-libraries-a-benchmark" target="_blank" rel="noopener noreferrer">The correct wat to import lodash</a></li>
<li><a href="https://www.youtube.com/watch?v=Da6VxdGU2Ig" target="_blank" rel="noopener noreferrer">Managing your bundle size (video)</a></li>
</ul>
Why does Rails 6 include both Webpacker and Sprockets?/blog/why-does-rails-install-both-webpacker-and-sprockets.html2020-05-09T00:00:00+00:002020-05-09T00:00:00+00:00Ross Kaffenberger<p>Since you're reading this post, chances are you've heard Rails 6 installs both Webpacker and Sprockets and you're wondering WTF is going on. By the way, it's <a href="https://weblog.rubyonrails.org/2020/5/7/A-May-of-WTFs/" target="_blank" rel="noopener noreferrer">a whole May of WTFs for Rails</a>.</p>
<p><strong>Wait, don't Sprockets and Webpacker basically do the same...</strong></p><p>Since you're reading this post, chances are you've heard Rails 6 installs both Webpacker and Sprockets and you're wondering WTF is going on. By the way, it's <a href="https://weblog.rubyonrails.org/2020/5/7/A-May-of-WTFs/" target="_blank" rel="noopener noreferrer">a whole May of WTFs for Rails</a>.</p>
<p><strong>Wait, don't Sprockets and Webpacker basically do the same thing?</strong></p>
<p>If this is what you're thinking, you're not alone.</p>
<blockquote>
<p>Curious about or need help with webpack? I may be able to help! I'm developing a course for Webpack on Rails and I frequently write about it on this blog.</p>
<p><a href="https://buttondown.email/joyofrails" target="_blank" rel="noopener noreferrer"><strong>Subscribe to my newsletter to get updates</strong></a>.</p>
</blockquote>
<p>The question keeps coming up, like in this <a href="https://www.reddit.com/r/rails/comments/9zg7fe/confused_about_the_difference_between_sprockets/" target="_blank" rel="noopener noreferrer">Reddit post</a>, or this <a href="https://stackoverflow.com/questions/55232591/rails-5-2-why-still-use-assets-pipeline-with-webpacker" target="_blank" rel="noopener noreferrer">StackOverflow question</a>, or this <a href="https://www.reddit.com/r/rails/comments/dfww82/best_practice_for_webpacker_in_rails_6_do_i_need/" target="_blank" rel="noopener noreferrer">other Reddit post</a>.</p>
<p>Here's my colleague <a href="https://twitter.com/danmayer" target="_blank" rel="noopener noreferrer">@danmayer</a>:</p>
<p><blockquote class="twitter-tweet" data-conversation="none"><p lang="en" dir="ltr">How and where to handle assets is in a confusing state, 1 foot in asset pipeline and one foot in webpacker... If that is going to be a long last direction vs a transition we should make the best practices more clear in guides and how to ensure they play nicely together</p>— Dan Mayer (@danmayer) <a href="https://twitter.com/danmayer/status/1258577270760804353?ref_src=twsrc%5Etfw">May 8, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script></p>
<p>Even <a href="https://twitter.com/avdi" target="_blank" rel="noopener noreferrer">@avdi</a> just last week:</p>
<p><blockquote class="twitter-tweet"><p lang="en" dir="ltr">Will someone please explain to me why after all the foofaraw about Rails 6 going to webpack, I'm still having to unfuck Sprockets in my application.rb</p>— Avdi Grimm (@avdi) <a href="https://twitter.com/avdi/status/1256742291890413570?ref_src=twsrc%5Etfw">May 3, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script></p>
<p>It's clear the Webpacker-Sprockets co-existence is catching many by surprise. There's good reason for that too.</p>
<p>You wouldn't be wrong to think Sprockets and webpack solve the same general problem:</p>
<p><em>Packaging assets (JavaScript, CSS, images, fonts) for the browser</em></p>
<p>The similarities exist. Both Sprockets and webpack will:</p>
<ul>
<li>combine many source files into one or a few destination bundles for production</li>
<li>transpile source files from one syntax to another</li>
<li>minify and fingerprint assets when building for production</li>
<li>rebuild modified source files in development incrementally</li>
<li>do all of the above for both JavaScript and CSS</li>
</ul>
<p>However, Sprockets and webpack solve asset bundling in very different ways.</p>
<p>Sprockets was introduced way back in 2007 (!), before Node.js, before the Cambrian explosion of JavaScript, before module specifications like CommonJS, AMD, and EcmaScript modules, before webpack, browserify and $ANY_MODULE_AWARE_ASSET_BUNDLER. Sprockets has not attempted to keep up with improvements in tooling, language features, and browser capabilities (save for source maps) as other projects in JavaScript community have.</p>
<p>Webpack, on the other hand, fully embraces the concept of JavaScript modules. It integrates with Babel, PostCSS, and just about any recent web framework. It supports a number of module syntaxes, including <a href="https://webpack.js.org/guides/code-splitting/#dynamic-imports" target="_blank" rel="noopener noreferrer">dynamic imports</a> for <a href="https://webpack.js.org/guides/code-splitting/" target="_blank" rel="noopener noreferrer">code splitting</a>. There are a wide variety of <a href="https://webpack.js.org/configuration/devtool/" target="_blank" rel="noopener noreferrer">configurable source map options</a>. Top to bottom, the webpack compilation process is extremely modular and customizable.</p>
<h3 id="so-why-would-rails-include-both-" class="title title-h3">
<a name="so-why-would-rails-include-both-" class="anchor" href="#so-why-would-rails-include-both-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
So why would Rails include both?
</h3>
<p>Here's the answer plain and simple straight from DHH back in 2016 when Webpack was first introduced as the recommended JavaScript compiler with Rails 5.1.</p>
<p><blockquote class="twitter-tweet"><p lang="en" dir="ltr">We will continue to use the asset pipeline for JavaScript sprinkles, CSS, images, and other static stuff. The two approaches coexist great.</p>— DHH (@dhh) <a href="https://twitter.com/dhh/status/808349072734027776?ref_src=twsrc%5Etfw">December 12, 2016</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script></p>
<blockquote>
<p>We will continue to use the asset pipeline for JavaScript sprinkles, CSS, images, and other static stuff. The two approaches coexist great.</p>
</blockquote>
<p>To elaborate on this decision, there was a telling response from DHH on his GitHub pull request to <a href="https://github.com/rails/rails/pull/33079#issuecomment-400140840" target="_blank" rel="noopener noreferrer">make Webpacker the default JavaScript compiler in Rails 6</a>:</p>
<p><a href="https://github.com/rails/rails/pull/33079#issuecomment-400140840" target="_blank" rel="noopener noreferrer"><img src="/assets/images/blog/webpack/dhh-awkward.png" alt="DHH: Webpack’s support is awkward in my opinion and does not offer any benefits over Sprockets. Unlike in the realm of JavaScript compilation." loading="lazy" /></a></p>
<blockquote>
<p><strong>@dwightwatson</strong> Out of curiousity, what is the argument to continue using Sprockets for CSS/static assets when Webpacker supports them by default out of the box?</p>
<p><strong>@dhh</strong> Webpack’s support is awkward in my opinion and does not offer any benefits over Sprockets. Unlike in the realm of JavaScript compilation.</p>
</blockquote>
<p>There's a lot to unpack there.</p>
<p>When it comes to asset bundling, the "Rails way" is webpack for JavaScript and Sprockets for everything else. The default setup in a fresh Rail 6 install, similar to what Basecamp uses, still compiles CSS, images, and fonts with Sprockets.</p>
<p>This means, if you're a member of the Basecamp camp, all your webpack JavaScript source files would live in <code>app/javascript</code> and all your Sprockets CSS and images would remain in <code>app/assets</code>. Running <code>rails assets:precompile</code> will first build all the Sprockets assets into the <code>public/assets</code> directory, then will build all the webpack assets into the <code>public/packs</code> directory.</p>
<p>To be very clear, this does not mean you need to run both Sprockets and Webpacker to serve assets for the browser. The two processes for bundling assets are completely separate and they do not share dependencies. Different helpers, different implementations, different directories, different, different, different. They are built in such a way that they can cohabitate a Rails application.</p>
<p>On the other hand, you could use <em>only</em> Sprockets or <em>only</em> Webpacker to bundle all your assets.</p>
<h3 id="feeling-awkward-" class="title title-h3">
<a name="feeling-awkward-" class="anchor" href="#feeling-awkward-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Feeling, awkward?
</h3>
<p>DHH calls webpack's approach to handling non-JavaScript assets <em>awkward</em>. Now, I happen to like webpack a lot. But he's not wrong.</p>
<p>He says this because, to bundle CSS and images in webpack, you need to <em>import CSS and images from your JavaScript files</em>.</p>
<pre><code class="javascript">import '../application.css'
import myImageUrl from '../images/my-image.jpg'
</code></pre>
<p>The reason for this is that webpack wants to treat <strong>everything</strong> as a JavaScript module. I mean <em>everything</em>.</p>
<p>All JavaScript imports are treated as JavaScript modules. To use CSS with webpack, you import it as you would a JavaScript module. To use an image with webpack, you import it as you would a JavaScript module. Depending on your perspective, this may be unusual—perhaps especially for Rails developers coming from Sprockets.</p>
<p>This isn't just a "Rails opinion". Consider this recent tweet from a prominent voice in the React community, <a href="https://twitter.com/ryanflorence" target="_blank" rel="noopener noreferrer">Ryan Florence</a>:</p>
<p><blockquote class="twitter-tweet"><p lang="en" dir="ltr">import url from "./whatever.jpg"<br>import html from "./some.md"<br>import str from "raw!./some.js"<br><br>So ... I gotta admit I love this stuff, but did we jump the shark here with JavaScript build tools? Should this stuff happen outside the JavaScript module bundler?</p>— Ryan Florence (@ryanflorence) <a href="https://twitter.com/ryanflorence/status/1258966331572928514?ref_src=twsrc%5Etfw">May 9, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script></p>
<p>Sounds a lot like discovering Sprockets in reverse? (I'm not surprised at that 50/50 split either).</p>
<p>While awkward to some, webpack's "Everything is a Module" mindset is also extremely powerful. There are some interesting possibilities when a tool goes <strong>all in</strong> with such a mental model. Think of what "Everything is an Object" has done for Ruby.</p>
<h3 id="choosing-webpacker-or-sprockets-or-both-" class="title title-h3">
<a name="choosing-webpacker-or-sprockets-or-both-" class="anchor" href="#choosing-webpacker-or-sprockets-or-both-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Choosing Webpacker or Sprockets (or both)
</h3>
<p>The good news is there's no need to stress about it. Rails defaults mirror the preferred approach of the Basecamp team, but that doesn't mean you have to agree or that it's the right way to do things for your application. You can use both, as Basecamp does, or choose one over the other.</p>
<p>To help you decide, I adapted <a href="https://github.com/reactjs/react-rails/wiki/Choosing-Sprockets-or-Webpacker" target="_blank" rel="noopener noreferrer">this excellent guide from the react-rails project</a>:</p>
<h4 id="why-sprockets-" class="title title-h4">
<a name="why-sprockets-" class="anchor" href="#why-sprockets-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why Sprockets?
</h4>
<ul>
<li>My Rails app does not need much JavaScript</li>
<li>I prefer global scripts and jQuery plugin enhancement, i.e. I don't need a proper JavaScript module system</li>
<li>Upgrading my legacy Rails app to Webpacker would be too costly</li>
<li>I don't need advanced tooling for local development</li>
<li>It just works and I don't have time to ramp up on alternatives</li>
<li>My Rails app relies on specific asset gems and I don't have NPM alternatives</li>
</ul>
<h4 id="why-not-sprockets-" class="title title-h4">
<a name="why-not-sprockets-" class="anchor" href="#why-not-sprockets-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why not Sprockets?
</h4>
<ul>
<li>Sprockets is slowing down my local development experience</li>
<li>I need more control over aspects of our asset compilation</li>
<li>My app has a lot of JavaScript and needs code-splitting features to avoid massive payloads</li>
<li>I'm concerned about long-term support</li>
</ul>
<h4 id="why-webpacker-" class="title title-h4">
<a name="why-webpacker-" class="anchor" href="#why-webpacker-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why Webpacker?
</h4>
<ul>
<li>I want to use a proper JavaScript module system to manage dependencies, i.e., limit global scope pollution and have an explicit dependency graph with import/export and require</li>
<li>I want to take advantage of the cutting edge features from ES6+, Babel, PostCSS</li>
<li>I want intelligent code-splitting features such as dynamic imports and webpack's splitChunks optimization</li>
<li>I want more flexibility with how my build system generates source maps</li>
<li>I want advanced tooling for local development including hot module replacement</li>
<li>I want to build Single Page Apps*</li>
</ul>
<p>*You don't need to have a Single Page App to use webpack; it works quite well for "Multi Page Apps".</p>
<h4 id="why-not-webpacker-" class="title title-h4">
<a name="why-not-webpacker-" class="anchor" href="#why-not-webpacker-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why not Webpacker?
</h4>
<ul>
<li>My Rails app does not need much JavaScript</li>
<li>I am a backend developer with limited knowledge of JavaScript ecosystem</li>
<li>I am not ready to invest time to understand webpack and Webpacker</li>
<li>It seems too complicated</li>
</ul>
<h4 id="why-use-both-" class="title title-h4">
<a name="why-use-both-" class="anchor" href="#why-use-both-"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Why use both?
</h4>
<ul>
<li>I prefer the "Rails way": Webpacker to compile JavaScript, Sprockets for CSS, images, and fonts</li>
<li>I'm upgrading from Sprockets to Webpacker incrementally</li>
</ul>
<h3 id="on-a-personal-note" class="title title-h3">
<a name="on-a-personal-note" class="anchor" href="#on-a-personal-note"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
On a personal note
</h3>
<p>I want to leave Sprockets behind. Sprockets was a huge leap forward for asset management when it was first introduced but it hasn't taken advantage of newer possibilities. It languishes while webpack's key features, such as performance optimizations through code-splitting, are first class.</p>
<p>Webpack is more complex and does require some investment. For me, it's been worth it.</p>
<p><a href="https://rossta.net/blog/reasons-to-switch-to-webpacker.html" target="_blank" rel="noopener noreferrer">I think webpack is a great choice for any application with a significant amount of JavaScript.</a></p>
<p>Which is right for you?</p>
How to debug webpack on Rails/blog/how-to-debug-webpack-on-rails.html2020-05-04T00:00:00+00:002020-05-04T00:00:00+00:00Ross Kaffenberger<p>It's nice that the Rails Webpacker gem and NPM package abstracts your webpack config... that is until you need to make changes.</p>
<p>In <a href="/blog/how-to-customize-webpack-for-rails-apps.html">my previous post</a>, I talked about how to customize the webpack config... but how can you be sure you're making the right...</p><p>It's nice that the Rails Webpacker gem and NPM package abstracts your webpack config... that is until you need to make changes.</p>
<p>In <a href="/blog/how-to-customize-webpack-for-rails-apps.html">my previous post</a>, I talked about how to customize the webpack config... but how can you be sure you're making the right change? The webpack config is JavaScript, so you can't simply jump into the Rails console to poke around. But you do have some other tools at your disposal though.</p>
<p>In this post, I'll share some tips for debugging the webpack config in your Rails app.</p>
<h3 id="the-one-liner" class="title title-h3">
<a name="the-one-liner" class="anchor" href="#the-one-liner"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
The one-liner
</h3>
<blockquote>
<p>For the following examples, I'm using Node v12.13.1.</p>
</blockquote>
<p>Here's a quick one-liner for printing the entire Rails webpack config in development:</p>
<pre><code class="sh">$ RAILS_ENV=development node -e 'console.dir(require("./config/webpack/development"), { depth: null })'
</code></pre>
<p>I like <a href="https://nodejs.org/api/console.html#console_console_dir_obj_options" target="_blank" rel="noopener noreferrer"><code>console.dir</code></a> as a nice alternative to <code>console.log</code> for inspecting JavaScript objects.</p>
<p>For inspecting the test or production configs, just update the RAILS_ENV and the target file:</p>
<pre><code class="sh">$ RAILS_ENV=development node -e 'console.dir(require("./config/webpack/development"), { depth: null })'
# OR
$ RAILS_ENV=test node -e 'console.dir(require("./config/webpack/test"), { depth: null })'
# OR
$ RAILS_ENV=production node -e 'console.dir(require("./config/webpack/production"), { depth: null })'
</code></pre>
<p>We ensure the RAILS_ENV is set so Webpacker's NPM package will load the correct settings from your <code>config/webpacker.yml</code> file.</p>
<p>To make it even easier, I'll put this into a script file in <code>bin/inspect_webpack</code> with my Rails projects.</p>
<pre><code class="sh">#!/usr/bin/env sh
env=${RAILS_ENV:-development}
RAILS_ENV=${env} node -e "console.dir(require(\"./config/webpack/${env}\"), { depth: null })"
</code></pre>
<p>Then to run:</p>
<pre><code class="sh">$ chmod a+x ./bin/inspect_webpack
$ ./bin/inspect_webpack
# OR
$ RAILS_ENV=test ./bin/inspect_webpack
# OR
$ RAILS_ENV=production ./bin/inspect_webpack
</code></pre>
<h3 id="on-the-console" class="title title-h3">
<a name="on-the-console" class="anchor" href="#on-the-console"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
On the console
</h3>
<p>For an interactive experience, you can run <code>node</code> to pull up the Node.js REPL. This is especially helpful for isolating pieces of the webpack config "tree":</p>
<pre><code class="javascript">$ RAILS_ENV=development node
> const config = require('./config/webpack/development')
undefined
> console.dir(config, { depth: null })
{
mode: 'development',
output: {
// displays the entire webpack config
// ...
> console.dir(config.plugins, { depth: null })
// displays the plugins ...
// ...
</code></pre>
<p>As with the script I showed earlier, change the RAILS_ENV to inspect the configs for the other environments.</p>
<p>From the node console, you can also access and play around with the Webpack <code>environment</code> object directly:</p>
<pre><code class="javascript">> const { environment } = require('@rails/webpacker')
undefined
> environment.plugins.get('Manifest')
// displays configured WebpackAssetsManifest plugin
</code></pre>
<h3 id="debugging-with-devtools" class="title title-h3">
<a name="debugging-with-devtools" class="anchor" href="#debugging-with-devtools"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Debugging with DevTools
</h3>
<p>While the above examples help inspect the webpack config in a REPL, it may help to debug the config within the build process itself. It's possible to <a href="https://medium.com/@paul_irish/debugging-node-js-nightlies-with-chrome-devtools-7c4a1b95ae27" target="_blank" rel="noopener noreferrer">use the <code>debugger</code> provided by Chrome DevTools on a Node.js process</a> (as opposed to a browser's JavaScript process).</p>
<blockquote>
<p>For the following examples, I'm using Chrome Version 80.0.3987.163</p>
</blockquote>
<p>We could, for example, drop a <code>debugger;</code> statement into our Webpacker webpack config:</p>
<pre><code class="javascript">const { environment } = require('@rails/webpacker')
debugger
// changes I want to debug ...
module.exports = environment
</code></pre>
<p>We can then run the webpack build with the <code>--debug</code> flag:</p>
<pre><code class="sh">$ ./bin/webpack --debug
</code></pre>
<p>which is in the local development environment is equivalent to:</p>
<pre><code class="sh">$ RAILS_ENV=development node --inspect-brk yarn webpack --config ./config/webpack/development.js
</code></pre>
<p>Running the webpack process in debug mode will open up a websocket to communicate with Chrome DevTools:</p>
<pre><code class="sh">$ ./bin/webpack --debug
Debugger listening on ws://127.0.0.1:9229/861b81ed-6f2f....
For help, see: https://nodejs.org/en/docs/inspector
</code></pre>
<p>Visit <code>chrome://inspect</code> in your Chrome browser and we can find a link for our running Node process in the menu:
<img src="/assets/images/blog/webpack/chrome-inspect-main.png" alt="Screenshot of the chrome://inspect page" loading="lazy" /></p>
<p>This will start a instance of the DevTools for your Node process where we can click "Play" to resume execution:
<img src="/assets/images/blog/webpack/chrome-inspect-webpack-debug-1.png" alt="Screenshot of Chrome DevTools debugger start" loading="lazy" /></p>
<p>The process halts when it hits our <code>debugger</code> statement and we can modify values on the console:
<img src="/assets/images/blog/webpack/chrome-inspect-webpack-debug-2.png" alt="Screenshot of Chrome DevTools console" loading="lazy" /></p>
<p>For larger (or misconfigured) projects, you may experience memory usage issues with the webpack build. The DevTools debugger also provides a Memory tab for taking heap snapshots and tracking down the memory hogs in your build process.</p>
<p><img src="/assets/images/blog/webpack/chrome-inpsect-memory-tab.png" alt="Screenshot of DevTools Memory tab" loading="lazy" />
<img src="/assets/images/blog/webpack/chrome-inspect-heap-snapshot.png" alt="Screenshot of DevTools heap snapshot" loading="lazy" /></p>
<p>There's more on <a href="https://medium.com/webpack/webpack-bits-learn-and-debug-webpack-with-chrome-dev-tools-da1c5b19554" target="_blank" rel="noopener noreferrer">using DevTools with webpack on the webpack blog</a>.</p>
<h3 id="speed-measure-plugin" class="title title-h3">
<a name="speed-measure-plugin" class="anchor" href="#speed-measure-plugin"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Speed measure plugin
</h3>
<p>To help isolate slow parts of your build, I highly recommend the <a href="https://github.com/stephencookdev/speed-measure-webpack-plugin#readme" target="_blank" rel="noopener noreferrer">Speed Measure Plugin</a> for webpack. This is a plugin you would install and configure in your project temporarily to get feedback about individual parts of the build process.</p>
<p>First, install the plugin:</p>
<pre><code class="sh">yarn add speed-measure-webpack-plugin
</code></pre>
<p>Then temporarily configure your production build (you could also do something similar for development or test):</p>
<pre><code class="javascript">process.env.NODE_ENV = process.env.NODE_ENV || 'production'
const environment = require('./environment')
const SpeedMeasurePlugin = require('speed-measure-webpack-plugin')
const smp = new SpeedMeasurePlugin()
module.exports = smp.wrap(environment.toWebpackConfig())
</code></pre>
<p>And then run the production build:</p>
<pre><code class="sh">$ RAILS_ENV=production NODE_ENV=production ./bin/webpack
</code></pre>
<p>The Speed Measure plugin will print information to $stdout which may help identify the slow parts:</p>
<pre><code class="sh"> SMP ⏱
General output time took 3.094 secs
SMP ⏱ Plugins
CaseSensitivePathsPlugin took 0.391 secs
TerserPlugin took 0.306 secs
WebpackAssetsManifest took 0.066 secs
CompressionPlugin took 0.019 secs
MiniCssExtractPlugin took 0.001 secs
OptimizeCssAssetsWebpackPlugin took 0.001 secs
EnvironmentPlugin took 0 secs
SMP ⏱ Loaders
modules with no loaders took 1.27 secs
module count = 365
babel-loader took 0.824 secs
module count = 4
</code></pre>
<p>See <a href="https://dev.to/slashgear_/how-to-boost-the-speed-of-your-webpack-build-16h0" target="_blank" rel="noopener noreferrer">How to boost the speed of your webpack build</a> and the <a href="https://webpack.js.org/guides/build-performance/" target="_blank" rel="noopener noreferrer">official webpack build performance docs</a> for a number of useful tips for improving build/compilation performance.</p>
<h3 id="wrapping-up" class="title title-h3">
<a name="wrapping-up" class="anchor" href="#wrapping-up"> <svg aria-hidden="true" class="octicon octicon-link" height="16" version="1.1" viewBox="0 0 16 16" width="16">
<path d="M4 9h1v1h-1c-1.5 0-3-1.69-3-3.5s1.55-3.5 3-3.5h4c1.45 0 3 1.69 3 3.5 0 1.41-0.91 2.72-2 3.25v-1.16c0.58-0.45 1-1.27 1-2.09 0-1.28-1.02-2.5-2-2.5H4c-0.98 0-2 1.22-2 2.5s1 2.5 2 2.5z m9-3h-1v1h1c1 0 2 1.22 2 2.5s-1.02 2.5-2 2.5H9c-0.98 0-2-1.22-2-2.5 0-0.83 0.42-1.64 1-2.09v-1.16c-1.09 0.53-2 1.84-2 3.25 0 1.81 1.55 3.5 3 3.5h4c1.45 0 3-1.69 3-3.5s-1.5-3.5-3-3.5z"></path>
</svg>
</a>
Wrapping up
</h3>
<p>Even though Webpacker hides away much of the complexity of webpack configuration, sometimes it's necessary to peel back the abstraction layer. Like anything else that's new, wrapping your head around webpack build can be intimidating, especially if you don't know where to start. If things go wrong, all is not lost. Hopefully this post helped illustrate some ways you can get insight into what's happening in your Rails webpack config.</p>