Webpack and yarn magic against duplicates in bundles

Setting up the scene

The problem of duplicated dependencies

Deduplication in yarn

modal-dialog@^3.0.0:
version "3.0.0"
resolved "exact-link-to-where-download-modal-dialog-3.0.0-from"
dependencies:
button@^2.4.1

button@^2.5.0:
version "2.5.0"
resolved "exact-link-to-where-download-2.5.0-version-from"

button@^2.4.1:
version "2.4.1"
resolved "exact-link-to-where-download-2.4.1-version-from"
modal-dialog@^3.0.0:
version "3.0.0"
resolved "exact-link-to-where-download-modal-dialog-3.0.0-from"
dependencies:
button@^2.4.1

button@^2.4.1, button@^2.5.0:
version "2.5.0"
resolved "exact-link-to-where-download-2.5.0-version-from"

Deduplication in yarn — not compatible version

modal-dialog@^3.0.0:
version "3.0.0"
resolved "exact-link-to-where-download-modal-dialog-3.0.0-from"
dependencies:
button@^1.0.0

editor@^5000.0.0:
version "5000.0.0"
resolved "exact-link-to-where-download-editor-5000.0.0-from"
dependencies:
button@^1.3.0

button@^2.5.0:
version "2.5.0"
resolved "exact-link-to-where-download-2.5.0-version-from"

button@^1.0.0, button@^1.3.0:
version "1.3.0"
resolved "exact-link-to-where-download-1.3.0-version-from"

Duplicated dependencies install

/node_modules
/editor
/modal-dialog
/tooltip

Duplicated dependencies and webpack

  • project/node_modules/editor/node_modules/button/index.js — when it’s requested from within editor
  • project/node_modules/modal-dialog/node_modules/button/index.js — when it’s requested from within modal-dialog

Deduplication in webpack — first attempt

Example of the real duplicates in Jira
Minimal implementation of the deduplication logic

Deduplication in webpack — actual solution

Findings and other curiosities

The non-deterministic behaviour is reproducible non-deterministically

NormalModuleReplacementPlugin was not built for the purpose.

{
request: "./styled",
context: "/project/node_modules/editor/node_modules/button"
}

Nevertheless, NormalModuleReplacementPlugin can actually be used if there is a need

{
request: "./styled",
context: "/project/node_modules/editor/node_modules/button"
}
{
request: "/project/node_modules/modal-dialog/node_modules/button/styled",
context: "/project/node_modules/editor/node_modules/button"
}

“naive” replace (string.startsWith) is not going to work

  • button@2.x and icon@3.x at the root
  • editor has button@1.x as a transitive dependency, which has icon@1.x as a transitive dependency
  • modal-dialog has button@1.x as a transitive dependency, which has icon@1.x as a transitive dependency PLUS it has icon@2.x as its own transitive dependency (those who survived and read til this moment — kudos to you, you’re heroes)
/node_modules
/editor
/node_modules
/button-1.3.0
/icon-1.0.0 // on the same level as button above
/modal-dialog
/node_modules
/button-1.3.0
/node_modules
/icons-1.0.0 // nested within button since on the lvl above there is another icon
/icon-2.0.0
/button-2.5.0
/project/node_modules/editor/node_modules/icon-1.0.0
/project/node_modules/modal_dialog/node_modules/button-1.3.0/node_modules/icon-1.0.0
 /project/node_modules/modal_dialog/node_modules/button-1.3.0
 /project/node_modules/editor/node_modules/button-1.3.0
/project/node_modules/editor/node_modules/button-1.3.0/node_modules/icon-1.0.0

The end

  • non-deterministic order of hooks on the webpack side
  • “naive” initial strings replacement
  • listening and replacing only “request” in the initial implementation
  • a few other edge cases that are not mentioned here and that caused not all modules to be resolved

--

--

--

Frontend architect, coder. Love solving problems, fixing things and writing in-depth tech articles: https://www.developerway.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Create react app error - error An unexpected error occurred: yarn.cmd

Publishing a Real NodeJS Package: Project Bootstrap

Publishing a real NodeJS package: Project Bootstrap

10 Important Interview Question of Javascript

useDApp update ‘22

Node Weekly Issue 17

//platform.twitter.com/widgets.js from Twitter https://twitter.com/UKCareGuide

For Front-end Development Beginners

WordPress-Style Shortcodes Using Vue.js

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nadia Makarevich

Nadia Makarevich

Frontend architect, coder. Love solving problems, fixing things and writing in-depth tech articles: https://www.developerway.com

More from Medium

A more granular useEffect

Write lean state-management

3 Ways of Passing Multiple Parameters to the onClick Handler in React

Improve JavaScript Performance with Tree Shaking