The following is a story of trials, tribulations, and triumphs, and stands as a testament to two young men’s ability to say “package” over a hundred times without giggling.
Earlier this week, Grady attempted to install this package and EVERYTHING BROKE.
On Mustafa’s machine:
$ node --version V0.12.10 $ npm --version 2.14.9 $ vena_tool --version 1.2.17
BUT ON GRADY’S MACHINE:
$ node --version V0.12.10 $ npm --version 2.14.9 $ vena_tool --version SyntaxError: Use of const in strict mode.
“Alright,” we thought, “time for the nuclear option.” We reinstalled everything from scratch.
Let’s try that again.
$ node --version V0.12.10
$ npm --version 2.14.9
Story checks out.
$ vena_tool --version SyntaxError: Use of const in strict mode.
Go home Node, you’re drunk.
Now here’s where things get interesting. Notice this error message:
One of our dependencies,
unirest, was having problems, so we checked the repo. The latest commit was almost two years ago…
H’okay. So we checked its dependency, request. Still no recent changes.
Descending further into the madness, we looked at its dependency,
tough-cookie. Lots of recent changes.
Looking at its dependencies, we see that one of those dependencies
ip-regex, recently stopped supporting versions of Node lower than 8.0.
Now, the current stable version of Node is 11.7.0.
We, however, are tied to Node version 0.12.10.
In mathematical terms, that would mean we were behind by two orders of magnitude.
So, to recap:
Our tool requires
unirest at version 0.4.0.
request at version ~2.51.0.
tough-cookie at version >=0.12.0
In the ip-regex README, they say:
This module targets Node.js 8 or later…
Now normally when adding a dependency you would use the ~ operator, which means use “approximately” this version. That’s just good manners.
The reason for this is simple:
In version numbering, standard practice is to mark incremental patches by changing the trailing digits (eg. 2.5.1 becomes 2.5.2).
More substantial changes (known as “minor” changes) are generally reflected in the second set of digits (eg. 2.5.2 becomes 2.6.0).
Finally, major changes to the API (also known as “breaking” changes) are indicated by the first set of digits (eg. 2.6.0 becomes 3.0.0).
Systems like this are what separate us from the animals. They are civilization. Without them we are just… monsters.
So when you see something like this:
… it tends to angry up the blood.
In human speak, “>=0.12.0” means “use any version of tough-cookie 0.12.0 or greater.”
Well, version 3.0.0 is definitely greater than 0.12.0.
What’s not particularly great is that because this happens three layers deep in our dependencies, fixing it is not a simple matter of specifying exactly the version we want. Because we were already doing that.
Our initial approach to solving this was to take the
node_modules folder off of Mustafa’s computer, put it on Grady’s computer, and Grady’s problem was solved. He could continue to use our internal tool, no problem.
But… that left this problem open for anyone who would try to install this tool in the future. So our “future proof” solution was simply to commit the entire contents of
node_modules into the git repo for this tool. The problem with that is, well,
This is because
node_modules contains all of our dependencies’ source code. Which includes their
node_modules folder was HUGE. This is what the initial pull request looked like:
Yup, you read that right. 1.5 MILLION lines of code changed, and almost 11 THOUSAND files added.
And a single line removed, in the first commit:
This PR got a few… choice… reactions from our fellow coworkers.
Yep, it’s absolute megaton of a PR.
But then, other comments came in that pointed us to the right spot.
Of course, we have to be careful:
As it turns out, the solution is to use a little piece of genius called
npm shrinkwrap to create a “publishable lockfile” (which it stores at
npm-shrinkwrap.json). This lockfile “describes an exact, and more importantly reproducible” dependency tree, which means that future installations will reproduce the exact same module structure whenever possible.
This means that we don’t need to commit the ENTIRE
node_modules folder, because
npm-shrinkwrap.json describes that folder structure exactly. When someone does an
npm will use the exact version numbers in
npm-shrinkwrap.json instead of doing the default version resolution, so we can specify that we need to download, say, version 2.3.3 of
tough-cookie instead of
npm fetching version 3.0.0.
To the author of this beautiful little tool, we would like to buy you a drink. Until then, we’ll just be here googling