v2.15.0 fails to install entirely

[ang-zeyu]

Tell us about your environment

• MarkBind Version: the next (unpublished) release

What actually happened? Please include the actual, raw output.

What did you do? Please include the actual source code causing the issue.

Suspected cause: we moved towards using relative file specifiers in package.json (@markbind/core: "file:packages/core") since the last release. I've confirmed that subpackage package.jsons are indeed published, however, when doing npm install -g, npm attempts to resolve the root package.json's dependencies first without having downloaded the entire package (including the subpackage's package.json). This causes the above error.

Attempted Solutions (from posts and articles)

• npm cache clean --force + npm cache verify - dosen't work, and users shouldnt have to do this to upgrade.
• using npm shrinkwrap, which is essentially a publishable package-lock.json. This causes a maximum call stack size exceeded error with a not very useful debug log either.

Proposed solution

We opted for the current structure (a root markbind-cli package with 2 subpackages) for reasons of simplicity for now. From searching up similar issues, the working solutions all tended towards:

_private_ (unpublished) root package.json managed by lerna / yarn workspaces (simply a meta file, possibly containing dev dependencies, but not actually a package)
packages (all published independently)
  - package A
  - package B
  - package C
  - ...

which is also the structure many well known projects use (babel, react, gatsby, ...).
The devtools often used here are lerna (+ yarn workspaces).

Solution: Let's move towards this more conventional structure, and publish each package independently and separately using lerna as a high level helper (yarn workspaces can come later if really needed).
This also paves the way for future packages #1248 (comment).

1. Move markbind-cli into packages/cli (but retain package naming for "backwards naming compatibility") (PR 1)
2. setup lerna, edit the devguide sections accordingly (PR 2)
3. publish

Reservations

It's a little strange for package.jsons to allow relative file:... specifiers but have them break when published. I haven't been able to find much on this (most solutions just tended towards lerna + yarn) so far either.

Likely, they are only intended to be used in development environments, but maybe I just haven't come across the right post / article. e.g. lerna uses this to hoist dependencies to the root in one of it's configurations. (lerna link convert) Would be nice if anyone has more relevant info on this.

[ang-zeyu]

@acjh @marvinchin any thoughts? :o

[marvinchin]

I'm not super experienced with scoped packages, so the following is just a guess.

I wonder if the @markbind scope is giving us issues here. According to https://docs.npmjs.com/misc/scope,

Each npm user/organization has their own scope, and only you can add packages in your scope.

markbind seems to be an npm org (https://www.npmjs.com/org/markbind) but we aren't inside it, so perhaps that might explain why we aren't able to publish to it.

I wonder if we can just drop the @markbind scope and just name the sub-package markbind-core. I can dig into this a little more tomorrow but these are some initial thoughts for now!

[ang-zeyu]

markbind seems to be an npm org (https://www.npmjs.com/org/markbind) but we aren't inside it, so perhaps that might explain why we aren't able to publish to it.

The publishing works but installing the published package fails (tried on another machine as well).

To clarify, we've never published the vue components / core package independently as of yet, if that's what you meant. (it's part of my suggestion above though)

I also just tried omitting the scope of the subpackage in my test package though, still dosen't work =X

[marvinchin]

Seems like this is a known issue: npm/npm#18266

Also, https://docs.npmjs.com/files/package.json#local-paths also suggests that

This feature is helpful for local offline development and creating tests that require npm installing where you don’t want to hit an external server, but should not be used when publishing packages to the public registry.
So maybe the relative file specifier isn't meant to be used the way we are attempting to use it.

I think if the goal is to keep all markbind packages in the same repo, then moving towards your proposed solution is a good idea. I personally like having them in the same repo because it streamlines the development process (and the two packages are so tightly coupled you really can't develop on one without the other anyway). @acjh do you have any thoughts on whether we should move ahead with this, or revert back to the old structure?

[ang-zeyu]

Thanks for investigating this! @marvinchin