Hi Peter,
Thank you for your feedback.
Sometimes people ask questions you hadn't thought of, and it helps
break through.
I'll explain below.
On Tue, Dec 18, 2018 at 9:57 PM Peter Robinson <pbrobinson(a)gmail.com> wrote:
Hi Troy,
Adding Jared so he sees it.
I'm preceding all the bits below with the caveat that my knowledge on
nodejs dependencies is limited.
> We've discussed my packaging of gateway [1] in our weekly IOT
> meetings. The decision was made to have it bundle all the
> dependencies, and that's what I've been trying to do. It looks like
> there are over 1100 node modules required for this. (Nope, that isn't
> a typo)
Yes, I'm not surprised by that figure at all.
Eventually I wonder if there's a tool where we can dig through these
to work out if any of the 1100 deps are duplicate packages with
different version requirements or even if they're unused but just
still hanging out in the package.json files and no longer used. Not a
priority now but something to think about, the more we can work with
upstream to minimise the deps over time the better for everyone's CVE
level sanity :)
> The first problem is that these aren't all pure nodejs (javascript)
> packages. Several of them are binary. So that means I need to
> download the modules to bundle them, and then build them in koji. By
> using yarn, instead of npm install, I've been able to do that, because
> yarn has an --offline mode.
How many binary based nodejs dependencies are we talking here out of
the total of 1100 deps? Doing a quick query on the nodejs packages in
Fedora there's 23 out of 1868 that are architecture specific so I
suspect we have a handful of them? I think what we do here is take a
hybrid approach here and bundle the noarch dependencies and package
the architecture specific ones as standalone packages, that will
likely also make it easier to bump those in case of need of rebuild
(eg a soname bump in a dependent library). What's your thoughts on
that? Jared you thoughts?
This is a very good question (How many binary nodejs dependencies are there).
When I looked at the packages I *thought* were binary ... they aren't.
As I look at all of them, none of them are binary.
I went back and re-read the README for installing gateway, and I
realized that I had misread it. All those binaries they want are not
for building any of the gateway, it's for running.
In short, I was over-complicating this, as well as putting the
Requires as BuildRequires, instead of the other way around.
So ... trying this differently.
Thanks for pointing me in this direction.
Hopefully this will get things working.
> > Then there is/was the problem with babel-core. During the bundling we
> > kept ending up with 6.25.0 I believe due to the package.json wanting
> > 7.0.0-beta.49. When it came to building in koji, yarn would freek out
> > because it wanted to download a newer babel-core, but it was in
> > offline mode. (Even if it did download the new package, it would
> > still end up with 6.25.0). So in the end, I hacked package.json so
> > that it wants 7.2.2 of babel-core.
>
> Again we take the hybrid approach here and package that separately, or
> if there's an existing version of this work with the maintainer to get
> that to a version that works for all dependencies?
>
> > Ya ... so now it builds on x86_64 in koji.
> > But ... it now *only* builds on x86_64 in koji [2]
> > I'm putting this up in this email so others can join in the fun if they
want.
> > Here is the source rpm I have,[3] the one that build the scratch build.
>
> Can you also push the .spec file.
>
> > Here is the script I use to make the bundled tarball.[4]
> > You need to have nodejs-yarn installed to run the tarball bundling script.
>
> If we can take the hybrid "Bundle noarch dependencies, package
> separately the arch specific deps" does this get us away from needing
> yarn in the build environment? Can we just use yarn to create the
> noarch deps bundle locally?
>
> > I'm going to give this another shot, but any help is appreciated.
>
> I don't have a lot of cycles ATM but I can look and provide opinion :)
>
> > [1] -
https://github.com/mozilla-iot/gateway/
> > [2] -
https://koji.fedoraproject.org/koji/taskinfo?taskID=31522050
> > [3] -
https://tdawson.fedorapeople.org/gateway/gateway-0.6.1-1.fc29.src.rpm
> > [4] -
https://tdawson.fedorapeople.org/gateway/gateway-tarball.sh
> > _______________________________________________
> > IoT mailing list -- iot(a)lists.fedoraproject.org
> > To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> _______________________________________________
> IoT mailing list -- iot(a)lists.fedoraproject.org
> To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org