> Hi Peter,
> Thank you for your feedback.
> Sometimes people ask questions you hadn't thought of, and it helps
> break through.
> I'll explain below.
>
> On Tue, Dec 18, 2018 at 9:57 PM Peter Robinson <pbrobinson(a)gmail.com> wrote:
> >
> > Hi Troy,
> >
> > Adding Jared so he sees it.
> >
> > I'm preceding all the bits below with the caveat that my knowledge on
> > nodejs dependencies is limited.
> >
> > > We've discussed my packaging of gateway [1] in our weekly IOT
> > > meetings. The decision was made to have it bundle all the
> > > dependencies, and that's what I've been trying to do. It looks
like
> > > there are over 1100 node modules required for this. (Nope, that
isn't
> > > a typo)
> >
> > Yes, I'm not surprised by that figure at all.
> >
> > Eventually I wonder if there's a tool where we can dig through these
> > to work out if any of the 1100 deps are duplicate packages with
> > different version requirements or even if they're unused but just
> > still hanging out in the package.json files and no longer used. Not a
> > priority now but something to think about, the more we can work with
> > upstream to minimise the deps over time the better for everyone's CVE
> > level sanity :)
> >
> > > The first problem is that these aren't all pure nodejs (javascript)
> > > packages. Several of them are binary. So that means I need to
> > > download the modules to bundle them, and then build them in koji. By
> > > using yarn, instead of npm install, I've been able to do that,
because
> > > yarn has an --offline mode.
> >
> > How many binary based nodejs dependencies are we talking here out of
> > the total of 1100 deps? Doing a quick query on the nodejs packages in
> > Fedora there's 23 out of 1868 that are architecture specific so I
> > suspect we have a handful of them? I think what we do here is take a
> > hybrid approach here and bundle the noarch dependencies and package
> > the architecture specific ones as standalone packages, that will
> > likely also make it easier to bump those in case of need of rebuild
> > (eg a soname bump in a dependent library). What's your thoughts on
> > that? Jared you thoughts?
> >
>
> This is a very good question (How many binary nodejs dependencies are there).
> When I looked at the packages I *thought* were binary ... they aren't.
> As I look at all of them, none of them are binary.
> I went back and re-read the README for installing gateway, and I
> realized that I had misread it. All those binaries they want are not
> for building any of the gateway, it's for running.
>
> In short, I was over-complicating this, as well as putting the
> Requires as BuildRequires, instead of the other way around.
>
> So ... trying this differently.
> Thanks for pointing me in this direction.
>
> Hopefully this will get things working.
>
So, I was wrong, but I'm still going to try your approach.
There are two binaries. Both of which are specifically listed in the
package.json.
nodejs-mysql3
nodejs-nanmsg
For nanomsg I have packaged up nng, which is the next gen of that, I'm
not 100% how backwards compatible, I did file this bug with moz-iot to
verify:
https://github.com/mozilla-iot/gateway/issues/1346
If nodejs-nanomsg doesn't work with nng we have two approaches,
package nanomsg up, file a bug with nodejs-nanomsg to support nng. I
suspect a comno
nodejs-mysql3 is already in Fedora.
nodejs-nanmsg is not in Fedora, but all of it's direct dependencies
(nodejs-nan and nodejs-bindings) are.
Yes, so I suggest we package it up separately then.
> So, I think we should be able to give bypassing them a try.
> Anyway ... progress
>
> Troy
>
> > > > Then there is/was the problem with babel-core. During the bundling
we
> > > > kept ending up with 6.25.0 I believe due to the package.json wanting
> > > > 7.0.0-beta.49. When it came to building in koji, yarn would freek
out
> > > > because it wanted to download a newer babel-core, but it was in
> > > > offline mode. (Even if it did download the new package, it would
> > > > still end up with 6.25.0). So in the end, I hacked package.json so
> > > > that it wants 7.2.2 of babel-core.
> > >
> > > Again we take the hybrid approach here and package that separately, or
> > > if there's an existing version of this work with the maintainer to get
> > > that to a version that works for all dependencies?
> > >
> > > > Ya ... so now it builds on x86_64 in koji.
> > > > But ... it now *only* builds on x86_64 in koji [2]
> > > > I'm putting this up in this email so others can join in the fun if
they want.
> > > > Here is the source rpm I have,[3] the one that build the scratch
build.
> > >
> > > Can you also push the .spec file.
> > >
> > > > Here is the script I use to make the bundled tarball.[4]
> > > > You need to have nodejs-yarn installed to run the tarball bundling
script.
> > >
> > > If we can take the hybrid "Bundle noarch dependencies, package
> > > separately the arch specific deps" does this get us away from needing
> > > yarn in the build environment? Can we just use yarn to create the
> > > noarch deps bundle locally?
> > >
> > > > I'm going to give this another shot, but any help is appreciated.
> > >
> > > I don't have a lot of cycles ATM but I can look and provide opinion :)
> > >
> > > > [1] -
https://github.com/mozilla-iot/gateway/
> > > > [2] -
https://koji.fedoraproject.org/koji/taskinfo?taskID=31522050
> > > > [3] -
https://tdawson.fedorapeople.org/gateway/gateway-0.6.1-1.fc29.src.rpm
> > > > [4] -
https://tdawson.fedorapeople.org/gateway/gateway-tarball.sh
> > > > _______________________________________________
> > > > IoT mailing list -- iot(a)lists.fedoraproject.org
> > > > To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> > > > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> > > _______________________________________________
> > > IoT mailing list -- iot(a)lists.fedoraproject.org
> > > To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> > > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> _______________________________________________
> IoT mailing list -- iot(a)lists.fedoraproject.org
> To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org