So, today things worked pretty well. I now have a "working" gateway
rpm. I put that in quotes because I haven't tested out the Requires
yet. According to the README we should have alot of dependencies
installed. But I don't know how many of those are for things that
aren't really in the gateway package, and instead are for the add-ons.
Anyway, what I have, I have put in my gateway directory. [5]
This included the gateway-0.6.1 and nodejs-nanomsg-4.0.2 packages.
gateway is noarch, nodejs-nanomsg is arch specific.
I have the spec files for both, and the gateway-tarball.sh, used to
created the tarball.
There is one part of the tarball that I'm debating putting into the
rpm spec file.
After we install node_modules, we run "webpack" which creates the
build directory and contents.
Everything in there is no-arch, so that part is fine, but I believe it
goes against the spirit of bundling. Which is that you just put
everything int he node_module directory and everything else happens
during the rpm build.
I need to test if we've got all the correct BuildRequires for
"webpack" to run, but I've just ran out of time. This is due to doing
real rpm builds take forever, due to the "provides" step. Remember we
have over 1000 modules in this package. So, that's something to do
when I (or someone else) has more time.
How to use/run gateway.
1 - (root) (Possibly optional) - install all the dependencies.
-- dnf -y group install "C Development Tools and Libraries"
-- dnf -y install autoconf pkgconfig curl libpng-devel git
boost-python2-devel boost-python3-devel boost-devel bluez-libs-devel
glib2-devel libudev-devel libusb1-devel nodejs python3-devel
2 - (root) Install gateway
-- dnf -y install
https://tdawson.fedorapeople.org/gateway/gateway-0.6.1-1.fc29.noarch.rpm
https://tdawson.fedorapeople.org/gateway/nodejs-nanomsg-4.0.2-1.fc29.x86_...
3 - (root) Setup setcap and firewall
setcap cap_net_raw+eip $(eval readlink -f `which node`)
setcap cap_net_raw+eip $(eval readlink -f `which python3`)
firewall-cmd --add-port=4443/tcp --permanent
firewall-cmd --add-port=8080/tcp --permanent
firewall-cmd --add-port=5353/udp --permanent
firewall-cmd --reload
4 - (user) (Optional but recommended) - Have it run locally instead of
globally, create self signed certs.
MOZIOT_HOME="${MOZIOT_HOME:=${HOME}/.mozilla-iot}"
SSL_DIR="${MOZIOT_HOME}/ssl"
[ ! -d "${SSL_DIR}" ] && mkdir -p "${SSL_DIR}"
openssl genrsa -out "${SSL_DIR}/privatekey.pem" 2048
openssl req -new -sha256 -key "${SSL_DIR}/privatekey.pem" -out
"${SSL_DIR}/csr.pem"
#answer questions
openssl x509 -req -in "${SSL_DIR}/csr.pem" -signkey
"${SSL_DIR}/privatekey.pem" -out "${SSL_DIR}/certificate.pem"
5 - (user) link the gateway module to your home area (or where you want it)
- Personally, I think this is a bug. the gateway module clearly is
not setup to be installed globally.
-- ln -s ln -s /usr/lib/node_modules/gateway ~/
6 - (user) Start gateway
-- cd ~/gateway/
-- node build/gateway.js
7 - (anyone) - Log into gateway via web
-- firefox
http://localhost:4443/
-- # If you are able to create a user and login, then things are working.
Note: You do not have any add-ons. The web interface will pretend
that they are installed, but if you look at the console, it will start
screaming about not being able to find the add-ons, meanwhile the web
page blissfully pretends it's loaded everything.
All of the add-ons are separate Mozilla things projects, and will need
to be installed (packaged) separately. Ya!!
Anyway, progress.
Troy
[5] -
https://tdawson.fedorapeople.org/gateway/
On Wed, Dec 19, 2018 at 5:31 PM Peter Robinson <pbrobinson(a)gmail.com> wrote:
>
> > > Hi Peter,
> > > Thank you for your feedback.
> > > Sometimes people ask questions you hadn't thought of, and it helps
> > > break through.
> > > I'll explain below.
> > >
> > > On Tue, Dec 18, 2018 at 9:57 PM Peter Robinson <pbrobinson(a)gmail.com>
wrote:
> > > >
> > > > Hi Troy,
> > > >
> > > > Adding Jared so he sees it.
> > > >
> > > > I'm preceding all the bits below with the caveat that my knowledge
on
> > > > nodejs dependencies is limited.
> > > >
> > > > > We've discussed my packaging of gateway [1] in our weekly
IOT
> > > > > meetings. The decision was made to have it bundle all the
> > > > > dependencies, and that's what I've been trying to do. It
looks like
> > > > > there are over 1100 node modules required for this. (Nope, that
isn't
> > > > > a typo)
> > > >
> > > > Yes, I'm not surprised by that figure at all.
> > > >
> > > > Eventually I wonder if there's a tool where we can dig through
these
> > > > to work out if any of the 1100 deps are duplicate packages with
> > > > different version requirements or even if they're unused but just
> > > > still hanging out in the package.json files and no longer used. Not a
> > > > priority now but something to think about, the more we can work with
> > > > upstream to minimise the deps over time the better for everyone's
CVE
> > > > level sanity :)
> > > >
> > > > > The first problem is that these aren't all pure nodejs
(javascript)
> > > > > packages. Several of them are binary. So that means I need to
> > > > > download the modules to bundle them, and then build them in koji.
By
> > > > > using yarn, instead of npm install, I've been able to do
that, because
> > > > > yarn has an --offline mode.
> > > >
> > > > How many binary based nodejs dependencies are we talking here out of
> > > > the total of 1100 deps? Doing a quick query on the nodejs packages in
> > > > Fedora there's 23 out of 1868 that are architecture specific so I
> > > > suspect we have a handful of them? I think what we do here is take a
> > > > hybrid approach here and bundle the noarch dependencies and package
> > > > the architecture specific ones as standalone packages, that will
> > > > likely also make it easier to bump those in case of need of rebuild
> > > > (eg a soname bump in a dependent library). What's your thoughts
on
> > > > that? Jared you thoughts?
> > > >
> > >
> > > This is a very good question (How many binary nodejs dependencies are
there).
> > > When I looked at the packages I *thought* were binary ... they aren't.
> > > As I look at all of them, none of them are binary.
> > > I went back and re-read the README for installing gateway, and I
> > > realized that I had misread it. All those binaries they want are not
> > > for building any of the gateway, it's for running.
> > >
> > > In short, I was over-complicating this, as well as putting the
> > > Requires as BuildRequires, instead of the other way around.
> > >
> > > So ... trying this differently.
> > > Thanks for pointing me in this direction.
> > >
> > > Hopefully this will get things working.
> > >
> >
> > So, I was wrong, but I'm still going to try your approach.
> > There are two binaries. Both of which are specifically listed in the
> > package.json.
> > nodejs-mysql3
> > nodejs-nanmsg
>
> For nanomsg I have packaged up nng, which is the next gen of that, I'm
> not 100% how backwards compatible, I did file this bug with moz-iot to
> verify:
>
https://github.com/mozilla-iot/gateway/issues/1346
>
> If nodejs-nanomsg doesn't work with nng we have two approaches,
> package nanomsg up, file a bug with nodejs-nanomsg to support nng. I
> suspect a comno
>
> > nodejs-mysql3 is already in Fedora.
> > nodejs-nanmsg is not in Fedora, but all of it's direct dependencies
> > (nodejs-nan and nodejs-bindings) are.
>
> Yes, so I suggest we package it up separately then.
>
> > So, I think we should be able to give bypassing them a try.
> > Anyway ... progress
> >
> > Troy
> >
> > > > > Then there is/was the problem with babel-core. During the
bundling we
> > > > > kept ending up with 6.25.0 I believe due to the package.json
wanting
> > > > > 7.0.0-beta.49. When it came to building in koji, yarn would
freek out
> > > > > because it wanted to download a newer babel-core, but it was in
> > > > > offline mode. (Even if it did download the new package, it
would
> > > > > still end up with 6.25.0). So in the end, I hacked package.json
so
> > > > > that it wants 7.2.2 of babel-core.
> > > >
> > > > Again we take the hybrid approach here and package that separately,
or
> > > > if there's an existing version of this work with the maintainer to
get
> > > > that to a version that works for all dependencies?
> > > >
> > > > > Ya ... so now it builds on x86_64 in koji.
> > > > > But ... it now *only* builds on x86_64 in koji [2]
> > > > > I'm putting this up in this email so others can join in the
fun if they want.
> > > > > Here is the source rpm I have,[3] the one that build the scratch
build.
> > > >
> > > > Can you also push the .spec file.
> > > >
> > > > > Here is the script I use to make the bundled tarball.[4]
> > > > > You need to have nodejs-yarn installed to run the tarball
bundling script.
> > > >
> > > > If we can take the hybrid "Bundle noarch dependencies, package
> > > > separately the arch specific deps" does this get us away from
needing
> > > > yarn in the build environment? Can we just use yarn to create the
> > > > noarch deps bundle locally?
> > > >
> > > > > I'm going to give this another shot, but any help is
appreciated.
> > > >
> > > > I don't have a lot of cycles ATM but I can look and provide
opinion :)
> > > >
> > > > > [1] -
https://github.com/mozilla-iot/gateway/
> > > > > [2] -
https://koji.fedoraproject.org/koji/taskinfo?taskID=31522050
> > > > > [3] -
https://tdawson.fedorapeople.org/gateway/gateway-0.6.1-1.fc29.src.rpm
> > > > > [4] -
https://tdawson.fedorapeople.org/gateway/gateway-tarball.sh
> > > > > _______________________________________________
> > > > > IoT mailing list -- iot(a)lists.fedoraproject.org
> > > > > To unsubscribe send an email to
iot-leave(a)lists.fedoraproject.org
> > > > > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > > > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> > > > _______________________________________________
> > > > IoT mailing list -- iot(a)lists.fedoraproject.org
> > > > To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> > > > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> > _______________________________________________
> > IoT mailing list -- iot(a)lists.fedoraproject.org
> > To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org
> _______________________________________________
> IoT mailing list -- iot(a)lists.fedoraproject.org
> To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org