Hi all,
I've been experimenting with adding some macros to make OCaml packaging a bit easier. You can see what I have so far here:
https://src.fedoraproject.org/fork/jjames/rpms/ocaml-srpm-macros/c/9554c8095...
With these macros, the main part of a spec file for a package that builds with dune can be as simple as this:
``` %build %dune_build
%install %dune_install
%check %dune_check
%files -f .ofiles %doc README.md %license LICENSE.md
%files devel -f .ofiles-devel ```
Packages that do not use dune to build can still use the %ocaml_files macro at the end of %install to generate .ofiles and .ofiles-devel files to pass to %files.
New macros: - %ocamldir: shorter replacement for %{_libdir}/ocaml - %odoc_package: inspired by %javadoc_package, everything you need in one place to declare a package that contains odoc documentation. - %relink_stublibs: automatically locate any dll*.so files created by ocamlmklib and relink them with Fedora linker flags - %_add_release_func: internal macro to add `--release` to flags passed to %dune_build or %dune_check if neither `--release` nor `-p` has been given. I should probably rewrite it in lua. - %dune_build: invoke `dune build` with a standard set of flags, followed by %relink_stublibs. Arguments can be passed to dune, e.g., `%dune_build @install`. - %dune_check: invoke `dune runtest` with a standard set of flags. Arguments can be passed to dune, e.g., `%dune_check -p foobar`. - %ocaml_files: generate .ofiles* files to pass to %files from files installed in the buildroot. Use of this macro requires python3 to be installed in the chroot. By default, files are split into a main package (.ofiles) and a devel package (.ofiles-devel). Use `%ocaml_files -n` to ump all files together into .ofiles. Use `%ocaml_files -s` to generate files with the OCaml module name embedded in them; e.g., .ofiles-module1, .ofiles-module1-devel, .ofiles-module2, .ofiles-module2-devel, etc. - %dune_install: invoke `dune install` with a standard set of flags, then: - Remove any .dune-keep markers left over from running odoc - Remove anything installed into /usr/doc - Remove all *.ml files that have matching *.mli files installed - Invoke %ocaml_files, with the same -s or -n options passed to %dune_install itself
I would appreciate any feedback you can offer. My questions:
1. Is ocaml-srpm-macros the best place for this? 2. Is the python dependency for %ocaml_files / %dune_install okay? I thought about writing it in lua to avoid the external dependency, but then the entire %install script has to be written in lua. If the entire %install script is just %dune_install, then that's okay, but it seems like a significant barrier otherwise. 3. Can you think of any other macros that would be useful? 4. I wonder if it would be useful to allow overrides for the logic used by %ocaml_files to split files into the main and devel packages. Perhaps something like this: -d file1,file2,...: comma-separated list of files to put in the devel package, even if they would normally go in the main package -m file1,file2,...: comma-separated list of files to put in the main package, even if they would normally go in the devel package Too bad for you if you want to do this to a file with a comma in its name. :-) Of course, one can always do surgery on the .ofiles* files after they are created, so maybe this would be overkill.
I was working on making a bunch of spec files for everyone to look at when the package notes feature derailed me. Sigh. I'm going to get that cleaned up first, then I hope to get back to making examples.
Regards,
On Wed, Feb 2, 2022 at 2:09 PM Jerry James loganjerry@gmail.com wrote:
I've been experimenting with adding some macros to make OCaml packaging a bit easier. You can see what I have so far here:
https://src.fedoraproject.org/fork/jjames/rpms/ocaml-srpm-macros/c/9554c8095...
I updated a few things. New URL:
https://src.fedoraproject.org/fork/jjames/rpms/ocaml-srpm-macros/c/29b9fd076...
This update changes %dune_add_release to a lua macro, which allows some simplifications in its use. I also turned off RPM option processing for %dune_build and %dune_check so that all options are passed to dune, without needing -- to separate options and arguments. However, -- will still be necessary if you want to pass any options to dune via %dune_install.
Finally, it is now possible (and meaningful) to combine the -n and -s %dune_install options. I actually encountered a package where I wanted to do that.
Once the package-notes issues have been dealt with, I'll create a COPR repo to show off what these macros can do.
On Wed, Feb 02, 2022 at 02:09:48PM -0700, Jerry James wrote:
I would appreciate any feedback you can offer. My questions:
- Is ocaml-srpm-macros the best place for this?
I believe that long term it'd be more ideal to try to get them upstream (in rpm) and to share them with SuSE. However in the short term / while road-testing them, ocaml-srpm-macros sounds fine.
- Is the python dependency for %ocaml_files / %dune_install okay? I thought about writing it in lua to avoid the external dependency, but then the entire %install script has to be written in lua. If the entire %install script is just %dune_install, then that's okay, but it seems like a significant barrier otherwise.
It's slightly unfortunate, but in practice you need to have python installed to do any serious Fedora packaging so I doubt it'll make any practical difference.
Rich.
On Fri, Feb 4, 2022 at 3:16 PM Richard W.M. Jones rjones@redhat.com wrote:
I believe that long term it'd be more ideal to try to get them upstream (in rpm) and to share them with SuSE. However in the short term / while road-testing them, ocaml-srpm-macros sounds fine.
Okay, I am still finding myself needing to tweak the macros here and there, so I'm going to leave them where they are until I feel like they are ready to share more broadly.
It's slightly unfortunate, but in practice you need to have python installed to do any serious Fedora packaging so I doubt it'll make any practical difference.
That's why I chose python in the first place. It is unfortunate. Would it make sense to add "Requires: python3" to the ocaml-dune package so that "BuildRequires: python3" doesn't have to be added to every single dune-using spec file (on the assumption that everybody will want to use these macros :-) )?
On Wed, Feb 09, 2022 at 08:41:06AM -0700, Jerry James wrote:
On Fri, Feb 4, 2022 at 3:16 PM Richard W.M. Jones rjones@redhat.com wrote:
I believe that long term it'd be more ideal to try to get them upstream (in rpm) and to share them with SuSE. However in the short term / while road-testing them, ocaml-srpm-macros sounds fine.
Okay, I am still finding myself needing to tweak the macros here and there, so I'm going to leave them where they are until I feel like they are ready to share more broadly.
It's slightly unfortunate, but in practice you need to have python installed to do any serious Fedora packaging so I doubt it'll make any practical difference.
That's why I chose python in the first place. It is unfortunate. Would it make sense to add "Requires: python3" to the ocaml-dune package so that "BuildRequires: python3" doesn't have to be added to every single dune-using spec file (on the assumption that everybody will want to use these macros :-) )?
My understanding is that because these are dune macros you'd have to have BR: ocaml-dune in the spec file using the macros; and because the macros need python3 this "Requires: python3" dependency (in ocaml-dune) would ensure that python3 is installed so the macros can run.
So yes, that sounds right.
Rich.
On Wed, Feb 9, 2022 at 9:16 AM Richard W.M. Jones rjones@redhat.com wrote:
My understanding is that because these are dune macros you'd have to have BR: ocaml-dune in the spec file using the macros; and because the macros need python3 this "Requires: python3" dependency (in ocaml-dune) would ensure that python3 is installed so the macros can run.
So yes, that sounds right.
All you OCaml packagers, would you mind taking a look at the following?
PR for the ocaml-srpm-macros package to implement the above-mentioned macros: https://src.fedoraproject.org/rpms/ocaml-srpm-macros/pull-request/1
Changes to the OCaml guidelines to discuss the macros, along with general modernization: https://pagure.io/fork/jjames/packaging-committee/c/814e5e9a951e463c1571e1fa...
Thank you!
Hi Jerry,
Jerry James loganjerry@gmail.com writes:
Hi all,
I've been experimenting with adding some macros to make OCaml packaging a bit easier. You can see what I have so far here:
https://src.fedoraproject.org/fork/jjames/rpms/ocaml-srpm-macros/c/9554c8095...
With these macros, the main part of a spec file for a package that builds with dune can be as simple as this:
%build %dune_build %install %dune_install %check %dune_check %files -f .ofiles %doc README.md %license LICENSE.md %files devel -f .ofiles-devel
I have just tried to use these new macros with Marián to package tiny_httpd which is one of the dependencies of google-drive-ocamlfuse.
Unfortunately we ran into a bit of a pickle, because tiny_httpd provides two packages: tiny_httpd itself and tiny_httpd_camlzip. We only need the former and thus tried to only build it via %dune_build -p tiny_httpd. That part works, but the installation step unfortunately does not. Invoking a full %dune_build fails with tiny_httpd_camlzip not being build and just installing tiny_httpd via:
%dune_build -p tiny_httpd
appears to be unsupported: --8<---------------cut here---------------start------------->8--- dune_install: invalid option -- 'p' error: /builddir/build/SPECS/ocaml-tiny_httpd.spec: line 41: Unknown option p in dune_install(sn) 1< (%) 0< (%dune_install) --8<---------------cut here---------------end--------------->8---
Otherwise, these macros are absolutely terrific and make ocaml packaging much, much easier!
Cheers,
Dan
P.S.: Full spec of tiny_httpd is attached
Adding Jose on this thread per his request.
We have just had a meeting with Dan just before Jose sent his email...
I took a notice on how builds are done in the ocaml ecosystem and I can tell that there are some things we have solved in our (Java) ecosystem. Including maven bootstrapping.
Usually adding a new Java package is really trivial as all the automation is done by `javapackages-tool`.
The spec files with ocaml-srpm-macros look quite similar to how we handle maven builds.
On 29. 6. 2022 15:59, Dan Čermák wrote:
Hi Jerry,
Jerry James loganjerry@gmail.com writes:
Hi all,
I've been experimenting with adding some macros to make OCaml packaging a bit easier. You can see what I have so far here:
https://src.fedoraproject.org/fork/jjames/rpms/ocaml-srpm-macros/c/9554c8095...
With these macros, the main part of a spec file for a package that builds with dune can be as simple as this:
%build %dune_build %install %dune_install %check %dune_check %files -f .ofiles %doc README.md %license LICENSE.md %files devel -f .ofiles-devel
I have just tried to use these new macros with Marián to package tiny_httpd which is one of the dependencies of google-drive-ocamlfuse.
Unfortunately we ran into a bit of a pickle, because tiny_httpd provides two packages: tiny_httpd itself and tiny_httpd_camlzip. We only need the former and thus tried to only build it via %dune_build -p tiny_httpd. That part works, but the installation step unfortunately does not. Invoking a full %dune_build fails with tiny_httpd_camlzip not being build and just installing tiny_httpd via:
%dune_build -p tiny_httpd
appears to be unsupported: --8<---------------cut here---------------start------------->8--- dune_install: invalid option -- 'p' error: /builddir/build/SPECS/ocaml-tiny_httpd.spec: line 41: Unknown option p in dune_install(sn) 1< (%) 0< (%dune_install) --8<---------------cut here---------------end--------------->8---
Otherwise, these macros are absolutely terrific and make ocaml packaging much, much easier!
Cheers,
Dan
P.S.: Full spec of tiny_httpd is attached
Hi Marián,
On Wed, Jun 29, 2022 at 8:20 AM Marián Konček mkoncek@redhat.com wrote:
We have just had a meeting with Dan just before Jose sent his email...
I took a notice on how builds are done in the ocaml ecosystem and I can tell that there are some things we have solved in our (Java) ecosystem. Including maven bootstrapping.
Usually adding a new Java package is really trivial as all the automation is done by `javapackages-tool`.
The spec files with ocaml-srpm-macros look quite similar to how we handle maven builds.
I would be very happy to receive suggestions on how to further improve the OCaml packaging experience, especially from those who have fought this battle before in another ecosystem. If you notice anything at all that can be improved, please let me know.
Incidentally, we should update ocaml-dune to 3.x soon, which should smooth away a couple of rough spots we currently have to deal with.
Hi Dan,
On Wed, Jun 29, 2022 at 7:59 AM Dan Čermák dan.cermak@cgc-instruments.com wrote:
I have just tried to use these new macros with Marián to package tiny_httpd which is one of the dependencies of google-drive-ocamlfuse.
Unfortunately we ran into a bit of a pickle, because tiny_httpd provides two packages: tiny_httpd itself and tiny_httpd_camlzip. We only need the former and thus tried to only build it via %dune_build -p tiny_httpd. That part works, but the installation step unfortunately does not.
You have run across something that makes me mildly unhappy. Suggestions on how to solve it are welcome. The problem is that the %dune_install macro is defined in lua, and has two options: -s and -n. These are processed by the macro itself. This means that to pass options to dune, such as -p, you have to use the -- separator, making it different from %dune_build and %dune_check. Anyway, for your case, try this:
%dune_install -- -p tiny_httpd
Regards,
ocaml-devel@lists.fedoraproject.org