On Mon, Aug 29, 2016 at 10:38:46AM +0200, Lukas Slebodnik wrote:
On (29/08/16 07:09), Fabiano Fidêncio wrote:
>Hoiwdy!
>
>
>On Fri, Aug 19, 2016 at 1:08 AM, Fabiano Fidêncio <fidencio(a)redhat.com> wrote:
>> This patch is a first attempt to make "make intgcheck" less
>> painful/time consuming than it is now.
>>
>> Although the patch provides a good improvement on having 5 new
>> targets, I know it's still not ideal. The ideal case, IMO, would be
>> being able to select which subset of tests would be run, but that's an
>> improvement that can be done later on.
>>
>> Tips are welcome.
>>
>> Best Regards,
>
>After a few dicussions and some tests done when we met personally, I
>found out that the patch was broken.
>Taking Lukaš suggestion (almost) I've merged the -prepare and
>-configure parts and re-worked the way we get the prefix.
>
>Now it seems to be working!
>
>Best Regards,
>--
>Fabiano Fidêncio
>From a33e1dffd063845e709f5fc1cfec93c330445ab4 Mon Sep 17 00:00:00 2001
>From: =?UTF-8?q?Fabiano=20Fid=C3=AAncio?= <fidencio(a)redhat.com>
>Date: Thu, 18 Aug 2016 16:24:17 +0200
>Subject: [PATCH v2] BUILD: Add a few more targets for intg tests
>MIME-Version: 1.0
>Content-Type: text/plain; charset=UTF-8
>Content-Transfer-Encoding: 8bit
>
>Running "make intgcheck" has been proven to be a bit painful (mainly
>when the developer is just writing down a single test case), as it
>cleans up the build directory and fireis a new build before, finally,
>run the tests.
>
>In order to make it a little less painful, let's break the whole
>operation into 4 new targets:
> intgcheck-{prepare,build,run,clean}.
>
>As expected, "make intgcheck" calls these 4 new operations in the same
>order they were presented, not changing then the current behavior.
>
>Each operation will trigger the previous one in case there is no
>"$$prefix" directory created and the directory is _only_ created in the
>very first operation (intghceck-prepare).
>
>Signed-off-by: Fabiano Fidêncio <fidencio(a)redhat.com>
>---
> Makefile.am | 52 ++++++++++++++++++++++++++++++++++++----------------
> 1 file changed, 36 insertions(+), 16 deletions(-)
>
>diff --git a/Makefile.am b/Makefile.am
>index 30d874e..8372b92 100644
>--- a/Makefile.am
>+++ b/Makefile.am
>@@ -3076,30 +3076,50 @@ endif
> # Integration tests #
> #####################
>
>-intgcheck:
>+intgcheck-prepare:
> echo "temporarily disabled"
> set -e; \
>- rm -Rf intg; \
>- $(MKDIR_P) intg/bld; \
>- : Use /hopefully/ short prefix to keep D-Bus socket path short; \
>- prefix=`mktemp --tmpdir --directory sssd-intg.XXXXXXXX`; \
>- $(LN_S) "$$prefix" intg/pfx; \
>- cd intg/bld; \
>+ rm -Rf intg ; \
>+ $(MKDIR_P) intg/bld ; \
>+ : Use /hopefully/ short prefix to keep D-Bus socket path short ; \
>+ prefix=`mktemp --tmpdir --directory sssd-intg.XXXXXXXX` ; \
>+ $(LN_S) "$$prefix" intg/pfx ; \
>+ cd intg/bld ; \
> $(abs_top_srcdir)/configure \
>- --prefix="$$prefix" \
>+ --prefix=$$prefix \
> --with-ldb-lib-dir="$$prefix"/lib/ldb \
> --enable-intgcheck-reqs \
> --without-semanage \
>- $(INTGCHECK_CONFIGURE_FLAGS); \
>- $(MAKE) $(AM_MAKEFLAGS); \
>- : Force single-thread install to workaround concurrency issues; \
>- $(MAKE) $(AM_MAKEFLAGS) -j1 install; \
>- : Remove .la files from LDB module directory to avoid loader warnings; \
>- rm "$$prefix"/lib/ldb/*.la; \
>- $(MAKE) $(AM_MAKEFLAGS) -C src/tests/intg intgcheck-installed; \
>- cd ../..; \
>+ $(INTGCHECK_CONFIGURE_FLAGS) ; \
>+ cd ../..
>+
>+intgcheck-build:
>+ if [ ! -d intg/pfx ]; then $(MAKE) intgcheck-prepare; fi ; \
>+ prefix=`readlink -e intg/pfx` ; \
>+ cd intg/bld ; \
>+ $(MAKE) $(AM_MAKEFLAGS) ; \
>+ : Force single-thread install to workaround concurrency issues ; \
>+ $(MAKE) $(AM_MAKEFLAGS) -j1 install ; \
>+ : Remove .la files from LDB module directory to avoid loader warnings ; \
>+ rm "$$prefix"/lib/ldb/*.la ; \
>+ cd ../..
>+
>+intgcheck-run:
>+ if [ ! -d intg/pfx ]; then $(MAKE) intgcheck-build; fi ; \
>+ cd intg/bld ; \
>+ $(MAKE) $(AM_MAKEFLAGS) -C src/tests/intg intgcheck-installed ; \
>+ cd ../..
>+
>+intgcheck-clean:
>+ prefix=`readlink -e intg/pfx` ; \
> rm -Rf "$$prefix" intg
>
>+intgcheck:
>+ $(MAKE) intgcheck-prepare
>+ $(MAKE) intgcheck-build
I would merge intgcheck-prepare and intgcheck-build bas well.
The reason is quite simple it simplify *my* workflow of
test development.
My current workflow is
* run make intgcheck
* break it before execution of pytest
* change dir to "intg/build"
* finish preparation
make install
* run tests (or just a particular test)
make -C src/tests/intg intgcheck-installed INTGCHECK_PYTEST_ARGS="-k
test_netgroup.py"
(optional)
* clean everything in custom prefix (after breaking stuff there)
rm -rf ../pfx/*
* prepare stuff for test one more time.
make install
* run tests one more time
make -C src/tests/intg intgcheck-installed INTGCHECK_PYTEST_ARGS="-k
test_netgroup.py"
And now the question is: Do others want to use the same workflow
or we should prefer to run all commands from current
build_dir and not from special build_dir for integration tests.
<userstory>
As a developer, I need a way to run a single test without
re-compiling SSSD every time and waiting for all the tests to
finish.
</userstory>
How we get there is not that relevant for me :) I'm fine with compiling
SSSD the first time I run integration tests. It would be nice if
multiple runs after I change SSSD code when debugging would allow me to
only compile the part that changed and run make install, but as a first
iteration, just having a cleaner way to run a single test than running
the full test suite would be nice.