vcswatch reports that
this package seems to have new commits in its VCS but has
not yet updated debian/changelog. You should consider updating
the Debian changelog and uploading this new version into the archive.
Here are the relevant commit logs:
commit 4698c227fc8377b3a59a1837c0592d616fd4f102
Merge: acfcec9f dfe2295e
Author: Antonio Terceiro <terceiro@debian.org>
Date: Fri May 20 11:36:43 2022 +0000
Merge branch 'as-installed' into 'master'
d/tests: Obey autopkgtest's own spec by only testing installed code
See merge request ci-team/autopkgtest!149
commit acfcec9fa261f66b5fb09f564e3d7f71e13cd2d8
Merge: 48fb6fb9 e78f52da
Author: Antonio Terceiro <terceiro@debian.org>
Date: Fri May 20 11:35:35 2022 +0000
Merge branch 'schroot-gitlab-ci' into 'master'
Test the schroot backend on Gitlab-CI
See merge request ci-team/autopkgtest!152
commit e78f52da1d14240a2e64b27a434d9ad9f0ec842d
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 11:29:14 2022 +0100
Gitlab CI: Exercise schroot backend
Unlike the runners we already test on Gitlab-CI, this one has a
virtualized filesystem interface (although it is not full container-level
virtualization), so letting Gitlab-CI run its tests will give us
noticeably better coverage.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 6bf1d53ba1d49ff247d74c52f7a4fd79c9d82788
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 23:11:56 2022 +0100
tests: Run tests on src:tap.py, rather than src:gdk-pixbuf
gdk-pixbuf was an early adopter of autopkgtest for compilation smoke
testing, but now that upstream has added GNOME installed-tests, its
test suite is considerably less trivial than it used to be.
When running the autopkgtest test-suite under schroot under Docker on
Salsa's Gitlab-CI runners, one of the test programs from gdk-pixbuf
was terminated with SIGKILL (possibly because it tried to allocate
too much memory?), resulting in a gdk-pixbuf test failure, which
in turn resulted in an autopkgtest test failure.
Exercising GDK pixbuf loading is not really the point of autopkgtest's
test suite - we just want to prove that we can run tests on a package
from the system deb-src apt sources - so choose a different package to
exercise this particular feature. tap.py doesn't change often, is fairly
quick to test, is maintained by an autopkgtest contributor and fits the
theme of CI and unit testing, so let's go for that one.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 48fb6fb911925c73db5467bfc0a11969c2f60687
Merge: 20f69ed9 6dfa72c1
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 20:38:08 2022 +0000
Merge branch 'fix-schroot-tests' into 'master'
Fix ability to run tests with schroot
See merge request ci-team/autopkgtest!150
commit 20f69ed9673f9aec7e27a8e529aad934d6d968b6
Merge: b27e17c0 591a26bb
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 20:37:30 2022 +0000
Merge branch 'run-result-context-manager' into 'master'
tests: Add a context manager to capture command output
See merge request ci-team/autopkgtest!151
commit 591a26bb57702403513f61355ccf1b50660cc5ef
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 10:37:28 2022 +0100
tests: Convert test_apt_source to RunResult context manager
I've seen this fail on the schroot backend on Gitlab-CI, and this change
will help to debug why.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 4a153a454f3a5169247d02fdc7c2c69df900eaf8
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 10:36:08 2022 +0100
tests: Convert DebTestsFailureModes to RunResult context manager
I've seen some of these failing under the unshare backend, and this
will help to debug what is going on.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 66b9503c16b18b6dd31492498636a13fe2afbe51
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 19 10:33:23 2022 +0100
tests: Add a context manager to capture command output
A recurring problem with failing tests is that the assertion message
on failure doesn't tell the full story: it shows the exit status but
not the stderr, or the stderr but not the stdout.
Another recurring problem is that the assertion message is displayed
in a way that makes it hard to read: as a large Python string literal
wrapped across many lines, or written directly to stdout/stderr in a
way that is easy to confuse with the output of an autopkgtest that is
running autopkgtest's tests.
Attempt to mitigate this by introducing a new context manager. This
does several things for us:
- if the environment variable AUTOPKGTEST_TEST_VERBOSE is set, dump the
command results to stderr *before* making any assertions;
- indent the command's stdout/stderr by a few spaces so that they are
not easily confused with autopkgtest's own output, and prefix '#'
in the style of TAP for the same reason;
- if an exception is raised by the block wrapped in the context manager,
dump the command results to stderr before the assertion failure is
allowed to occur
This commit only converts one test to this new setup, as an example.
Bulk conversion of the other tests should be done separately.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 6dfa72c1662c0c7ebcef37d8cc430bf05acf8ffa
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 00:48:31 2022 +0100
d/tests: Add automated test for the schroot backend
This can be run under lxc or qemu.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 03a6bbbe389e92ab37e791c09c75d363950bef80
Author: Simon McVittie <smcv@debian.org>
Date: Mon May 16 13:53:52 2022 +0100
tests: Cope with schroot not always having AUTOPKGTEST_NORMAL_USER
The schroot backend only sets AUTOPKGTEST_NORMAL_USER if it is invoked
as an ordinary (non-root) user who has permission to run schroot.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* sudo apt-get --no-install-recommends build-dep autopkgtest
* sudo apt-get --no-install-recommends install \
autodep8 debootstrap sbuild schroot
* sudo mkdir /srv/chroot
* sudo sbuild-createchroot --arch=amd64 bullseye \
/srv/chroot/bullseye-amd64-sbuild
* sudo AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest
* minimized reproducer:
sudo AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest \
SchrootRunner.test_dsc_build_needed_success \
SchrootRunner.test_setup_commands
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 5ce2233076432d1a440e3967c61bb8b46bdd0914
Author: Simon McVittie <smcv@debian.org>
Date: Sun May 15 10:23:44 2022 +0100
tests: Fix build of mock test package if a second binary is added
Some of autopkgtest's tests add extra binary packages to this source
package, resulting in the installed files not implicitly being included
in the only binary package. Those tests do not actually need the package
to be non-empty, so we can ignore the non-installed files for now.
Unfortunately, the failing tests are only run with certain virt backends
(schroot), so they were not routinely run by autopkgtest developers
or on ci.debian.net.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* sudo apt-get --no-install-recommends build-dep autopkgtest
* sudo apt-get --no-install-recommends install \
autodep8 debootstrap sbuild schroot
* sudo mkdir /srv/chroot
* sudo sbuild-createchroot --arch=amd64 bullseye \
/srv/chroot/bullseye-amd64-sbuild
* as a non-root member of the sbuild group:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest
* minimized reproducer:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild \
./tests/autopkgtest \
SchrootRunner.test_unversioned_provides_with_conflicting_alternatives \
SchrootRunner.test_unversioned_provides_with_nonconflicting_alternatives \
SchrootRunner.test_versioned_provides_with_conflicting_alternatives \
SchrootRunner.test_versioned_provides_with_nonconflicting_alternatives
Fixes: dd32cc6 "test: modernize testpkg to get rid of debhelper warnings"
Signed-off-by: Simon McVittie <smcv@debian.org>
commit c9e8aaab74d4d9a8fee7f938a30842b80e2935c5
Author: Simon McVittie <smcv@debian.org>
Date: Sat May 14 23:58:08 2022 +0100
tests: If dpkg-buildpackage fails, output why before failing
This will make it easier to identify the root cause for regressions in
this area, as in the commit that follows.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 1e0716ac98f677d24f7026f4f5ef9ec8f39d13e2
Author: Simon McVittie <smcv@debian.org>
Date: Mon May 16 13:57:22 2022 +0100
tests: Skip test_transient_apt_failure until investigated
This apparently worked in the past, but has regressed because schroot
tests were not routinely run. See <https://bugs.debian.org/1011204>
for details.
I would like to be able to use these tests as an acceptance gate for
new changes, but we can't do that until they reliably pass again.
Mitigates: #1011204
Signed-off-by: Simon McVittie <smcv@debian.org>
commit c25cc1e89baf26613a0385e05cca1d637a22bb39
Author: Simon McVittie <smcv@debian.org>
Date: Sat May 14 23:57:51 2022 +0100
tests: Accept more variations of apt output
apt's output when asked to download a nonexistent apt suite has changed
since this test was written: it no longer mentions 'nonexistent/main'
in the output, because it tries and fails to download
'dists/nonexisting/Release' and 'dists/nonexisting/InRelease' first, and
doesn't get as far as discovering that
'dists/nonexisting/main/binary-amd64/Release' doesn't exist either.
Unfortunately, this particular test-case only runs under schroot, which
the autopkgtest maintainers and CI do not routinely run, so it regressed
unnoticed.
Relax the screen-scraping to match more variations.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* sudo apt-get --no-install-recommends build-dep autopkgtest
* sudo apt-get --no-install-recommends install \
autodep8 debootstrap sbuild schroot
* sudo mkdir /srv/chroot
* sudo sbuild-createchroot --arch=amd64 bullseye \
/srv/chroot/bullseye-amd64-sbuild
* as a non-root member of the sbuild group:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest
* minimized reproducer:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild \
./tests/autopkgtest \
SchrootRunner.test_persistent_apt_failure
Signed-off-by: Simon McVittie <smcv@debian.org>
commit dfe2295e74798b9cdaa783d632ff2f516441a9e2
Author: Simon McVittie <smcv@debian.org>
Date: Mon May 16 09:48:52 2022 +0100
d/tests: Don't run pyflakes as an autopkgtest
This is not testing the installed code (as required by the autopkgtest
specification), and is also prone to regressions or false positives if
an updated version of pyflakes starts detecting new issues.
The Gitlab-CI pipeline runs pyflakes during the quicktests step, which
seems a more appropriate time to do QA checks like this.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 67a130ae1c9b788f056c9bf3677cf1a118bf4cf3
Author: Simon McVittie <smcv@debian.org>
Date: Mon May 16 09:44:14 2022 +0100
d/tests: Test the installed autopkgtest runner
The autopkgtest specification calls for the tests to test the installed
code, not the contents of the source package (which is after all the
whole point of as-installed testing), and it seems unfortunate for
autopkgtest to disregard its own specification.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 7ddc91ed6e2611e20ec7503f107dc472331ed407
Author: Simon McVittie <smcv@debian.org>
Date: Sat May 14 23:57:10 2022 +0100
tests: Update for perl autodep8 behaviour changes
The autopkgtest-pkg-perl tests are now correctly marked superficial,
which changes the exit status, and are now named, which changes the
output that we screen-scrape from autopkgtest.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* sudo apt-get --no-install-recommends build-dep autopkgtest
* sudo apt-get --no-install-recommends install \
autodep8 debootstrap sbuild schroot
* sudo mkdir /srv/chroot
* sudo sbuild-createchroot --arch=amd64 bullseye \
/srv/chroot/bullseye-amd64-sbuild
* as a non-root member of the sbuild group:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest
* minimized reproducer:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild \
./tests/autopkgtest \
SchrootRunner.test_apt_autodep8 \
SchrootRunner.test_apt_autodep8_with_control
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 9345e35980e93707d61c6671a528ac4e85f61dd1
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 09:49:38 2022 +0100
tests: Speed up test_git_source_build
Among other things, running autopkgtest's test suite (locally) runs
autopkgtest's test suite (from git), which doesn't seem like the
extra coverage from repeating the tests justifies the time taken.
Streamline it to just running a smoke test.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit bf9721d62498c4fb1a90fbf3a30e167c65a38450
Author: Simon McVittie <smcv@debian.org>
Date: Sat May 14 23:55:57 2022 +0100
tests: Skip test_apt_autodep8_with_control if autodep8 not installed
When running our tests using autopkgtest, this is installed by
d/tests/control, and when actively using autopkgtest, it will often
be installed by Recommends, but when running tests by hand in a
minimal/reproducible environment during development, it might not be.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* sudo apt-get --no-install-recommends build-dep autopkgtest
* sudo apt-get --no-install-recommends install debootstrap sbuild schroot
* sudo mkdir /srv/chroot
* sudo sbuild-createchroot --arch=amd64 bullseye \
/srv/chroot/bullseye-amd64-sbuild
* as a non-root member of the sbuild group:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild ./tests/autopkgtest
* minimized reproducer:
AUTOPKGTEST_TEST_SCHROOT=bullseye-amd64-sbuild \
./tests/autopkgtest SchrootRunner.test_apt_autodep8_with_control
Signed-off-by: Simon McVittie <smcv@debian.org>
commit a33a723f2563fedffc37cc181a74bb7727901423
Author: Simon McVittie <smcv@debian.org>
Date: Sun May 15 16:01:58 2022 +0100
d/tests/control: Our tests now require debhelper 13
When building from source, the Build-Depends provide that, and when
running on any suite >= Debian 11 or Ubuntu 20.10, the version available
is >= 13 anyway; but other parts of autopkgtest go to great lengths to
be backportable, so it seems better to be explicit rather than implicit.
Fixes: dd32cc6 "test: modernize testpkg to get rid of debhelper warnings"
Signed-off-by: Simon McVittie <smcv@debian.org>
commit ae3f64505e4e7aab72f220c37fddd40e00e3dda1
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 09:26:32 2022 +0100
d/control: Add Build-depends and Recommends on fakeroot
autopkgtest's own tests build some test packages which do not have
Rules-Requires-Root: no, and all the tests ought to pass or be skipped
with only the Build-Depends installed.
fakeroot is required for the null runner whenever a test has needs-build,
even if the package has Rules-Requires-Root: no (autopkgtest does not
currently support that field). I would like to remove that restriction
and teach autopkgtest to respect Rules-Requires-Root, but first I want
to get its test suite passing with minimal changes in order to catch
regressions.
Reproducer for test failure:
* have a minimal qemu virtual machine with Debian 11
* apt-get --no-install-recommends build-dep autopkgtest
* as non-root: ./tests/autopkgtest
* minimized reproducer: ./tests/autopkgtest NullRunner.test_build_parallel
Signed-off-by: Simon McVittie <smcv@debian.org>
commit d8c67f80c4d27c90cf0423b0c43a03331e1b19bc
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 09:20:41 2022 +0100
d/tests/control: Wrap restrictions one per line, canonicalize order
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 70f3ee11b8adb3a6b14608d6c213b1552b6d6415
Author: Simon McVittie <smcv@debian.org>
Date: Wed May 18 09:20:26 2022 +0100
d/tests/control: Wrap dependencies one per line, canonicalize order
This will reduce the diff when adding more dependencies later.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit b27e17c0efcc0ce7019f2d3dcf29fbe3a233c2da
Merge: 6ef0a329 1ff2d076
Author: Paul Gevers <elbrus@debian.org>
Date: Sun May 15 19:20:13 2022 +0000
Merge branch 'unshare-util-linux-requirement' into 'master'
unshare: Note required version of util-linux
See merge request ci-team/autopkgtest!147
commit 1ff2d07628a6c77ce25f5e5229eefc93623a164f
Author: Simon McVittie <smcv@debian.org>
Date: Sun May 15 14:36:27 2022 +0100
unshare: Note required version of util-linux
This is available in Debian testing and should arrive in Ubuntu 22.10
soon, but is not in Debian 11, Ubuntu 22.04 or older releases.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 6ef0a32933779211302f66ae78dc79b3ea0e7f52
Merge: 167611ba 95b4765b
Author: Paul Gevers <elbrus@debian.org>
Date: Sat May 14 20:41:58 2022 +0000
Merge branch 'one-per-line' into 'master'
List restrictions and static capabilities one per line in canonicalized order
See merge request ci-team/autopkgtest!146
commit 95b4765b9bd1331ab196b2ddac6368f2698d06f1
Author: Simon McVittie <smcv@debian.org>
Date: Thu May 12 10:33:19 2022 +0100
virt: List static capabilities one per line in alphabetical order
This will minimize merge conflicts when we add more capabilities.
Signed-off-by: Simon McVittie <smcv@debian.org>
commit 4ad0209aa502d6f3783cc4e579419e1d4bbf5baa
Author: Simon McVittie <smcv@debian.org>
Date: Fri Apr 29 10:57:04 2022 +0100
testdesc: List known restrictions one per line
If we keep these in a canonicalized order, this will reduce merge
conflicts when new restrictions are added.
Signed-off-by: Simon McVittie <smcv@debian.org>
![[bug history graph]](https://qa.debian.org/data/bts/graphs/a/autopkgtest.png){kind=link}