Bug#1077958: nmu: rust-async-broadcast_0.7.1-1

[Reinhard Tartler]

Package: release.debian.org
Severity: normal
X-Debbugs-Cc: rust-async-***@packages.debian.org, debian-***@lists.debian.org
Control: affects -1 + src:rust-async-broadcast
User: ***@packages.debian.org
Usertags: binnmu

nmu rust-async-broadcast_0.7.1-1 . riscv64 . unstable . -m "Rebuild to fix previous broken build"

https://packages.debian.org/sid/riscv64/librust-async-broadcast-dev/filelist
looks really weird, and this is not seen on other architectures.

I suspect this broken binray package is the reason why the package fails on
debci and is holding up the rust-zbus transition

[Debian Bug Tracking System]

Post by Reinhard Tartler
affects -1 + src:rust-async-broadcast

Bug #1077958 [release.debian.org] nmu: rust-async-broadcast_0.7.1-1
Added indication that 1077958 affects src:rust-async-broadcast

[Jeremy Bícha]

That is not the actual contents of the .deb. Download the .deb and
open it in file-roller or something to check for yourself. Or look at
the build log.

https://buildd.debian.org/status/package.php?p=rust-async-broadcast

Some Rust autopkgtests take a very long time to complete which is
especially noticeable on riscv64 which I think may be the slowest
autopkgtest infrastructure.

Some rust autopkgtests have additional trouble because
1. No riscv64 packages are in Testing yet
2. The Debian Rust team sets skip-not-installable by default. That
means the migration-reference autopkgtest will always show as neutral
since the dependencies are uninstallable when migration-reference is
the only trigger. I think skip-not-installable by default is bad for
other reasons but I think this situation makes it worse.

See for instance
https://ci.debian.net/packages/r/rust-gtk4/testing/riscv64/ which
passes once in a while. But I don't believe riscv64 is delaying the
rust-zbus transition.

Thank you,
Jeremy Bícha

[Paul Gevers]

Hi,

Post by Jeremy Bícha
Some Rust autopkgtests take a very long time to complete which is
especially noticeable on riscv64 which I think may be the slowest
autopkgtest infrastructure.

That's why the test timeout on riscv64 is double that of other
architectures:
https://salsa.debian.org/ci-team/debian-ci-config/-/blob/master/config/production/nodes.d/riscv64.yaml?ref_type=heads#L11

Post by Jeremy Bícha
Some rust autopkgtests have additional trouble because
1. No riscv64 packages are in Testing yet

This is indeed a problem, largely because of 2.

Post by Jeremy Bícha
2. The Debian Rust team sets skip-not-installable by default. That
means the migration-reference autopkgtest will always show as neutral
since the dependencies are uninstallable when migration-reference is
the only trigger. I think skip-not-installable by default is bad for
other reasons but I think this situation makes it worse.

I regret I added skip-not-installable to autopkgtest. It does more harm
than it helps.

Post by Jeremy Bícha
See for instance
https://ci.debian.net/packages/r/rust-gtk4/testing/riscv64/ which
passes once in a while. But I don't believe riscv64 is delaying the
rust-zbus transition.

Until yesterday at least rust-libseccomp was blocking a lot. I was
hoping that once that migrated (happend yesterday) new tests would have
more chance of succes, as it seems that several tests pass in a pure
unstable environment. So this hints at missing *versioned* dependencies.
But maybe it will resolve itself by time ordering.

Paul

[Reinhard Tartler]

Post by Jeremy Bícha
That is not the actual contents of the .deb. Download the .deb and
open it in file-roller or something to check for yourself. Or look at
the build log.
https://buildd.debian.org/status/package.php?p=rust-async-broadcast
Some Rust autopkgtests take a very long time to complete which is
especially noticeable on riscv64 which I think may be the slowest
autopkgtest infrastructure.
Some rust autopkgtests have additional trouble because
1. No riscv64 packages are in Testing yet
2. The Debian Rust team sets skip-not-installable by default. That
means the migration-reference autopkgtest will always show as neutral
since the dependencies are uninstallable when migration-reference is
the only trigger. I think skip-not-installable by default is bad for
other reasons but I think this situation makes it worse.
See for instance
https://ci.debian.net/packages/r/rust-gtk4/testing/riscv64/ which
passes once in a while. But I don't believe riscv64 is delaying the
rust-zbus transition.

Is that the same reason why we see
https://ci.debian.net/packages/r/rust-async-broadcast/testing/riscv64/
being retried over and over again without progress?

In contrast to the rust-gtk5/testing/risc64 example, in
rust-async-broadcast/testing/riscv64 failures have logfiles that indicate
that the actual tests don't even run, still it is being counted as a
failure.

Also, looking at the description of "skip-not-installable":

This restrictions may cause a test to miss a regression due to
installability issues, so use with caution. If one only wants to
skip certain architectures, use the ``Architecture`` field for
that.

This test might have test dependencies that can't be fulfilled in
all suites or in derivatives. Therefore, when apt-get installs the
test dependencies, it will fail. Don't treat this as a test
failure, but instead treat it as if the test was skipped.

I need some help to understand how skipped tests lead to delaying the
package migration to testing. My naive understanding is that this flag
would rather allow issues to go through to testing?

Changing that would be a one-liner to
https://salsa.debian.org/rust-team/debcargo/-/blob/master/src/debian/control.rs?ref_type=heads#L149,
but before proposing that, I'd need to understand your concern better.


My question is basically: what needs to be done so that
https://tracker.debian.org/pkg/rust-event-listener can actually migrate
testing?

[Paul Gevers]

Hi

Post by Reinhard Tartler
I need some help to understand how skipped tests lead to delaying the
package migration to testing. My naive understanding is that this flag
would rather allow issues to go through to testing?

What skipping means isn't up to autopkgtest to determine, but up to the
consumers of the test results. For migration to testing, that britney2
that's owned by the Release Team. We, the Release Team, want britney2 to
prevent migration:
a) the autopkgtest of a package doesn't fail in testing, but the test
fails with $something from unstable (regression). That $something should
be blocked.
b) the test is new to testing, but it fails (weird definition of
regression).

Post by Reinhard Tartler
My question is basically: what needs to be done so that
https://tracker.debian.org/pkg/rust-event-listener
<https://tracker.debian.org/pkg/rust-event-listener> can actually
migrate testing?

Good question. It seems that britney2 did schedule some combination
"correctly", but not all. That is probably due to the order in which
packages were uploaded. I.e. if a package needs some other package,
britney2 will schedule the test for both triggers and if it passes, it
counts for both. However, if the dependent-on package is later updated
again, britney2 doesn't see the relation and will not schedule the
combination. If the test than fails, the package is blocked. If there
would be a versioned Breaks (not sure if the package is really broken,
or merely the test, then a Breaks is a bit overkill), than britney2
would again trigger the combination.

Does this help to tell the right answer? If this is only a test issue,
and not really a versioned Breaks missing, I'm willing to manually
trigger the right test combination. But then people need to hold off
with uploading, otherwise the results might come too late.

Paul

[Reinhard Tartler]

Post by Paul Gevers
Hi

Post by Reinhard Tartler
I need some help to understand how skipped tests lead to delaying the
package migration to testing. My naive understanding is that this flag
would rather allow issues to go through to testing?

What skipping means isn't up to autopkgtest to determine, but up to the
consumers of the test results. For migration to testing, that britney2
that's owned by the Release Team. We, the Release Team, want britney2 to
a) the autopkgtest of a package doesn't fail in testing, but the test
fails with $something from unstable (regression). That $something should
be blocked.
b) the test is new to testing, but it fails (weird definition of
regression).

Post by Reinhard Tartler
My question is basically: what needs to be done so that
https://tracker.debian.org/pkg/rust-event-listener
<https://tracker.debian.org/pkg/rust-event-listener> can actually
migrate testing?

Good question. It seems that britney2 did schedule some combination
"correctly", but not all. That is probably due to the order in which
packages were uploaded. I.e. if a package needs some other package,
britney2 will schedule the test for both triggers and if it passes, it
counts for both. However, if the dependent-on package is later updated
again, britney2 doesn't see the relation and will not schedule the
combination. If the test than fails, the package is blocked. If there
would be a versioned Breaks (not sure if the package is really broken,
or merely the test, then a Breaks is a bit overkill), than britney2
would again trigger the combination.

So I think I managed to find the right combination of packages.
Basically, we need all of
- rust-async-channel
- rust-async-process
- rust-event-listener
- rust-async-broadcast

to migrate together. When triggering with all of these four sources from
unstable, I end up with a successful autopkgtest:
https://ci.debian.net/data/autopkgtest/testing/riscv64/r/rust-async-broadcast/50019012/log.gz

How to convey that to britney/debci?

Is that something that could be added as a hint by the release team, or
does that require adding what Breaks to what package exactly?

-rt

[Peter Green]

Unfortunately in many cases, the performance difference seems a lot more
than a factor of two.

Take rust-gix for example, looking at the results for testing migration
tests I see

18-28m on amd64
53-59m on arm64
33-47m on armel
33-48m on armhf
1h59m-2h2m on i386
49m-1h37m on ppc64el
23m-55m on s390x

on riscv64 it was timing out after over 8 hours. To unblock things I modified it
to only test a single feature configuration. It then passed in 29 minuites.

If we assume all feature configurations take the same time (very rough) then
that suggests running the full set of tests on riscv64 would take of the order
of 23 hours. Over 10 times worse than the next-slowest architecture.

Post by Paul Gevers
So this hints at missing *versioned* dependencies.

It seems to me that britney's autopkgtest scheduler sometimes fails to pick
up that stuff needs to go in together. Even though the main part of britney
knows that they do.

The end result of this is frequently that the fallback dependency solver
gets triggered and autopkgtest tries to run the tests from testing against
the package from unstable. This often results in a "crate directory not
found" error. In other cases the fallback depsolver is not invoked or
doesn't succeed and the testsuite fails with a "fail badpkg" error.

Why this is happening I'm not entirely sure. Though I have some theories

In some cases I suspect it happens because the dependencies are indirect.
or even involve going down the tree and back up again.

In some cases I suspect it happens because riscv64 testing is currently
a basket case, so the packages are already uninstallable in testing.

In some cases I suspect it happens due to the special treatment of arch
all packages (rust team packages are arch any, but Jonas's packages
are arch all). For example if we look at the migration page for
rust-event-listener, it seems britney has done the right thing on amd64
and arm64 (where arch all packages are required to be installable) but
not done the right thing on architectures where arch all packages are
not required to be installable.

[Matthias Geiger]

On Mon, 5 Aug 2024 12:57:27 +0100 Peter Green

Post by Peter Green
Unfortunately in many cases, the performance difference seems a lot more
than a factor of two.
Take rust-gix for example, looking at the results for testing migration
tests I see
18-28m on amd64
53-59m on arm64
33-47m on armel
33-48m on armhf
1h59m-2h2m on i386
49m-1h37m on ppc64el
23m-55m on s390x
on riscv64 it was timing out after over 8 hours. To unblock things I modified it
to only test a single feature configuration. It then passed in 29 minuites.
If we assume all feature configurations take the same time (very rough) then
that suggests running the full set of tests on riscv64 would take of the order
of 23 hours. Over 10 times worse than the next-slowest architecture.

Post by Paul Gevers
So this hints at missing *versioned* dependencies.

It seems to me that britney's autopkgtest scheduler sometimes fails to pick
up that stuff needs to go in together. Even though the main part of britney
knows that they do.
The end result of this is frequently that the fallback dependency solver
gets triggered and autopkgtest tries to run the tests from testing against
the package from unstable. This often results in a "crate directory not
found" error. In other cases the fallback depsolver is not invoked or
doesn't succeed and the testsuite fails with a "fail badpkg" error.
Why this is happening I'm not entirely sure. Though I have some theories
In some cases I suspect it happens because the dependencies are indirect.
or even involve going down the tree and back up again.
In some cases I suspect it happens because riscv64 testing is currently
a basket case, so the packages are already uninstallable in testing.
In some cases I suspect it happens due to the special treatment of arch
all packages (rust team packages are arch any, but Jonas's packages
are arch all). For example if we look at the migration page for
rust-event-listener, it seems britney has done the right thing on amd64
and arm64 (where arch all packages are required to be installable) but
not done the right thing on architectures where arch all packages are
not required to be installable.

Converting the librust-* packages to arch:all is wrong anyway since this
can break crosscompling in some cases, but that's only part of the
problem here.

From #debian-release:

"jochensp    werdahias: rust-async-channel breaks
rust-async-executor/1.12.0-3 and rust-mpris-server/0.7.0-1 and probably
more, if I get it right"

rust-libseccomp migrated but rust-async-broadcast can't migrate due to
rust-event-listener which seems to be missing a B+R.


best,

werdahias

[Debian Bug Tracking System]

Your message dated Thu, 02 Jan 2025 21:39:51 +0000
with message-id <***@adam-barratt.org.uk>
and subject line Re: Bug#1077958: nmu: rust-async-broadcast_0.7.1-1
has caused the Debian Bug report #1077958,
regarding nmu: rust-async-broadcast_0.7.1-1
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ***@bugs.debian.org
immediately.)