commit 8114e425ac5463081a145e075f5fc0a04f4c68f2
Merge: af0c7739e 4b6ccb475
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 31 09:11:33 2025 +0000
Merge branch 'make-signed-source-multiple-signing-outputs' into 'devel'
Pass all signing outputs through to assemble-signed-source
Closes #692 and #555
See merge request freexian-team/debusine!1596
commit 4b6ccb475747b3585dc785e97067487a6cc9db7f
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 30 17:06:27 2025 +0000
Pass all signing outputs through to assemble-signed-source
When I fixed #555, I arranged for the `make_signed_source` workflow to
handle the fact that `ExtractForSigning` may fan out multiple
`debusine:signing-input` artifacts, and pass each of those as inputs to
the `Sign` task. However, the `Sign` task produces one
`debusine:signing-output` artifact per `debusine:signing-input`, and so
we also need to fan those back in when setting up the
`AssembleSignedSource` task.
This requires adding an optional `binary_package_name` field to
`debusine:signing-output`, so that we can distinguish the signing
outputs when adding them to the workflow's internal collection.
Fixes: #692
commit af0c7739e0b67d16aaec79b6d9f7cc6f269569da
Merge: 2a49951cf b6f1fd180
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 30 12:06:52 2025 +0000
Merge branch 'fix-dependencies-between-workflows' into 'devel'
Fix handling of dependencies between workflows
Closes #690
See merge request freexian-team/debusine!1593
commit 2a49951cf270ed698bf82ea578c007d97f798e2a
Merge: c259981da 5f0b84f57
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 30 11:55:06 2025 +0000
Merge branch 'migrations-reload-model' into 'devel'
Work around a subtle Django migration bug
See merge request freexian-team/debusine!1594
commit b6f1fd1809bcbeb9c36c624070518085c4c433ce
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 29 12:31:14 2025 +0000
Fix handling of dependencies between workflows
Create sub-workflows as pending rather than running, to give the parent
orchestrator a chance to possibly add dependencies before marking them
as running.
I audited all workflows for this. In most cases they were incorrectly
marking sub-workflows as having dependencies via `requires_artifact`
when the sub-workflow orchestrators didn't need those artifacts
directly; the only dependencies should be from the child work requests
that actually use those artifacts. However, `make_signed_source` is an
exception: it creates an `sbuild` sub-workflow that really does need to
wait for the `AssembleSignedSource` task to complete before it can
create further children, because it needs to know details of the
assembled source package that aren't available beforehand. In that
case, it makes sense for the `sbuild` sub-workflow to have a direct
dependency on `AssembleSignedSource`.
The `autopkgtest` workflow omitted making its child work requests depend
on the work request producing the source artifact. In most cases this
is irrelevant as the source artifact already exists by the time the
`autopkgtest` workflow is populated, but it might matter if we wanted to
run `autopkgtest` on assembled signed source packages in future, so I
added this dependency while I remembered about it.
Fixes: #690
commit 5f0b84f57c0723cffdff68f80b9818ebab0862f2
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 29 18:33:28 2025 +0000
Work around a subtle Django migration bug
Enrico and I spent some time investigating a `ValueError` when deleting
a collection in the reverse side of a data migration, and came up with
the analysis in
https://salsa.debian.org/freexian-team/debusine/-/merge_requests/1588#note_577725.
Fortunately, it's possible to work around it with a small hack in a
couple of our existing migrations.
commit c259981daf73be47db4592085dadcb749e6cc6b2
Merge: 8d66f1b59 02a7274b2
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 29 10:42:00 2025 +0000
Merge branch 'update-collection-with-artifacts-variables' into 'devel'
update-collection-with-artifacts: Always pass through variables
Closes #689
See merge request freexian-team/debusine!1591
commit 02a7274b2073d01d3b89b5b0b2c9c94f1ad40849
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 28 18:02:19 2025 +0000
update-collection-with-artifacts: Always pass through variables
In some workflows (such as `make_signed_source`), we need to create work
requests that add items to the internal collection using
`update-collection-with-artifacts`, and then filter those items in
subsequent work requests using a lookup that filters on properties other
than their name. This is all a lot easier if the event reaction always
passes variables through to the collection manager, rather than having
the somewhat arbitrary rule that it only passes variables if you don't
provide a `name_template`.
The tests for `make_signed_source` are now significantly stricter to
catch problems of this kind.
Fixes: #689
commit 8d66f1b594b0281c208d177febf079705449817e
Merge: 029849b0e e626899cc
Author: Stefano Rivera <stefanor@debian.org>
Date: Mon Jan 27 12:40:27 2025 +0000
Merge branch 'create_group_role' into 'devel'
Refactor playground.create_group_role
See merge request freexian-team/debusine!1586
commit e626899cc28c3a7239259c282dbc23546cc1fd9c
Author: Stefano Rivera <stefano@freexian.com>
Date: Sun Jan 26 10:55:35 2025 -0400
Use playground.create_group() where useful
commit 968194fd7688c706826b40e5ca0581e78c662586
Author: Stefano Rivera <stefano@freexian.com>
Date: Sun Jan 26 10:45:03 2025 -0400
Factor create_group() out of create_group_role()
commit 3cd0e287650f92c7ead788f3c86ab0a7a99115d8
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 25 22:04:13 2025 -0400
Use create_group_role's users argument
commit fe703c40c2883cad78366c48af9ce52bbe0193c2
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 25 21:58:11 2025 -0400
Refactor create_group_role to avoid *args for users
*args gets awkward when there are several arguments and some are
optional. Let's just pass a list.
commit 029849b0e67a9d1a8a2032b0329e033c796a2a99
Merge: 70bf76d92 21a527297
Author: Carles Pina i Estany <carles@pina.cat>
Date: Fri Jan 24 15:01:15 2025 +0000
Merge branch 'consistent-task-tests-naming' into 'devel'
Consistent task tests naming
See merge request freexian-team/debusine!1585
commit 21a5272973891429670c26a56fcfa197b8b3ce9d
Author: Carles Pina i Estany <carles@pina.cat>
Date: Fri Jan 24 15:01:09 2025 +0000
Rename task test classes (so they match the class under test) and improve docstrings
commit 70bf76d923e4ee38da7aafe1d534447685e0aa12
Merge: 4952efe88 0135425b6
Author: Stefano Rivera <stefanor@debian.org>
Date: Fri Jan 24 13:28:37 2025 +0000
Merge branch 'state-in-class-vars' into 'devel'
Store SystemBootstrap state in instance attributes not class attributes
Closes #664 and #686
See merge request freexian-team/debusine!1583
commit 0135425b68e876af26e205dacd0372d217fb39ae
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 23 20:35:24 2025 -0400
Drop empty __init__()
commit 6d7d74865faba4ebef797abf3aefd5d3c9ccfbf6
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 23 20:35:05 2025 -0400
Store SystemBootstrap in instance attributes not class attributes
Fixes: #686
commit 4952efe8838da22990fcddc2a091a1d7b21a63ad
Merge: 507fedf34 0e91891eb
Author: Stefano Rivera <stefanor@debian.org>
Date: Thu Jan 23 23:20:26 2025 +0000
Merge branch 'signing-key-artifact-cleanup' into 'devel'
Recommend admins cleanup signing-key artifacts
See merge request freexian-team/debusine!1582
commit 0e91891eb0a1999c69521863de06824250c7c4bb
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 23 08:35:11 2025 -0400
Recommend admins cleanup signing-key artifacts
commit 507fedf345902a368775c5ada19276da246cb529
Merge: 44a130ad5 c34e3444b
Author: Carles Pina i Estany <carles@pina.cat>
Date: Thu Jan 23 06:49:33 2025 +0000
Merge branch 'subject-context-sbuild' into 'devel'
Subject context sbuild (#679)
See merge request freexian-team/debusine!1565
commit c34e3444ba08907e5228b54a569edaf9b2e2a74e
Author: Carles Pina i Estany <carles@pina.cat>
Date: Thu Jan 23 06:49:33 2025 +0000
Sbuild.compute_dynamic_data: computes `subject` and `runtime_context`
commit 44a130ad5ebb5d2af93e992a59e73b5ad2ff87da
Merge: 0f79ccfe3 3d8418fdb
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 13:35:10 2025 +0000
Merge branch 'design-dynamic-storage' into 'devel'
Add blueprint for dynamic cloud storage scaling
See merge request freexian-team/debusine!1563
commit 3d8418fdbabde7e4610091c60e06cfa428474a79
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 11:16:02 2025 +0000
Add storage policy recommendations
commit 9b264df8bc9b6887dc4838b57e4d73416a84d7ad
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 11:15:31 2025 +0000
Add FileStore.soft_max_size
commit 4a116761f6f1948b9daed41e06ee36d716126961
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 10:56:48 2025 +0000
Note that the shared local storage is a file store
commit 6d21227846baa0c86455f579ce550a92a098f9d0
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 10:53:21 2025 +0000
Add maximum size controls, and refine drain policy
commit 0f79ccfe3dc3ec2263f0b3fd3249e7af229c17c1
Merge: 92bb45295 0a0015d89
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 22 10:27:08 2025 +0000
Merge branch 'remove-schroot' into 'devel'
Remove schroot support
Closes #660
See merge request freexian-team/debusine!1580
commit 92bb45295297d2c141aa64e7eb485266d0a3ddc3
Merge: 8594b2994 59c2491b3
Author: Carles Pina i Estany <carles@pina.cat>
Date: Wed Jan 22 05:38:17 2025 +0000
Merge branch 'fix-title-homepage-scope' into 'devel'
Fix title of homepage and scope pages
See merge request freexian-team/debusine!1577
commit 645e250b6411b225582c6123efe5c47acb4cff0a
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 21 17:08:46 2025 +0000
Build storage maintenance job from monthly_cleanup, not delete_expired
commit 0a85182fe51e47485d4e885be9618cd9d3b80504
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 21 17:05:34 2025 +0000
Move Scope.file_store_policies to extra relationship data
`priority` is split into `upload_priority` and `download_priority`, the
split between `default_file_store` and `other_file_stores` goes away,
and we have a new `instance_wide` field on `FileStore`.
commit 8594b299430235a0e243ed7499ae13ed905c308b
Merge: fcc106dc9 66bc14b32
Author: Stefano Rivera <stefanor@debian.org>
Date: Tue Jan 21 16:41:49 2025 +0000
Merge branch 'asset-migration-cleanup-relations' into 'devel'
Signing Key Artifact to asset migration: Clean up relations before deleting signing-key artifacts
See merge request freexian-team/debusine!1579
commit 91b0c9c82ef9eaceeab6dba2d8519f04ea95cf2c
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 21 16:31:27 2025 +0000
Add read_only and write_only policies
commit fcc106dc9f71949ed2171d2231ffa86273f0494b
Merge: 711bd6f49 a33fbe158
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 21 15:54:31 2025 +0000
Merge branch 'break-up-server-view-tests' into 'devel'
Break up debusine.server.views.tests.test_views
Closes #666
See merge request freexian-team/debusine!1574
commit 711bd6f494e4ba65a171d07412dcb43f5f4d60a2
Merge: bd3ac8ea9 a120c32f4
Author: Stefano Rivera <stefanor@debian.org>
Date: Tue Jan 21 15:15:42 2025 +0000
Merge branch 'fp-migration-null-dd' into 'devel'
Signing Fingerprint Work Request Migration: Handle null work_request dynamic_task_data
See merge request freexian-team/debusine!1578
commit 66bc14b32bf345d56fb29f22cb6f7db2c5f2baaf
Author: Stefano Rivera <stefano@freexian.com>
Date: Tue Jan 21 10:38:05 2025 -0400
Signing Key Artifact to asset migration: Clean up relations before deleting signing-key artifacts
commit a33fbe15890e8a8d82942d90dc5ff1cd341bfc73
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 20 12:32:16 2025 +0000
Break up debusine.server.views.tests.test_views
This corresponds to the module layout of the views themselves. No code
changes, just code motion.
Fixes: #666
commit 0a0015d890212ea703fbffb19808cc17cf0831b0
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 21 14:06:32 2025 +0000
Remove schroot support
Work requests that use it will no longer pass Pydantic validation, but
there are no such work requests on debusine.debian.net.
Fixes: #660
commit a120c32f4661522cbb24505b0dfcc9f7cf138b6e
Author: Stefano Rivera <stefano@freexian.com>
Date: Tue Jan 21 10:06:16 2025 -0400
Signing Fingerprint Work Request Migration: Handle null work_request dynamic_task_data
commit 59c2491b3fc6f2eb179e320f80bb4ad6ae8f4210
Author: Carles Pina i Estany <carles@pina.cat>
Date: Tue Jan 21 10:03:38 2025 +0100
Homepage title change from "Debusine - " to "Debusine - Homepage"
commit bd3ac8ea95963f4154c17b5a883b0e9241016413
Merge: 568391c8b 0ebdb014a
Author: Stefano Rivera <stefanor@debian.org>
Date: Mon Jan 20 22:18:39 2025 +0000
Merge branch 'select2-dep' into 'devel'
select2 binary package dependency
See merge request freexian-team/debusine!1575
commit 0ebdb014a0e6d6e04fd589528dd113ac66a9d3cd
Author: Stefano Rivera <stefano@freexian.com>
Date: Mon Jan 20 11:09:17 2025 -0400
Drop <!nodoc> from libjs-select2.js, not used for docs
commit 99eafd2c2efd79cf8df9ac5bcbd9599fe239a68c
Author: Stefano Rivera <stefano@freexian.com>
Date: Mon Jan 20 11:07:45 2025 -0400
Explicitly depend on select2 in the debusine-server package
commit 568391c8bd57484b632223231207d94b267908a4
Merge: 1fff6ad2d eeae4c457
Author: Stefano Rivera <stefanor@debian.org>
Date: Mon Jan 20 14:01:40 2025 +0000
Merge branch 'asset-work-request-migration' into 'devel'
WorkRequest fingerprint migration: Skip null key lookups
See merge request freexian-team/debusine!1572
commit 48db20989cc40ef51cbcd2fda0c3063711ad0c88
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 13:20:41 2025 +0000
Specify debusine-admin delete_file_store
commit eeae4c457889ed820ebe6526a3653f828d5896fe
Author: Stefano Rivera <stefano@freexian.com>
Date: Sun Jan 19 19:03:56 2025 -0400
WorkRequest fingerprint migration: Skip null key lookups
commit 1fff6ad2d0fa9275ff01709d86105d7a24d543ee
Merge: c0fbb2935 59001bd6a
Author: Stefano Rivera <stefanor@debian.org>
Date: Sun Jan 19 15:48:31 2025 +0000
Merge branch 'assets' into 'devel'
Implement Assets and use them to hold signing keys
See merge request freexian-team/debusine!1538
commit 59001bd6ac205c8d2bc9dd052ceb5efc6cb8dbc5
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 4 21:25:59 2025 -0400
Migrate debusine:signing-key artifacts to assets
commit 6937d47614bdc3c13ed555965e15bfe57153d313
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 4 19:09:27 2025 -0400
Remove debusine:signing-key artifacts
commit a58645de98f86ea90d1728b95e811b9f40205db5
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 4 18:53:10 2025 -0400
Update integration tests to use signing key assets
commit 51a51a7e30aba6af07253e18e481c5b4e0f17670
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 4 18:31:06 2025 -0400
Add a CLI client for listing assets
Needed for the integration tests.
commit 0712aba0ecdf17cd003917e3b265194021e91c7c
Author: Stefano Rivera <stefano@freexian.com>
Date: Fri Jan 3 21:32:34 2025 -0400
API client for listing assets
commit d8623080b2ac0d39ab64e081f88a69d79b1ba497
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 2 22:15:23 2025 -0400
API to list assets
We need this so that we can look up which assets were created by a work
request.
commit f78971843fac92b4da0d4220b89ac75ae11f963e
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 2 16:59:41 2025 -0400
CLI to create assets
We need it for manually importing HSM keys
commit 1834cf94b7f80d59d8338b312eb031f82f557606
Author: Stefano Rivera <stefano@freexian.com>
Date: Mon Dec 30 15:36:33 2024 -0400
Remove debian:suite-signing-keys
commit 2dbb8f028c729de4172cb72fbfa29f02d3866e11
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 16 21:46:39 2025 -0400
Migrate WorkRequests to use fingerprints
commit 552091f569359febbdf7d61f53edcca7d5f69bff
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 2 14:20:02 2025 -0400
Signing tasks: Specify the key by fingerprint
Rather than using debusine:signing-key artifact lookups.
commit 0086bc4103fe1f79b645bc441d6caf554704ad6e
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Jan 2 08:35:01 2025 -0400
Create signing-key assets in the GenerateKey task
commit 5afe35d14288237826a3fcfcbda13ade7d821008
Author: Stefano Rivera <stefano@freexian.com>
Date: Wed Jan 1 20:24:39 2025 -0400
Create Assets with the debusine client
commit 54780a432d39faa188c2aa1c3c9c8b90afa120bb
Author: Stefano Rivera <stefano@freexian.com>
Date: Tue Dec 31 16:44:18 2024 -0400
API to create assets
commit 15d7bc457e3e9e6e4bc56515e446fccdc7b2915b
Author: Stefano Rivera <stefano@freexian.com>
Date: Mon Dec 30 22:27:08 2024 -0400
Factor out GetOrCreateAPIView.get_existing_object
commit 0ab17c124ebf67950b56b9b2991a833a3cd17c20
Author: Stefano Rivera <stefano@freexian.com>
Date: Mon Dec 30 18:48:30 2024 -0400
Move GetOrCreateAPIView to base
So we can use it for assets too
commit 8174bfcc1a3f6be4b836112463fda63a02cb1d19
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Dec 28 15:28:09 2024 -0400
Implement Asset models
commit b0216df2d729bd1483f757e21be489ed68219909
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Jan 4 21:49:48 2025 -0400
Playground: Set the worker token in create_worker_token, when a worker is specified
commit 6fd3b0f027a3b00a2fe10acf457c0194ac141e63
Author: Stefano Rivera <stefano@freexian.com>
Date: Wed Jan 8 09:31:54 2025 -0400
Adjust spec: assets must be unique by fingerprint
This avoids attacks where a (scope owner) user is able to create an
asset for an existing key, with a different purpose. This would
conceivably be a much more serious issue when we add more purpose
values.
commit c0fbb29358ff902797082d50d8bc279942efae99
Merge: 2ccf36d90 f92e35543
Author: Stefano Rivera <stefanor@debian.org>
Date: Fri Jan 17 14:52:29 2025 +0000
Merge branch 'duplicate-workflow-templates' into 'devel'
Filter WorkflowTemplateDetailView to the current workspace
Closes #680
See merge request freexian-team/debusine!1571
commit 2ccf36d903928a4c36e434c5b9a9fd925f57e96b
Merge: 570293394 e29fcca5f
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 13:30:53 2025 +0000
Merge branch 'fix-backports-version' into 'devel'
Fix versioning for backports
See merge request freexian-team/debusine!1570
commit f92e3554396d9e0ebb87a24192b674d7f0007904
Author: Stefano Rivera <stefano@freexian.com>
Date: Fri Jan 17 08:39:51 2025 -0400
Fiter WorkflowTemplateDetailView to the current workspace
Fixes: #680
commit e29fcca5f8670a2086fc9a2ff48882db3d71a0e4
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 12:28:25 2025 +0000
Fix versioning for backports
dh-python's default handling is designed for packages that have an
upstream rather than ones that are native to Debian, and it doesn't
quite get everything quite right on its own. This resulted in a build
failure in bookworm-backports:
https://buildd.debian.org/status/fetch.php?pkg=debusine&arch=all&ver=0.8.1%7Ebpo12%2B1&stamp=1737039067&raw=0
commit 5702933940f9fd8f4a1e1808e81ad78b65d4c30c
Merge: 8da62b273 4172509f9
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 11:44:44 2025 +0000
Merge branch 'doc-split-artifacts' into 'devel'
Split artifacts documentation by category
See merge request freexian-team/debusine!1569
commit 8da62b273bef64c1ce45cac5ad41959ac4f91c2b
Merge: 5a0754cb8 91627ec26
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 11:44:15 2025 +0000
Merge branch 'duplicate-recommends' into 'devel'
Remove duplicate Recommends
See merge request freexian-team/debusine!1566
commit 5a0754cb820f018603d4f9059b39ac66b5630070
Merge: 8c98f3f82 7303852ea
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Jan 17 11:43:17 2025 +0000
Merge branch 'collection-item-detail-slash' into 'devel'
Fix collection item detail URLs to allow slashes in names
Closes #665
See merge request freexian-team/debusine!1568
commit 4172509f9194d600b418e77adda4e8e5554f30fb
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 16 15:22:25 2025 +0000
Split artifacts documentation by category
Part of #541.
commit 7303852eac0fc1ef6bce713b03e617878d379bb1
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 16 14:59:41 2025 +0000
Fix collection item detail URLs to allow slashes in names
Fixes: #665
commit 91627ec26b6f06e86779166a3faafa50045cd972
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Jan 16 09:25:58 2025 +0000
Remove duplicate Recommends
commit 8c98f3f82af6a5106acc1fe52b3b75de44f4213a
Merge: 3977c3505 c34e87e52
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 15 14:32:39 2025 +0000
Merge branch 'output-data-runtime-statistics' into 'devel'
Record runtime statistics in output_data
See merge request freexian-team/debusine!1562
commit c34e87e52e3efb43c98601fd98a1eb6a46c2cee9
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 12:38:23 2025 +0000
Record runtime statistics in output_data
Part of #510.
commit 3977c35055950befbb48b42d92a70a8f5b96c32a
Merge: 496adcd03 b79344b20
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Jan 15 11:33:14 2025 +0000
Merge branch 'worker-async-fixes' into 'devel'
Fix various worker asyncio issues
See merge request freexian-team/debusine!1564
commit b79344b201bd351233b7e5ca8c1c89e7f13f6f71
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 22:27:28 2025 +0000
Move Worker._send_task_result to asyncio event loop thread
While debugging an occasional test failure in !1562, I ran into some
awkward problems caused by `Worker._send_task_result` running in a
thread and thus not being able to use `asyncio` normally. I think it
makes more sense to call it from the event loop, and then its
environment is more consistent with that of the rest of the worker.
commit ee28ba10293946f18c018330947e136dc6c05190
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 22:18:37 2025 +0000
Run work_request_completed_update in a separate thread on the worker
We shouldn't be doing blocking I/O in the asyncio event loop thread.
commit eef83bece8066193843896933dec7bcf3f70f27b
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 22:10:48 2025 +0000
Don't exit worker when reporting completion fails
Since commit e5349048994c32cb3bb175c6cd06340d4a46c4a4, this is no longer
necessary: the server will retry the running work request when its
worker next manages to connect and requests a new work request to run,
so we don't need to worry about leaving the work request stuck in the
Running state any more.
commit 00168c56b64217945922fa5f9c7d49f5b4ea4638
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 19:11:11 2025 +0000
Test Worker._reset_state in more situations
commit 8478a5c43c21cfc2f6fc1c874ccd66f992bfc612
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 19:07:04 2025 +0000
Fix spelling of PYTHONASYNCIODEBUG
As far as I can tell, this environment variable has never been spelled
`PYTHONASYNCDEBUG`, but always `PYTHONASYNCIODEBUG`. See
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONASYNCIODEBUG.
commit c69b82b4d679ae819b522a5cadbe06810f0eff88
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 15:18:01 2025 +0000
Add blueprint for dynamic cloud storage scaling
Part of #539.
commit 496adcd0333ac42474915f9e8d6d14228fca8e01
Merge: f7a182d1b 1ec08a529
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 14 13:44:59 2025 +0000
Merge branch 'task-history-collection' into 'devel'
Implement debusine:task-history collection
See merge request freexian-team/debusine!1542
commit 1ec08a529084beeea25f0925e5954fe2a0a13dbf
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 14:40:02 2025 +0000
Implement debusine:task-history collection
Part of #510.
commit f7a182d1bd3bc8fbecfd3a17b7fec9cb49ae8114
Merge: 91d5905ac ac4792423
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 13 17:28:13 2025 +0000
Merge branch 'extend-task-database-interface' into 'devel'
Extend TaskDatabaseInterface to include artifact category and data
See merge request freexian-team/debusine!1558
commit 91d5905ac5cbd3c96ba4540c3a52caf259bab6bd
Merge: 3c6713c52 1c858f7e9
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 13 17:27:14 2025 +0000
Merge branch 'fix-lintian-label' into 'devel'
Handle empty Lintian artifacts
Closes #677
See merge request freexian-team/debusine!1559
commit 6374da33085271feb543a7fb949be6aac39f9f8b
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 14:24:13 2025 +0000
Clarify semantics of old_items_to_keep
I found the previous text a little unclear when trying to implement it.
This tightens up the wording with my best guess as to what we meant when
designing this.
commit 51b5f53e5e2e967b6311f1969e3c6b1bb84b9ba5
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 14:21:54 2025 +0000
work_request_id/result are required for all task history items
`work_request_id` is needed to construct the item name, and `result` is
useful so that lookups don't always have to join with `WorkRequest`.
commit d68b59b5726b8b699906d6e3ed7e921873ec6c6a
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 14:20:05 2025 +0000
Improve ergonomics of creating bare data items
Having the option to pass a pydantic model rather than a dictionary
helps to avoid typos.
commit fa14529a27416a8f230b2df6c52a36320ddbfdb4
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 13:35:51 2025 +0000
Move RuntimeStatistics to debusine.artifacts.models
I'm about to want to use it in the model for
`debusine:historical-task-run` bare data items.
commit b3a5f0dde2abdc1f605f0c53c13c9fb8cd742516
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Jan 7 13:24:39 2025 +0000
Add explicit cast to build log lookup filtering
This makes the query slightly more accurate
(`(("db_collectionitem"."data" -> work_request_id))::integer IN (1)`
instead of `("db_collectionitem"."data" -> work_request_id) IN ('1')`).
It doesn't seem to make a practical difference in this case, but this
pattern works better when the right-hand side of the `IN` is something
more complicated such as a subquery.
commit 1c858f7e9dc30d195a00333c79cfe61656845afa
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 13 12:29:31 2025 +0000
Handle empty Lintian artifacts
By default, the Lintian task creates an analysis artifact for each of
source, binary-all, and binary-any; the binary-all and binary-any
artifacts can be effectively empty if there are no binary packages of
the corresponding type.
I think reducing Lintian artifacts to the shortest-length package name
they contain for the purposes of generating labels is a bit _too_
abbreviated and quite confusing, so I reworked this to include all the
package names in sorted order, as well as handling the empty case.
Fixes: #677
commit ac4792423975589beaf8a8dd13e90f0a4b6159fc
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 13 12:04:56 2025 +0000
Introduce MultipleArtifactInfo
This gives us somewhere to put helper methods.
commit b62a57e98f16b5d9084aa1325bcdffc276b733fd
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Jan 13 10:58:34 2025 +0000
Expand TaskDatabase docstrings
commit 64a6afe744ce9cca958bf22682fc920ed53e37ce
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Jan 12 17:52:44 2025 +0000
Extend TaskDatabaseInterface to include artifact category and data
This will allow tasks to include more useful information in dynamic data
that they can't do using only artifact IDs.
Doing this does mean that artifact data is now validated in more places,
so it's possible for this to result in new failures for old artifacts.
In practice this should mostly affect tests (and indeed I had to fix
several of those) since production code normally already goes through
some kind of validation of artifact data; if we encounter failures on
production then we can decide whether to fix artifacts or relax
validation.
Part of #510.
Automatic checks made by the Debian l10n team found some issues with the translations contained in this package. You should check the l10n status report for more information.
Issues can be things such as missing translations, problematic translated strings, outdated PO files, unknown languages, etc.
![[bug history graph]](https://qa.debian.org/data/bts/graphs/d/debusine.png){kind=link}