commit 0d7f8eacd6ff68d159133d73abf6b8243ace3d69
Merge: 26d03400b 2578d2993
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Mar 2 14:05:39 2025 +0000
Merge branch 'doc-dynamic-storage' into 'devel'
Move dynamic-storage blueprint to main documentation
See merge request freexian-team/debusine!1670
commit 26d03400bcf41559aea01de2e88a5cff32d7889f
Merge: cb255bd0e 2b82fbac5
Author: Stefano Rivera <stefanor@debian.org>
Date: Fri Feb 28 20:57:50 2025 +0000
Merge branch 'dynamic-worker-pools-management' into 'devel'
Dynamic worker pools management
See merge request freexian-team/debusine!1666
commit 2578d2993aa5d8c97ea36f430eb6686eceff164f
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Feb 28 17:20:14 2025 +0000
Move dynamic-storage blueprint to main documentation
I added "File stores" explanation and reference sections to cover this,
and included details of the file backend configuration models.
Part of #541.
commit cb255bd0ed79c42ac71bf20bceb2692ddd9ed3c1
Merge: 8ad5cd448 2ce1492e4
Author: Enrico Zini <enrico@debian.org>
Date: Fri Feb 28 15:18:05 2025 +0000
Merge branch 'worker-manager-method' into 'devel'
Use self instead of Worker.objects in WorkerManager
See merge request freexian-team/debusine!1667
commit 2ce1492e41c85370d09bd7f3120d3de0fcd7e041
Author: Enrico Zini <enrico@enricozini.org>
Date: Fri Feb 28 14:49:35 2025 +0100
Use self instead of Worker.objects in WorkerManager
commit 8ad5cd448740b032b8a169f6a8d43f6c14e360d7
Merge: 09f772311 7bfdc6d4f
Author: Enrico Zini <enrico@debian.org>
Date: Fri Feb 28 14:10:47 2025 +0000
Merge branch 'schedule-static-workers-first' into 'devel'
Fill static workers before scheduling on cloud workers (#721)
See merge request freexian-team/debusine!1665
commit 7bfdc6d4f2c5790809ebaf5fb90f72cc25fb89c3
Author: Enrico Zini <enrico@enricozini.org>
Date: Fri Feb 28 14:29:56 2025 +0100
Fill static workers before scheduling on cloud workers
commit 09f772311781f526af3969e235400fe98ff01716
Merge: 879dcec22 96f85a4ff
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Feb 28 13:15:51 2025 +0000
Merge branch 'translation-pt_BR' into 'devel'
Add Brazilian Portuguese translation
See merge request freexian-team/debusine!1664
commit 879dcec2261801441faa2a1daf4918e2d189ac0e
Merge: 938e04ae2 40db1a127
Author: Enrico Zini <enrico@debian.org>
Date: Fri Feb 28 12:42:02 2025 +0000
Merge branch 'group-admin' into 'devel'
Add role for user in group (#697)
See merge request freexian-team/debusine!1611
commit 938e04ae2f2ab106c73f079fa2aaed60dfc012bd
Merge: 315d14e73 ade07d1ba
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Feb 28 12:18:50 2025 +0000
Merge branch 'vacuum-storage-soft-max-size' into 'devel'
Implement soft_max_size and max_size limits in vacuum_storage
Closes #684
See merge request freexian-team/debusine!1662
commit 96f85a4ff0ecbf304e6a13a383ced187de105a95
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Feb 28 12:09:14 2025 +0000
Add Brazilian Portuguese translation
Thanks, Paulo Henrique de Lima Santana.
Closes: #1098447
commit 40db1a12787896fcb72d523b1b9fb10e591d81c3
Author: Enrico Zini <enrico@enricozini.org>
Date: Thu Feb 6 16:01:24 2025 +0100
Add user role in group
commit b461583e0f3e9e2a97a22a80bef2f00a604896eb
Author: Enrico Zini <enrico@enricozini.org>
Date: Thu Feb 6 15:40:38 2025 +0100
Convert group-user m2m to use a through table
commit ade07d1bafd73e88bd16581f0d168940b35bc3aa
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Feb 27 15:31:28 2025 +0000
Implement soft_max_size and max_size limits in vacuum_storage
I changed the definition of `soft_max_size` slightly, since we don't
keep records of when files were used, only when their containing
artifacts were created. That seems close enough.
Fixes: #684
commit 61b98364f198d53d7b087cb050544246aea497d1
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Feb 28 00:46:21 2025 +0000
Add a migration utility to rename a table and its associated relations
Maybe it's possible to extract the necessary information from the Django
migrator state, but I couldn't figure out how, and I think doing it in
SQL - complex though it is - is actually clearer. The result for this
migration is:
debusine=*> \d db_groupmembership
Table "public.db_groupmembership"
Column | Type | Collation | Nullable | Default
----------+--------+-----------+----------+------------------------------------------------
id | bigint | | not null | nextval('db_groupmembership_id_seq'::regclass)
group_id | bigint | | not null |
user_id | bigint | | not null |
Indexes:
"db_groupmembership_pkey" PRIMARY KEY, btree (id)
"db_groupmembership_group_id_01df2747" btree (group_id)
"db_groupmembership_group_id_user_id_a71f1436_uniq" UNIQUE CONSTRAINT, btree (group_id, user_id)
"db_groupmembership_user_id_88dea0fd" btree (user_id)
Foreign-key constraints:
"db_groupmembership_group_id_01df2747_fk_db_group_id" FOREIGN KEY (group_id) REFERENCES db_group(id) DEFERRABLE INITIALLY DEFERRED
"db_groupmembership_user_id_88dea0fd_fk_db_user_id" FOREIGN KEY (user_id) REFERENCES db_user(id) DEFERRABLE INITIALLY DEFERRED
commit 2b82fbac5a4738f2fd2e2ca67cbacbfeb22ac235
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Feb 27 19:52:18 2025 -0400
Management command to manage worker pools
commit a738f48169c703ad4c5b0a7fdc4a66e27e45804e
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Feb 27 14:44:33 2025 -0400
Add WorkerPoolManager with an enabled query
commit 6e23c16d32d74c23289c9b51df0a93fe45919431
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Feb 27 19:43:11 2025 -0400
Playground: Allow linking workers to pools
commit a6f838340329ed47102a763a2c1e7abc87e57f89
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Feb 27 14:12:40 2025 -0400
Playground helper to create worker pools
commit 315d14e73500de297ba0d4a51152ae5ff85f0e0e
Merge: 2d4f4a87f 20fab04ae
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Feb 27 15:23:57 2025 +0000
Merge branch 'vacuum-storage-drain' into 'devel'
Implement drain storage policy in vacuum_storage
See merge request freexian-team/debusine!1660
commit 20fab04aea8b8b7330d7a9dfe9fa94f4264fa080
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Feb 27 10:56:21 2025 +0000
Implement drain storage policy in vacuum_storage
Part of #684.
commit 424a1d2eb33c2e9d5abbf20de36b1820a9979e31
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Feb 27 10:44:20 2025 +0000
Only upload to write-only stores when applying the populate policy
In principle, a write-only store can accept uploads. In practice, it
doesn't make sense to do so in most situations: uploading the only copy
of a file to a store only to find that it cannot be downloaded again is
unlikely to be desirable. Limit uploads to these stores to the case
where we're applying the `populate` policy, since that makes sense for
long-term archival.
commit 2d4f4a87faa78384231b8a0c848c77fc8c292c24
Merge: 29b4cc151 12498d44e
Author: Stefano Rivera <stefanor@debian.org>
Date: Wed Feb 26 21:46:14 2025 +0000
Merge branch 'dynamic-worker-pools' into 'devel'
Dynamic Worker Pool models
See merge request freexian-team/debusine!1650
commit 12498d44eb1144290892b251267ddb8bd7876111
Author: Stefano Rivera <stefano@freexian.com>
Date: Sun Feb 23 11:05:51 2025 -0400
Worker Pool statistics models
With some minor design changes.
commit 02e64d7f1a5ca0026489474308f128610cb5888d
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Feb 22 15:03:15 2025 -0400
Worker Pool database models
commit cbfc7b8844f7dd4ed53682f07aad100269cb2ad6
Author: Stefano Rivera <stefano@freexian.com>
Date: Sat Feb 22 16:15:49 2025 -0400
Pydantic models for Worker Pools
commit 29b4cc1515a9e4c5eaadc05a3e30e61bf1183cf3
Merge: 959c115c8 9468fffce
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Feb 26 18:40:22 2025 +0000
Merge branch 'vacuum-storage-populate' into 'devel'
Implement populate storage policy in vacuum_storage
See merge request freexian-team/debusine!1659
commit 9468fffceeacbf8a409c8373aababb5ae106c63f
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Feb 26 00:44:36 2025 +0000
Run vacuum_storage with higher verbosity
It's usually helpful for logs to have more details about progress.
commit 60ed3e456af32cd4255d8895f9ba984a64850ea5
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Feb 26 00:44:23 2025 +0000
Implement populate storage policy in vacuum_storage
Part of #684.
commit b240d2f60aadd10bc419cad3191316006b23aa91
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 22:35:53 2025 +0000
Add optional s3_endpoint_url to AWSProviderAccountConfiguration
I ran into a case where presigned URLs for a recently-created bucket
didn't work because (I think) the bucket hadn't propagated everywhere
yet and so I was getting temporary redirects which broke the request
signature. Explicitly passing an endpoint URL seems to allow working
around this, and it seems a reasonable thing to expose as an option.
commit 6380dd0bf3a062d501466b4871592da3f54f7186
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 20:30:22 2025 +0000
Fix S3FileBackend.list_entries for empty buckets
Experimentation reveals that S3 returns a response without a `Contents`
key in this case.
commit 4d9b6ff9addbb57465b6e58c207599af39d54634
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 18:01:24 2025 +0000
Allow Scope.upload_file_stores to enforce soft limits
Uploads of new files are explicitly allowed to exceed soft limits, but
`vacuum_storage` needs a way to do similar queries for available file
stores while also enforcing soft limits. Add an option allowing it to
do so.
I tried to figure out how to enforce `FileStoreInScope.soft_max_size`
here, but working out the total size to compare against is difficult.
We can come back to that later.
commit 0e9c7c92856480529c27b0c560ec72bf2521e01c
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 17:55:19 2025 +0000
Forbid nonsensical combinations of FileStoreInScope policies
commit cd23806ec8212d07452aed63e8ff9bc2164d3742
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 17:26:11 2025 +0000
Add FileBackendInterface.get_temporary_local_path
Some callers want to be able to get a local copy of a file even if it
isn't normally available locally. The existing `get_local_path` and
`get_url` methods are inconvenient for this because they're both
optional. `get_stream` works, but it makes an unnecessary copy if the
file is already in local storage, and for S3 it may be less efficient
than allowing `boto3` to do a multipart download.
Add `get_temporary_local_path`, which is a context manager so it can
remove a temporary file if it needs to make one, and which can be
implemented in whichever way is most efficient for the backend.
commit 2648573a182ddb0351dd807fa775ebcae0bf3d2c
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 17:19:04 2025 +0000
Add boto3-stubs[s3]
This provides stricter annotations for the S3 client.
commit 959c115c87f671cf32bad50edd9ad9b680349031
Merge: b262c0313 23baa52fe
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 19:36:51 2025 +0000
Merge branch 'vacuum-storage-transactions' into 'devel'
Run vacuum_storage's DB stages in transactions
See merge request freexian-team/debusine!1655
commit 23baa52fe83641bb2f9814e7a4c11d45ca9a2494
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Feb 25 10:14:00 2025 +0000
Run vacuum_storage's DB stages in transactions
Otherwise it's possible for them to race with other cleanup processes
and find that files retrieved from previous queries now don't exist.
Automatic checks made by the Debian l10n team found some issues with the translations contained in this package. You should check the l10n status report for more information.
Issues can be things such as missing translations, problematic translated strings, outdated PO files, unknown languages, etc.
![[bug history graph]](https://qa.debian.org/data/bts/graphs/d/debusine.png){kind=link}