vcswatch reports that
this package seems to have new commits in its VCS but has
not yet updated debian/changelog. You should consider updating
the Debian changelog and uploading this new version into the archive.
Here are the relevant commit logs:
commit d4fc328b4f0dd1ba0b3c8b71d12ad9bbbb06025e
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 20:31:41 2026 +0200
A refused allocation loses the buffer the charset converters were growing (#1703)
`realloct` in `src/htssafe.h` is a plain `realloc` alias. A site that
assigns its result back over the pointer it passed in loses the old
block when the allocation fails. Five places did that.
`src/htsselftest.c`'s hashtable reader then used the NULL it had just
stored.
Four of them now `assertf`. The exported `hts_convertStringToUTF8` must
not kill its embedder, per #1688. So the two sites it reaches through
`hts_convertStringCharset` free the buffer and return NULL, and
`src/htscharset.h` says so at the declaration. Two siblings on the same
helper get that for free.
`-#test=ucs4-oom` starves the growth under `RLIMIT_AS` and requires an
abort. It covers `INCREASE_CAPA()` alone. A 1 MiB cap also breaks
iconv's gconv module load, and the probe would read that legitimate
"charset not found" return as a missing abort.
Closes #1705
Finding originally reported by @sparktron in #1693 (closed).
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 13eaf0453dc0faf88c40cb7302993298b091cbee
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 19:37:48 2026 +0200
The SIGPIPE self-test fails on macOS when the socket reset arrives late (#1711)
`471_engine-sigpipe.test` failed twice in 17 macOS arm64 runs, and never
on Linux. The self-test resets the peer socket, writes 64 times, and
requires one of those writes to fail, but nothing waits for the reset to
arrive. macOS hands loopback input to its own thread. On a loaded 3-core
runner all 64 writes can finish before the kernel processes the RST, and
then none of them fails. Linux drains that backlog in the sending
thread, which is why only macOS saw this.
The self-test now waits up to 10 seconds for the socket to report the
reset. It consumes the `ECONNRESET` with a read, then requires the next
write to fail.
Three mutants still red the test:
- `sendc()` without `HTS_MSG_NOSIGNAL` dies on SIGPIPE.
- `sendc()` that reports success on a failed write aborts.
- A peer that is never reset reds after the 10-second bound rather than
hanging.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 4eb206eb08e60c602bd6c8a5fc967140327d34b5
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 18:20:15 2026 +0200
A robots.txt group naming httrack never ends when the `*` group comes after it (#1702)
When a site writes a `User-agent: httrack` group and then a `User-agent:
*` group, httrack obeyed both. `robots_parse()` reset the group only
when it was not already locked to ours, so the wildcard group's
`Disallow` lines were appended to ours. Pages the site allows us were
skipped, and nothing in the log said so. RFC 9309 2.2.1 makes the groups
exclusive: once a group names us, no generic group applies.
The parser now tracks where a group ends. A user-agent line that follows
a rule line opens a new group, and consecutive user-agent lines name one
group. The same rule fixes two cases where master crawled what the site
had forbidden us. A second agent name on our group discarded that group,
and two groups both naming us kept only the last. So the bug runs in
both directions.
A blank line does not end a group. RFC 9309's ABNF allows an empty line
inside the user-agent run, and Google's reference parser agrees. So
`User-agent: httrack`, a blank line, then `User-agent: *` is one group,
and its rules do bind us. An assertion pins it, since that is the case
most likely to be read the other way.
The fixture disallows `/secret` for httrack and `/public` for everyone.
The master binary mirrors neither page, the fixed one mirrors
`/public/page.html`. `st_robots()` gains the group orderings it was
missing. Five of its new assertions fail on master, and it catches four
mutants, one of which wrongly starts allowing rather than blocking. No
existing assertion had to be inverted.
Closes #1695
Finding originally reported by @sparktron in #1693 (closed).
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 771e838e640da0a27c8cf82d4fed8e9b7734426e
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 17:14:46 2026 +0200
Two worker threads that fail at the same time print each other's error message (#1707)
`strerror()` answers with a buffer the caller does not own. The engine
calls it on the socket, cache and logging paths, all of which run on
crawler workers.
`hts_strerror()` now copies the message into the caller's buffer through
`strerror_r()`, or through `strerror_s()` on MSVC. Which `strerror_r` is
in scope is decided in C on `__USE_GNU`, not by `AC_FUNC_STRERROR_R`.
That probe defines `_GNU_SOURCE`, while `htsglobal.h` reaches
`<stdio.h>` before `config.h`, so the tree always compiles against the
XSI one and the two answers disagree. A MinGW build has neither form and
keeps plain `strerror()`. The self-test says so when it runs there.
Separately, a `--disable-ipv6` build resolves with `gethostbyname()`,
whose `struct hostent` is shared, and every resolve runs on its own
worker thread. That call and the copy out of it now run under one lock.
The arm cannot use `getaddrinfo()`, because configure also defines
`HTS_INET6` as 0 when libc has none.
`-#test=strerror` drives eight threads and eight codes. No CI leg
compiles the resolver arm, so the test guards that one at the source.
`htsweb.c` keeps its one-shot `strerror()`, because `htsserver` links
the library and `hts_strerror` is hidden.
Closes #1697
Finding originally reported by @sparktron in #1693 (closed).
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 322838fb65bbf5ac653c1816244712f058250a96
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 16:44:11 2026 +0200
The --catchurl capture proxy listens on the LAN address, not on this machine only (#1701)
`httrack --catchurl` opens a throwaway proxy so the user can point a
browser at it and hand the engine a URL, cookies and all.
`catch_url_init()` bound it to whatever address `gethostname()` resolved
to, which on most machines is the LAN or the public address. Anyone who
can reach that port can take the slot before the browser does.
It now binds `127.0.0.1`, and the address printed as "Proxy's address"
follows it, so the browser still finds the proxy. No IPv6 is lost.
`hts_dns_resolve_nocache()` takes the first `getaddrinfo` result, so on
a dual-stack host the old code bound the global v6 address rather than
`::1`.
The port was read from the requested address rather than the bound one,
so the ephemeral fallback in `catch_url_init_std()`'s port list reported
0. It comes from `getsockname()` now.
The new `catchurl-bind` self-test, in test 438, checks both entry points
and captures a request over the socket. The `--disable-ipv6` build was
checked too.
Finding originally reported by @sparktron in #1693 (closed).
Closes #1696
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit f82c11bf1d114fe2a1454a71bac4d7f5864f5e9f
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 16:35:11 2026 +0200
A cache repair writes to a file it failed to open (#1699)
`unzRepair()` in the vendored `src/minizip/mztools.c` opens three files:
the damaged archive, the repaired archive, and a temporary file for the
central directory. It checks the first two and writes to the third
regardless, so a failed third open sends every central-directory
`fwrite` through a NULL `FILE*`. That same `else` branch leaves the
handles that did open unclosed. `cache_repair()` runs it whenever
`hts-cache/old.zip` fails to open, so a corrupt cache plus a descriptor
shortage is enough.
I reproduced the crash at `mztools.c:165` with an uncreatable temporary
path, and again with `RLIMIT_NOFILE` lowered. The fix checks the third
handle, then closes what opened and removes the partial files. Upstream
minizip does not remove them. Every path here is engine-owned scratch,
but another caller could lose a file it owns.
The new `-#test=zip-repair-openfail` self-test fails each of the three
opens in turn and checks that nothing is left behind and no descriptor
leaks. `mztools.c.diff` is regenerated, or the next re-sync from
upstream drops the fix.
Closes #1698
Finding originally reported by @sparktron in #1693 (closed).
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 024123c567bbb78cf34c9ba3fa4048f832f4252b
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 11:30:09 2026 +0200
libhttrack can kill its host process with SIGPIPE (#1691)
Only `httrack.c`, `htsserver.c` and `htsweb.c` ever set a SIGPIPE
disposition, so every other embedder died on a write to a peer that had
gone. The FreeBSD canary found it through `tests/mirrorverdict`.
A library must not touch its host's signals, so this stops raising the
signal rather than swallowing it. `HTS_MSG_NOSIGNAL` goes on every
`send()`, `SO_NOSIGPIPE` on every socket we write to, and a
thread-scoped mask around the four OpenSSL calls that write. Each covers
a gap the others leave: Linux has no such socket option, and macOS lacks
`MSG_NOSIGNAL`, which OpenSSL's own `write(2)` never sees anyway.
The mask drains with `sigtimedwait()`, because `sigwait()` hangs for
good when another thread takes a process-directed SIGPIPE first. Darwin
has no `sigtimedwait` and does have `SO_NOSIGPIPE`, so the mask is a
no-op there.
A custom OpenSSL BIO was the other candidate. It works, but
`BIO_meth_get_*` is absent under the `-DOPENSSL_NO_DEPRECATED` leg we
build, so it costs 86 lines reimplementing `bss_sock.c` on the
non-blocking crawl path.
`-#test=sigpipe` restores the default action before it writes, which is
what lets a regression reach the test as a signal. Six mutants kill it,
and two source checks name any OpenSSL call that loses its mask or any
`send()` that loses the flag.
Closes #1689
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 6eb345cba9fbcdd1519207a1a62c8f15875fbeb2
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 10:15:04 2026 +0200
Tests 436, 445 and 459 throw away the engine log that names why a mirror stopped (#1692)
Three WSL2 jobs since 10 September left a four-line crawl log and
nothing else, stopping at `Mirror not completed (see log file)`. The
engine prints that line whenever a mirror did not complete, and it
points at `hts-log.txt` for the reason. Test 436 dumps the crawl log and
then removes the output directory, so the file it names is gone. Tests
445 and 459 drop the same engine log.
Left alone the same crawl completes and prints `Done.`, so
`--timeout=30` does not reap a two-byte-a-second transfer. Pointing the
arm at a dead port reproduces the four lines, and `hts-log.txt` then
names `"Connect Error" (-4)` on the seed.
The three tests now dump the engine log beside the crawl log. Test 436
also waits for a trickle page to start transferring before it asks for
the abort. A mirror that ends before any transfer starts fails on that
wait, and the dump carries the reason.
Closes #1680
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 27d068e26a8db2bbf3bc4f9b8606bbce18321c9b
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 06:21:18 2026 +0200
The FreeBSD canary builds with base make and keeps two gaps it has closed (#1690)
Four things red this weekly job, and only the last is an engine bug.
The prepare step symlinks gmake over `/usr/local/bin/make`, but the
login PATH puts `/usr/bin` first, so every run so far built with base
make. Test 397 noticed, because its `ifeq` fixture is GNU make syntax.
The port builds with gmake, so `build.sh` now puts `/usr/local/bin`
first.
144 and 215 leave the known list. #1507 gave `hts_self_path` a BSD arm,
and the canary reds on a known entry that passes again.
Test 257 gated a block on `test -d /proc`, and FreeBSD ships `/proc`
empty, so the read failed. Test 425 looked up `bash` in `/usr/bin` and
`/bin`, and FreeBSD keeps it in `/usr/local/bin`. Each now tests what it
is about to use.
470 still fails, and the test discarded mirrorverdict's exit status so
it could not say why. It now names the signal, which is SIGPIPE, filed
as #1689.
Dispatched on this branch the canary passes 445 of 492, up from 443,
without 257, 397 or 425 among its failures.
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 6de3ed40ed294c087d7e48994875b05015f932ca
Author: Xavier Roche <roche@httrack.com>
Date: Thu Sep 17 06:21:15 2026 +0200
The drift canary dies when a packager deletes a patch it watches (#1688)
Termux deleted all six patches it carried over this tree in its 3.50.1
bump, which is the outcome we asked them for. The canary has been red on
it since 9 September, because `carried_patch` fetched each patch through
`fetch`, which aborts the run on any curl failure.
A deleted patch is now a finding that says "prune this check", the way
`expect` already treats a premise that is gone. Any other HTTP status
still aborts, so an unreachable host cannot read as a dropped patch. The
check floor drops from 21 to 15 with the six Termux ids.
`./bootstrap` was in the workflow only for those patches, which targeted
generated files. Nothing left in the table reads a build product, so
that step and the autotools install go with it.
The script now runs 15 checks and exits 0. Against live URLs a 404
reports DRIFT, a 500 still aborts, and an empty 200 still aborts.
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit d75873b39dfd9858df141fbc9bdfb242980bd469
Author: Xavier Roche <roche@httrack.com>
Date: Wed Sep 16 10:52:46 2026 +0200
The mirror footer is built with unbounded sprintf into a fixed buffer (#1686)
The mirror footer is a fixed `char[8192]` that seven successive
`sprintf(finalInfo + strlen(finalInfo), ...)` calls appended to, none of
which could see the end of the buffer. `infoupdated[256]` was written
the same way. The widest possible footer is under 800 bytes, so nothing
reaching it today can overflow it. This is hardening, not a fix.
`-Wformat-truncation` fires only on a bounded `snprintf` whose return is
discarded. An unbounded `sprintf` never reaches compiler output, so a
pass scoped off warnings misses this function entirely.
The appends go through `slcatprintfbuff_clip()`, added beside
`slprintfbuff_clip()` in `htssafe.h`. That family already has this shape
for text meant to be clipped rather than dropped whole. It shares
`vslprintfbuff()` as its engine, so it inherits the `dest[size - 1]`
terminator that pre-C99 `_vsnprintf` needs. `infoupdated` moves to
`slprintfbuff_clip()` for the same reason, because `_WIN32` rewrites
`snprintf` to `_snprintf`.
The footer text does not change. The same crawl through this build and
through master gives byte-identical footers. That was checked on a plain
run, an aborted run, both arms of an update, compression stats and
keep-alive.
The compression-ratio and requests-per-connection lines are in no test.
That predates this change, which only routes them through a checked
helper.
Closes #1685
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit f6426310d8bbbd376dac70bffe444b1d147bfc06
Author: Xavier Roche <roche@httrack.com>
Date: Tue Sep 15 19:48:28 2026 +0200
A mirror run with no log file reports zero errors, however many it hit (#1683)
`stat_errors`, `stat_warnings` and `stat_infos` counted log lines. A
front end that turns the log file off read 0 errors after a crawl where
every link failed. httrack-android emits `-Q` for that and its finish
pane printed `Success! (0 errors)`.
`hts_log_vprint()` now runs the level test above the `opt->log` gate and
counts the event there. Counts are unchanged when a log file is open,
because counting still happens after the verbosity filter.
The counting moved out of `fspc()` into `fspc_count()` rather than
losing its `fp != NULL` guard. `fp == NULL` is `fspc()`'s read path, so
an unguarded increment would count every read, and `hts_get_stats()` is
polled during a mirror. The self-test pins that.
A panic now counts as an error. `LOG_PANIC` mapped to a string no
counter matched, so a mirror that died on a fatal condition also
published 0 errors. That changes `stat_errors` for runs that panic, with
or without a log file.
Found by the httrack-android session while reviewing its PR #221.
Closes #1681
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 12706f4b62234750d4f61983418875431afe9396
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue Sep 15 16:50:01 2026 +0000
Bump github/codeql-action from 4.37.9 to 4.38.0 (#1679)
commit 6f46e95632ec4c14a78cca1d8f55fe4b4cca799f
Author: Xavier Roche <roche@httrack.com>
Date: Tue Sep 15 18:12:27 2026 +0200
Test 258 removes a directory with rm -rf, which drvfs leaves half-done (#1682)
The last case of `258_crawllib.test` removes the crawl output while
`write_lock_request` retries, and expects the helper to name the
removal. On the WSL2 Windows leg, `rm -rf` deletes the contents and
leaves the directory behind (#1639). The blocking child went with it, so
the retry wrote the request, and the case failed for a removal it never
saw. Both open dependabot bumps went red on it on 15 September, while
master passed.
A rename takes the directory in one step, so nothing survives for the
write to land in. Control: with the helper's removal check dropped, the
case still fails.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit c95e91ac54f00bed3467b80a64a75580279ca62a
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue Sep 15 15:59:13 2026 +0000
Bump vmactions/freebsd-vm from 1.5.5 to 1.5.6 (#1678)
commit 9b15ccd0ff479da817ee99044d9e50aa8c5658bb
Author: Xavier Roche <roche@httrack.com>
Date: Tue Sep 15 13:51:55 2026 +0200
A front end cannot tell a link the server refused from one whose transfer failed (#1676)
A front end cannot tell a link the server refused from one whose
transfer failed. `stat_errors` counts LOG_ERROR log lines, so a 404 and
a timeout differ only in a number. `-Q` writes no log and leaves it at
zero. An Android crawl firewalled by Doze times out on every link and
reports success over a holed mirror. WinHTTrack and WebHTTrack share the
blind spot.
This adds a per-mirror count of links given up on a failed transfer,
published as `stat_transport_failures` beside `stat_errors`. It keys on
`back_transfer_failed()`, the classifier the engine already uses for
that decision. robots.txt is excluded, because a host that never
answered it has lost no content.
Two things a reader should know. The count includes a body cut off after
a `200`, because the engine gives that the same status code as a refused
connection. No predicate on the code can separate them. A GUI polling
the loop callback sees the count only while the crawl continues past the
failure. A front end deciding at the end should read it there.
Both structs take a tail append and no member moves, measured by diffing
`offsetof` tables. `HTS_STAT` is an exported object and grows with it,
from 168 to 176 bytes. No installed header declares it, so reaching it
takes a hand-written extern. That is the exposure to weigh against the
ruling that no soname bump is needed.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit f4ee93c3f55e6f959cc755c546232d91aeaca0e0
Author: Xavier Roche <roche@httrack.com>
Date: Mon Sep 14 09:13:25 2026 +0200
Tests 436 and 459 give up when one write into their crawl directory is refused (#1675)
Three WSL2 jobs failed in `436_local-abort-lock` since 10 September, in
runs 34447609795, 34451707795 and 34487486427. Each was the only failing
test in its job, and all three report the same failure. A write of
`hts-abort.lock` into the crawl output directory was refused while that
directory was still there. The wait one line above had just read a file
inside it.
The review of PR #1631 dropped a retry here, because the evidence then
could not say whether anything had removed the directory. The walk added
by #1650 has now answered that. The helper retries for three seconds,
and reads the directory again after each failed write, so a removal is
named rather than waited out. It prints the try that landed on stderr,
which the Windows job uploads with the test log.
Test 258 covers three cases: the retry landing, the retry running out,
and a removal during the retries.
This is a mitigation and not a fix. Nothing here explains why drvfs
refuses a write into a directory it holds, and no upstream ticket is
filed. Two of the three jobs also show that the engine had ended its
mirror by the time of the failure. That is unexplained, and it belongs
on the issue.
Closes #1639
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 5f71d45c689138486a5db281ea786a8e6b48421d
Author: Xavier Roche <roche@httrack.com>
Date: Sun Sep 13 20:53:37 2026 +0200
httrack-android cannot see the -#c crash-test markers (#1673)
The two `-#c` markers went to stderr only. httrack-android recovers a
worker fault through coffeecatch, and it reads the engine's log callback
rather than stderr. So it cannot tell a deliberate crash test from a
real crash.
Both markers now also go to `hts_log_vprint(NULL, LOG_ERROR, ...)`. NULL
matters, because the callback runs before the `opt != NULL` guard. A
worker reaches the front end's log without touching opt, which is what
the recovery design needs. A front end that captures both channels sees
the marker twice.
`01_engine-crash-announce` checks stderr, then arms a real `-#c=dnssegv`
and faults, so both markers are checked on the path Android recovers.
Reverting either call site fails it.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 5108864f3d978f7a964c4520628b8ba91b9973cd
Author: Xavier Roche <roche@httrack.com>
Date: Sun Sep 13 20:36:41 2026 +0200
A WSL2 interop outage reads as failing tests on the Windows leg (#1674)
When WSL2's `init` cannot reach the Windows side it writes
`UtilAcceptVsock:235: accept4 failed 110` and `httrack.exe` never
launches. Whichever tests called it read empty output and reported it as
a content failure of their own. PR #1672 collected three unrelated reds
that way, and they looked like a buffer bug.
The driver now reads the failing tests' own logs for that line. When
every failure carries it, the leg ends on 3. That is the status a lost
worker already ends on, and it means repeat this leg rather than
investigate it. The failing tests are still named either way, so an
outage cannot hide a real red that picked the line up for another
reason.
Test 469 covers the verdict, test 337 the driver's gate.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 03d67612dc1c1ffc64d8ed7c6c6fb883b87e3191
Author: Xavier Roche <roche@httrack.com>
Date: Sat Sep 12 18:26:59 2026 +0200
-#c cannot fault a live worker, so the recovery path is unreachable (#1672)
`-#c=threadsegv` and `-#c=threadstack` cannot reach the engine's
fault-recovery path. The pre-pass runs the kind, then frees the option
set and returns −1 before any mirror starts. So there is no crawl
thread, and no engine worker to fault. A front end installs
`hts_set_thread_runner()` to recover a worker fault. Verifying #1671 on
a device took two hand-patched worker bodies for want of a way in.
`-#c=dnssegv` and `-#c=ftpsegv` arm a fault instead of taking one, and
the pre-pass lets the command line through so the mirror runs into it.
The resolver worker faults with its answer in hand and not yet
published. The FTP worker faults with its backlog slot already
registered. Those are the two states worth recovering from.
The option parser gains a `#c` case, because an armed kind now reaches
it.
Test 468 asserts what no other kind can: the project directory is on
disk when the fault lands, so the mirror was running. It resolves the
trace through addr2line to check the fault came from the worker's own
function, and it reads the signal by name. Its control is `-#c=segv`,
which leaves no project behind. Six mutants turn it red, one by arming
the control's own kind.
The httrack-android session asked for this, to replace its patches.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit a9bdf1c8ad6b0b9287cc09dc3c98fbe2cbbc3972
Author: Xavier Roche <roche@httrack.com>
Date: Sat Sep 12 13:07:29 2026 +0200
The crawl waits forever for an FTP worker a crash recovery cut short (#1671)
A crawl could hang forever at teardown. `back_launch_ftp()` took its
worker off the live-worker list on its last line. `ftp_stop_workers()`
polls for an empty list with no bound, so a worker that never reached
that line wedged the crawl thread.
A worker reaches it unless something cut its body short, which is what a
front end's thread runner does. httrack-android now installs one through
coffeecatch, to catch a crash signal on an engine worker and recover
from it instead of dying.
The thread layer now runs a tail callback once the body is over, whether
or not the body got there. FTP hands its slot back from that tail, and
the DNS resolver releases its job there.
A worker is not resumable after a crash, so the mirror ends rather than
failing the one link. The thread layer raises a flag, and the crawl
thread turns it into `HTS_EXIT_MIRROR_ABORTED`, being the only thread
that may touch `opt`. A worker carries the round it was spawned in, so
one an earlier mirror abandoned cannot abort this one.
Three selftests and seventeen mutants cover it. Two older paths dropped
that exit status on the way out, and one of them also swallowed the
cache-write abort.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 0ae7bcdb5e9f11281ccaee86973c84444d8fd442
Author: Xavier Roche <roche@httrack.com>
Date: Fri Sep 11 19:38:49 2026 +0200
Test 250 reads an exited process as still running on GNU/Hurd (#1668)
3.50.2-1 failed to build on hurd-amd64 and hurd-i386, and
`250_timeout-poll-nofork` was the only failure on both. 3.50.1-1 built,
so this is a regression from #1554, which added `pid_is_zombie`.
Neither of that helper's two routes works on GNU/Hurd. Its procfs builds
`/proc/PID/stat` out of Mach data an exited process no longer has, so
the read fails where Linux answers `Z`. The fallback `ps -o state= -p
PID` fails too, because Hurd's `ps` rejects `-p` and takes the pid
positionally. `kill -0` then keeps succeeding on the zombie until the
grace runs out.
The fix is the second of those. Hurd's `ps` does know the state, from
proc-info, which a zombie still carries, so `pid_state` now tries both
spellings and takes whatever the host says.
Four earlier drafts of this branch tried instead to read the failure
itself as proof of death, and review broke each one. The last one shows
why none of them could work. `/proc/PID/stat` needs `PSTAT_THREAD_WAIT`,
libps sets that only while the message port is usable, and
`should_suppress_msgport` turns it off for a suspended task. A live
process that is merely stopped reads EIO on Hurd, so no test on the file
can tell it from a dead one.
`pid_state_readable` rests on the same answer, which is what it always
claimed to report. A host neither route answers for now makes the test
skip rather than assert. The guard also runs after the kill now, because
asking about a live process is how the state Hurd cannot report came to
be graded.
One fix here is older than the branch. Linux writes `comm` into
`/proc/PID/stat` unescaped. A process named `") Z<LF>q"` therefore
splits the file, and the first line ends in a state it is not in.
Reading one line reported `Z` for a sleeping process.
`pid_state` takes one row from `ps` or none. A header, or an answer
covering several pids, says nothing about this one, and folding the rows
together invents a state no column held.
Fourteen mutants of the helper each fail a test, one of them a stderr
leak an earlier draft left behind.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 45e28c1876ecfeab23bb5209a59986b128a2e5e8
Author: Xavier Roche <roche@httrack.com>
Date: Fri Sep 11 16:44:02 2026 +0200
A Windows job whose runner vanishes needs a human to re-run it (#1669)
A `windows-build` job killed the way #1228 describes fails with no
failed step, no log and no artifact, so only a re-run clears it. Since
#1585 that is about one job a fortnight rather than one in eight, so
this covers the residual, not the old rate.
`tools/windows-kill-grade.sh` decides, and refuses anything ambiguous.
It collects every job that did not succeed and is not still running. All
of them must be plain failures, each with no named step and an
unfinished one, or no steps at all. Test 466 pins that with ten mutants,
each killed by a case it does not share.
The step re-reads `run_attempt` from the run rather than from the event,
and allows exactly one re-run. A superseded run is left alone, because
windows-build cancels in-progress runs of the same ref, so re-running
the old one would cancel the new one. A push landing between that check
and the POST still races it.
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit e61d483d412d1bca3513bd5dd12027ef7395e23f
Author: Xavier Roche <roche@httrack.com>
Date: Fri Sep 11 15:29:43 2026 +0200
windows-build.yml does not record that the MSYS suite leg was killing the runner (#1667)
The MSYS test-suite leg was killing the hosted runner. Removing it in
#1585 took the Windows runner-death rate from 12.5% to 0.3% over 1515
jobs. `windows-build.yml` records only that MSYS wedged, so nothing
there warns the next person off adding a leg back. The measurement is on
#1228.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 600adfd0d06d4c6cf4732f71f25b82ed62b135eb
Author: Xavier Roche <roche@httrack.com>
Date: Fri Sep 11 14:05:40 2026 +0200
Nothing tells a translator to be a native reader, or which files a new language needs (#1665)
Two gaps in `lang/README.md`, both found while answering a forum request
for a Thai translation.
It never states the rule that matters most. Only a native reader should
send a translation, and never machine or LLM output, because nobody here
can check one.
Its "Adding or updating a language" section is written for updating a
catalog that already exists. Follow it for a new language and the PR
fails tests 62, 225 and 373. Nothing told the contributor about
`lang.def`, `lang.indexes`, the two per-catalog pins under `tests/`,
`install-manifest.txt`, or the credits roster.
So the section is split by role. A translator sends one file, and the
maintainer steps move under their own heading.
Checked by staging a 31st catalog and running the full suite. That
turned up two traps, now written down. `lang.indexes` wants the ISO code
lowercased, and `373_credits.test` needs a `LABEL` row when
`LANGUAGE_WINDOWSID` is not the English name.
---------
Signed-off-by: Xavier Roche <xroche@gmail.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit b276d1bb83bc21c6bed3ddca8bf59b4f716635c8
Author: Xavier Roche <roche@httrack.com>
Date: Fri Sep 11 10:48:26 2026 +0200
hts_set_thread_runner() is exported but no installed header declares it (#1664)
The 3.50-2 release notes tell embedders about three calls, and
`hts_set_thread_runner()` was not reachable. The symbol is exported, but
its only declaration sat in `src/htsthread.h`, which no install carries,
so a caller had to write the prototype by hand.
The declaration now lives in `src/httrack-library.h`, which is
installed. Installing `htsthread.h` would drag `htswin32.h` in with it,
and installing `htstools.h` would publish internal link types. The
`hts_find*` family was already declared in both headers. This drops the
copy in `htstools.h` and lets that header include the public one. No
exported symbol is added or removed, so `VERSION_INFO` does not move.
The new test compiles and links a consumer against the installed headers
alone. Without the change it fails with `error: unknown type name
'hts_thread_runner'`. `html/library.html` needs no edit, because the
installed header list is unchanged.
Closes #1663
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
![[bug history graph]](https://qa.debian.org/data/bts/graphs/h/httrack.png){kind=link}