vcswatch reports that
this package seems to have new commits in its VCS but has
not yet updated debian/changelog. You should consider updating
the Debian changelog and uploading this new version into the archive.
Here are the relevant commit logs:
commit de8886b4c5afe426ce38e1d6074853a9faf88480
Author: Xavier Roche <roche@httrack.com>
Date: Sat Aug 15 12:22:05 2026 +0200
A filter rule longer than the matcher's cap is stored and never matches (#1283)
* Refuse a filter rule the matcher would never look at
strjoker() ignores any pattern past STRJOKER_MAXLEN, but nothing told the
code that builds a filter about that cap: a longer rule was accepted, stored
and then skipped on every match, with no warning and no way to tell it apart
from a rule that simply does not apply.
Route every write into the filter array through one filters_insert(), which
drops an over-long rule with a warning naming the length and the limit rather
than storing one that cannot fire. The wizard declines the rule and carries
on, so a link too long to filter still gets its verdict.
Closes #1270
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Review pass: collapse the slot-stride duplication, pin the caps, tighten the test
filters_init() hardcoded the slot stride as (HTS_URLMAXSIZE * 2) while
htswizard.h mirrored it as HTS_FILTER_SLOT_SIZE with a comment asking a human
to keep the two in sync. Now that filters_insert() copies into a slot using
that macro, a widened macro would have written past the real stride without
tripping anything, so the constant moves next to filters_init() in htscore.h
and the allocation strides by it.
Two compile-time pins keep HTS_FILTER_MAXLEN inside both caps it is derived
from, so a hardcoded limit cannot silently stop being either.
The self-test now requires the warning to name the rule's own length and quote
the rule, and st_wizardinsert asserts the count hts_wizard_insert_filters()
returns, which this branch changed from filters emitted to filters stored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 838f60657f8f3cfa7a4b32e76d23b450677ee86b
Author: Xavier Roche <roche@httrack.com>
Date: Sat Aug 15 09:15:59 2026 +0200
A wizard answer the engine cannot honour decides one link and records nothing (#1259)
Two wizard answers decided the single link in front of them, recorded no rule and said nothing, so the next link on the same host asked the same question again.
An unreadable reply reaches the engine as -999 and left the verdict undecided, which `htsparse` reads as allowed. A reply the prompt cannot parse therefore mirrored the link, leaving no trace beyond a "What did you say?" on the console. It now refuses, matching the empty reply the prompt documents as the default, and records `-host/file` so the same link is never asked about twice.
A host-scope answer (#1239) that `hts_wizard_host_scope` cannot resolve had the same shape. An IP literal, a single label, or an index past the last scope offered got a verdict and no filter. No domain lives below such a host, so the answer now applies to the host itself: that is exactly what the user asked for in three of the four cases, and a conservative narrowing in the fourth. A warning names the fallback either way.
A long link still kills the run, and that is deliberate. `adr` and `fil` are each 2 KB and the wizard joins them into two buffers of the same size, 160 lines before any prompt exists, so `strcatbuff` aborts on a scheme-relative link the parser accepts. I first sized those buffers to fit the pair, and that was wrong. The abort is the only thing keeping such a link out of `url_savename_addstr`, an unbounded append with no destination size that writes eight bytes past `afs.save` inside `htsparse`'s frame, silently, with exit 0 and no warning under -O2. An abort is a controlled failure and that overflow is not, so the bound stays and now carries a comment saying why. The overflow is filed as #1269.
The prompt itself still clips, which is what AGENTS.md asks for on anything read off the wire, and `-#test=wizardprompt` pins the string it builds, separator included, since that is what every front end puts in front of the user.
Answer 4, "get this page only", was named in the issue and is not a bug. It asks again only about a different URL, which the user never spoke about; the same URL is asked once. I have corrected the issue text. The slash-less `fil` case it names is unreachable: every link form I could feed the parser arrives with a leading slash.
Test 298 covers the three answer paths.
Closes #1251
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
commit 2cbcd4d36abde621be3d490ba94dec455e37ced5
Author: Xavier Roche <roche@httrack.com>
Date: Sat Aug 15 08:41:35 2026 +0200
A doit.log argument that begins with a quote is stripped twice
Arguments reloaded from `hts-cache/doit.log` reach the parser already unquoted, because `next_token` does that as it splits them. The "Treat parameters" pass then strips a quote pair off any argument whose first character is a quote, and panics when there is no closing one. So a plain argument, a URL or a filter, whose value legitimately begins with a quote either lost it or aborted the reprise. The abort runs before the writer, so doit.log kept its contents and every later no-argument run failed the same way. The mirror could only be resumed by passing its URL again.
Command-line arguments meet that pass once and reloaded ones meet it a second time, so no encoding the writer picks is right for both readers. #1261 made the writer a faithful codec, which is what brought this into view: the token now survives intact and is then stripped again.
The token block carries the provenance instead. `cmdl_argv` grows a per-slot flag and `cmdl_ins_unquoted` sets it for tokens whose reader has already unquoted them, so the pass skips those. `htsalias.h` is not installed, so this is internal. Option values were never affected: the option switch consumes them with `na++`, so the pass never saw them, which is why they round-tripped while plain arguments did not.
Section 5 of `01_engine-doitlog.test` covers it. It asserts doit.log is a fixed point of its own replay rather than checking one token, so a reader and writer that agree on a wrong value still fail. The eleven-shape option-value differential is unchanged from #1261.
Closes #1263
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit c0d94f867ffe11cd534ca99da350bc0e3cba1da9
Author: Xavier Roche <roche@httrack.com>
Date: Sat Aug 15 00:50:02 2026 +0200
The wizard-precedence comments restate the code (#1272)
Review of #1257 flagged seven comment blocks that restate the code or narrate the change rather than saying why. This trims them to the house one-line default: a five-line re-listing of httrack.c's query3 answer codes, an aside about where the scenarios abort, wording that called the counter a block size, and a six-line header on `hts_wizard_insert_filters`.
The contract facts stay. The log-echo caller reads back the range the function reports, so the return count and the placement of the inserted filters at the tail of the block are still documented at the declaration.
Comment-only: a whitespace-ignoring diff shows no code change.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit d26d7e44a2fcb17d2f599e65333d85f378c08bc6
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 23:38:33 2026 +0200
A wizard answer is outranked by every filter that came before it (#1257)
`fa_strjoker` lets the last matching filter win, but every wizard answer was inserted at index 0, at the bottom of the precedence order: the command line outranked it, and so did every earlier answer. Answer "ignore this link" and then "mirror the whole host" for the same host and the first link stays excluded, with no way to take it back.
The answers now form a block at the low indices of the filter array, each new one appended at the end of that block. A later answer beats an earlier one, which is how a user corrects a misclick, and the command-line filters stay above the block, so a standing `-*.zip` survives a "mirror this domain" answer given mid-crawl. `opt->wizard_filters` carries the block size, appended at the tail of `httrackp`, so no ABI break.
The primary link's own scope filter, recorded before any question is asked, is now the weakest entry of the block instead of the strongest, and a later answer can override it. Nothing relied on the old order: a link that filter matches is authorized before the question is reached, so the two never compete. The old "renversement wizard/primary filter" note is about answer 4 capping recursion depth, not about filter indices.
The insertion path is one function now, `hts_wizard_insert_filters()`, so test 297 asserts the ordering without a crawl. It goes red on the pre-fix tree, and on an append-at-the-end variant that would let an answer outrank the command line.
#1249 rewrites the same insert loop, so whichever of the two merges second has to make its log echo read the slot the filter was inserted at rather than `_FILTERS[0]`, or its claim that the log cannot drift from the applied filter stops holding.
Closes #1250
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
commit 57fa73ec9d05e7277cf5d0be316d800a18cedd54
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 23:35:18 2026 +0200
local_crawl cannot feed the engine's stdin (#1265)
`local_crawl` takes a `--stdin FILE` option, plus the word `closed` for a test that must hand the engine no descriptor at all. The redirect goes on the engine, not on the `local_crawl` call: `run_with_timeout` backgrounds the job, and where job control is off bash gives a background job `/dev/null` for stdin, so a redirect written on the call never arrives. Both wizard tests had rebuilt the helper's body around a wrapper carrying the redirect, which cost them its watchdog and `--max-time` backstop; they go back to the helper here.
`run_with_timeout` applies the redirect itself rather than through a wrapper function, so the engine is still its direct child and `kill_tree` signals the engine instead of a shell in between. 258 reads the three cases off a shim: a file, a closed descriptor, and the default. Dropping the forwarding reds 258 and 296; dropping the closed branch reds 258 and 294.
Closes #1258
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 417d4cc4674bbc54d89cc72f1904b29e817a74bc
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 18:05:37 2026 +0200
Test 295 expects a wizard verdict string that #1254 retired (#1267)
An answer that does not forbid now resolves to 0 in hts_wizard_apply_verdict(),
so an accepting answer logs "allowed" where the test still expected "no
verdict". Each PR was green alone; the pair is red.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 492c239dcde7dac814a74e0d049491cbb22141c6
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 16:39:06 2026 +0200
The credit tag in every mirrored page is dated 2014 (#1243)
* The credit tag in every mirrored page is dated 2014
HTTRACK_AFF_AUTHORS reads "[XR&CO'2014]" and lands in every mirrored page's
footer, every generated index and the version banner, so a year nobody has
maintained since 2014 is stamped on the output of a 2026 crawl. Drop the year
rather than move it: the next bump would go stale the same way, and the release
already dates itself through the footer's {date}.
The index templates carried their own copies, which had drifted apart in the
meantime: the compiled-in ones said 2014 and the shipped templates/ ones 2008,
with the web designer's name reversed between them. Both now credit 1998, the
start year the source headers already use, and the name matches the served
WebHTTrack pages.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Collapse the clang-format note to one line
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Keep the clang-format note under the column limit
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 2472f8f4172ccde00fd37cfacbb7e85e85ae1e95
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 16:30:24 2026 +0200
The per-option argument caps are literals the front ends have to copy
`htscoremain.c` held the caps for six option arguments as bare `254` and `62` literals at the option sites. The GUI front ends carry their own copies of three of them, and nothing links the two, so a changed cap shows up only when a user's mirror dies on a value the GUI accepted.
They move to `htsglobal.h`, which already reaches the front ends through `HTTrackInterface.h`, and keep the exclusive bound the existing checks use, so no boundary shifts by one. Naming all six rather than the three asked for avoids leaving a shared-looking `254` next to them for someone to cross-wire later.
No behavior change: every boundary was probed against master, one byte under and exactly at each cap, with identical results. `299_option-arg-caps.test` reads each limit out of the header rather than copying it, so it pins the engine against `htsglobal.h` instead of against a second copy of the numbers.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit f94da62bad889a5bb06b943a535c545b25e34115
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 16:03:24 2026 +0200
Strings added since 3.49 are still English in every catalog (#1255)
`LANG_M10` and `LANG_M11` join `LANG_M1`..`LANG_M9` for the wizard's two domain-scope answers, which #1239 left hardcoded in the CLI menu with no key for WinHTTrack (xroche/httrack-windows#96). They are worded exactly like that menu, and test 62 now requires both to appear verbatim in `English.txt`, `lang.def` and `src/httrack.c`, so the front ends cannot drift the way #1117 warned about.
Two other sets were still English everywhere: the host-alias field and tooltip #1166 copied into every catalog, and the fifteen Android strings that had only ever reached Danish, French and Brazilian Portuguese, which `tests/62_lang-integrity.test` waived by name. Both are translated across the thirty catalogs now and the waiver is gone, so the next gap reds the test.
Each file keeps its own on-disk charset and line endings. Uzbek stays LF, Romanian still spells s-comma, t-comma and a-breve with the Latin-1 lookalikes it uses throughout, and the catalogs whose charset has no typographic quote pair take ASCII ones. Three mutants prove test 62 catches drift: a dropped msgid, a stray UTF-8 byte in a Latin-1 file, and `LANG_M10` pointed at a msgid missing from `English.txt`.
A review pass over the translations themselves fixed a feminine pronoun standing in for the masculine host in the Czech, Slovak, Croatian and Slovene scope answers, plus per-language defects in nine more catalogs. `Chinese-BIG5` also spelled `Ignore all` with the wrong character in the same wizard block; that is corrected here.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 2294aa401962271bf4aa1a1325777649041b75f2
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 15:54:22 2026 +0200
The footer field names are copied by hand in three places (#1252)
The ten `-%F` footer field names were written out twice inside the formatter's own reach, in htsparse.c and in the htsselftest.c fixture, and a third time in WinHTTrack's preset validator with nothing to link against: `hts_footer_format` sits behind `HTS_INTERNAL_BYTECODE` and the field table was function-local. That third copy could catch a GUI typo and nothing else; an engine rename sailed straight past it.
The formatter's two callers now share one table in htstools.c and pass a positional `values[]` indexed by a new `hts_footer_field_id`, and front ends get `HTSEXT_API hts_boolean hts_footer_field_ok(const char *name)` to test a name against it. The export is additive and `hts_footer_format` is internal, so nothing about the ABI moves. The help text, man page and cmdguide.html still list the names by hand; that is #1256, not this PR.
Asked for by the httrack-windows session, which wires the GUI half once this lands.
Review caught the trap in the positional design and the second commit closes it: reordering the enum without touching the name table passed all 326 tests, because the named path resolves by name while the crawl assigns by enum symbol, so `{mime}` would have carried the charset. The fixture now fills `values[]` by id the way htsparse.c does, which makes the test's expected strings pin each name to its slot. A name missing from the table is a compile error instead of a NULL slot. Both mutants were confirmed to red, and a dropped name to fail the build.
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 4729035a6e7aaadde07c11495cb652907e2545ea
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 15:45:32 2026 +0200
doit.log writes a leading quote raw and swallows the options after it
httrack wrote an argument into `hts-cache/doit.log` unquoted and unescaped whenever its first character was already a quote. Against the raw command line that guard makes sense. But by the time the writer runs, the `-O` pass has stripped one layer of surrounding quotes in place, so a leading quote is data, not syntax. Written raw it opens a quote that never closes, and the next run pulls everything after it into that one argument.
A footer of `"<!-- x -->` loses every option recorded after `-%F`. The reprise then rewrites doit.log from the corrupted parse, so the loss carries into every later update, and the mirrored page footer differs between the first run and the second. Reaching it takes two leading quotes on the command line, since the `-O` pass eats one pair, which is the shape the GUI front ends produce when they wrap an argument in literal quotes (xroche/httrack-windows#122).
One class is not fixed and changes failure mode, so it is worth stating plainly. A plain argument (not an option value) whose value starts with a quote and does not end with one now aborts the reprise with `Missing quote`, where master silently dropped the quote instead. The abort precedes the writer, so the no-argument update stays dead until the mirror is re-run with its URL. Option values are unaffected: they bypass the second strip pass. It takes three leading quotes typed by hand to reach, since the single-pair shape a front end emits already panics on the first run, on master too. #1263 has the asymmetry behind it.
Section 4 of `01_engine-doitlog.test` checks the footer on the mirrored page rather than the doit.log text alone, so a writer and reader that agree on a wrong value still fail it. A differential over 11 argv shapes moves only the leading-quote cases.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit ed30cde2a4a2b45dbc1ad9f4f88afbe5ed4a0d06
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 13:53:28 2026 +0200
A wizard session leaves no record in hts-log.txt (#1249)
Two problems, same log. Driving the wizard by hand left no trace: hts-log.txt recorded the crawl going on around the questions, but not what was asked or what the user replied, so a session could not be reviewed afterwards. The levels themselves were mislabelled too. `hts_log_vprint` mapped LOG_NOTICE onto the "warning" prefix, so every notice read as a warning, while the footer's message count tracked LOG_INFO, which the default `opt->debug` filters out. That count was always zero.
The engine now writes one line per wizard answer at LOG_NOTICE: the link, the raw reply, the parsed answer number, the verdict, and the filters that answer inserted, read back from the slots they were written to. `ask_continue()` gets the same line for the yes/no prompts, and nothing is logged unless a front end registered a callback to answer with, so a non-interactive crawl gains no noise. LOG_NOTICE also gets its own arm in the level switch, mapping to "info", so notices print `Info:` and move from the warning column of `(N errors, N warnings, N messages)` to the message one. `html/faq.html` already documents the robots.txt notice as an `Info:` line. Two notice sites report a failure rather than progress, and this promotes them to LOG_WARNING so they render as before.
`tests/295_wizard-log.test` drives three answers through a `file://` crawl under `-W` and checks the line each one produces, the two-filter host-scope answer included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
commit 5e8fea2a72b0d51cbfa264ac7251d26c8bf07d5e
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 13:53:12 2026 +0200
A link the wizard authorized is mirrored under its .delayed placeholder name (#1254)
Answer the wizard with anything that takes the link (4, 5, 6, 7, 50, or a host-scope include) and the verdict came back undecided at -1 rather than 0. The parser records a link on `!= 1` but only resolves its delayed type on `== 0`, so an authorized link went into the hash still called `<base>.<hexid>.delayed`. Every later reference gets that stored name back, `hts_wait_delayed()` then finds the link's own record and gives up ("Duplicate entry in hts_wait_delayed() cancelled", then "link is probably looping, type unknown, aborting"), and the page lands in the mirror as a placeholder file no browser opens. Answering "mirror this domain and below" on httrack.com is enough.
The question is only asked while the verdict is undecided, so an answer that did not forbid has authorized. `hts_wizard_apply_verdict()` now resolves that once at the end instead of in six cases plus the scope-include branch, and it still never overturns a verdict the crawl had already computed. An unreadable reply (-999) used to arrive here undecided too; #1259 makes it refuse the link, so it reaches this normalization already set to 1 and the two changes compose in either merge order.
Two other effects move with it, both in the same direction. `htsAddLink()` (htscore.c:3786) gates on `if (!forbidden_url)`, so under -1 a link handed back by an external module parser was never saved: it logged "(module): file not caught" and shipped as an absolute external URL. It is now saved and rewritten local, and the function returns 1 where it returned 0. The `check_link` callback also sees this value, and html/plug.html documents -1 as "no decision has yet been taken by the engine", so a plugin keyed on -1 used to get a say on wizard-answered links and now defers. The user answering is a decision, so I read that as a correction rather than a regression, but it crosses the public plugin surface.
The two gates are unchanged. Outside `-W` the verdict reaching them is already 0 or 1, so aligning them would change no link I could build. `--addurl` is a second door onto the same state, filed as #1253. The "looks like binary" and octet-stream pair from the field log did not reproduce here and is untouched.
tests/296 crawls two references to one authorized foreign link and checks the mirror: the page is there with its body, both hrefs point at it, and no placeholder survives. tests/293's battery now drives every answer from the undecided verdict the question is actually asked in. The mutant that matters most, an unconditional `*forbidden_url = 0` erasing a refusal, is caught by 293 alone. 296 never crawls a link the crawl had already refused, so it cannot see that one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
commit c5c931093a9b8fa5650c3355c5de8bb3b7738404
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 11:39:27 2026 +0200
The interactive wizard spins forever when stdin reaches EOF (#1248)
`httrack -W <url> < /dev/null` never returned. At EOF `linput()` hands back an empty line and reports no failure, so the answer loop in `htsshow_query3()` re-prompted forever at 100% CPU. Any non-interactive wizard run hits it: a piped answer script that runs out of answers, or a service whose stdin is closed.
An unanswerable prompt now reads as the `*` answer, which refuses the link and stops asking. No further answer can arrive, and refusing mirrors nothing the user did not ask for. The guard tests both stream flags, since a closed descriptor sets only `ferror` while EOF sets only `feof`. On a closed stdin the old loop printed 19 million prompts and 58 MB in 25 seconds.
tests/294 crawls a page linking twice to a second local server, once with stdin on `/dev/null` and once with it closed. It asserts the question was asked exactly once, so an answer that refuses this link but keeps asking fails, and the foreign host is served rather than a black hole, so an answer that mirrors it fails too. Each case needs its own half of the guard to pass.
Closes #1247
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 935ea5a89b2a63c6295da23b9663d146e14fb635
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 10:30:21 2026 +0200
An LTO build collapses the installed-header symbol test's candidate list (#1246)
Ubuntu's 3.49.21-2 failed on four architectures (amd64, ppc64el, riscv64, s390x) with `207_install-headers-symbols.test` reporting "only 1 symbols reached the link probe, the candidate list is broken". Debian built the same source everywhere. Ubuntu compiles with `-flto=auto -ffat-lto-objects` by default and Debian does not.
The test picks its probe candidates by matching identifiers in the installed headers against the library's hidden symbol names. Under LTO, GCC renames a local to `abortf_.lto_priv.0`, the names stop matching, and the candidate list empties, so the `probed >= 5` floor fires. The floor was right; what fed it was broken. Stripping the clone suffixes before the comparison takes an `-O3 -flto` build from 1 candidate to 58, and an ordinary build from 32 to 65 by recovering its `.isra` and `.constprop` clones.
That floor turns out to be a weak detector on its own: a strip handling only `.lto_priv` still passed while losing 30 of 54 candidates. The canary header now declares one clone-only name per suffix and requires each to be reported, so dropping any single suffix fails the test.
Nothing in CI builds with LTO, which is how this reached the archive. The deb job runs inside debian:sid and gets Debian's flags; every other leg uses configure's defaults. The matrix gains an `-O3 -flto` leg.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 3a0cf262d6851ffb011d78f1299988685e14f2b3
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 10:22:44 2026 +0200
The wizard verdict switch is unreachable by any test (#1245)
The verdict half of a wizard answer moves into `hts_wizard_apply_verdict()`, leaving `hts_acceptlink_()` one call where a switch used to be. #1239 made the filter half a testable function and left this one unreachable: `hts_acceptlink_()` is static, and only the interactive `query3` callback gets there.
The self-test drives the new function against a real `httrackp`, so it asserts what an answer does to the crawl rather than what it means. An accepting answer must not clear a refusal the crawl already computed, answer 4 caps the recursion instead of deciding the link, `*` switches the wizard to automatic, and an answer in no known range only warns. Eight mutants each turn tests/293 red, including the three shapes the call site used to hide.
Behavior is unchanged for every int answer, both scope-range boundaries included. The dead commented-out blocks under answers 3 and 4 went with the move. The widened filter reservation is still untested: reaching it needs a crawl driving `query3`.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 47a41fb39e7dc7307fcb09ee9cdc0930cf3f5a43
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 09:10:54 2026 +0200
webhttrack's browser dependency drags httrack toward autoremoval (#1235)
The autoremoval gatherer reads only the first alternative of a disjunction and never looks at Recommends, so webhttrack's `chromium | firefox-esr | www-browser` in Depends put the whole httrack source on chromium's RC bug clock. [#1128867](https://bugs.debian.org/1128867) has 3.49.14-1 marked for removal from testing on 9 September. Nothing here wants chromium in particular: `src/webhttrack.in` searches `x-www-browser`, then `www-browser`, then a dozen binaries by name.
#436 flipped this list the other way round in June, back when firefox-esr's RC bugs were doing the same to us, and six weeks later chromium got one. Whichever real browser leads the list is big and bug-prone, so the disjunction moves to Recommends, where the gatherer cannot follow it. Apt installs Recommends by default, so a normal install is unchanged. The `webhttrack` wrapper does still exit when it finds no browser (`src/webhttrack.in:97`), but the package also ships `htsserver`, which serves the same UI over HTTP to a browser on another machine, so the dependency is strong rather than absolute.
This only beats the removal if it is uploaded and migrates before 9 September. Testing is still on 3.49.14-1, with i386 and riscv64 at Needs-Build.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit fabbf80ea50d136a121314e902952700c86e00fb
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 08:51:10 2026 +0200
The wizard can only answer about one exact host (#1239)
The interactive wizard could only answer about the exact host it asked
about, so a mirror of www.example.com stopped again at
download.example.com, and Skip All stayed unusable while any unknown
subdomain might still turn up. Reported against the GUI as
xroche/httrack-windows#96.
The engine now enumerates the domain scopes for a host and takes two new
answer ranges over them. hts_wizard_host_scope() does the splitting so a
front end never has to: it derives the scopes from the same question
string it was handed, which may carry a protocol, credentials and a port.
Answers HTS_WIZARD_SCOPE_INCLUDE+k and HTS_WIZARD_SCOPE_EXCLUDE+k then
take or drop the k-th scope. An index is only safe instead of a finished
pattern because of that helper: the menu label and the applied filter come
out of the same code and cannot drift.
Each answer emits two filters, since +*.example.co.uk/* misses the apex,
which also meant widening the filter-array reservation before the answer
switch. It made room for exactly one insert while HT_INSERT_FILTERS0
asserts on overflow, so the second insert would have aborted on the
boundary at filptr == maxfilter - 2. hts_wizard_scope_answer() carries the
range decision for both the filter and the verdict halves, so the two
cannot disagree.
No public suffix list is involved: every suffix down to a two-label domain
is offered and the user picks the boundary. A bare TLD is never offered,
an IP literal has no scopes, and a fully-qualified www.foo.com. stops at
foo.com. instead of offering the root label as com.
Only the CLI and WinHTTrack need the new menu entries. WebHTTrack answers
"" to every question and Android registers no query3 at all, which is
#1237.
Closes #1117
Signed-off-by: Xavier Roche <xroche@gmail.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit af0e36e32601de817ed5542f0c6766e4ab77dde8
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 03:14:40 2026 +0200
A -%F footer that exactly fills the page buffer aborts the crawl (#1241)
* A footer that exactly fills the page buffer aborts the crawl
#670 dropped an oversized footer instead of emitting an unterminated buffer,
but left the success path alone: it appends the closing newline with
strcatbuff, which aborts rather than clips, so the one expansion length that
fills tempo exactly has nowhere to put that byte. Reserve the newline before
formatting, so an expansion that no longer fits takes the existing drop path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Format the touched call to clang-format
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Size the sweep's path instead of inheriting the temp one
The footer template caps at 253 chars, so the number of {path} references and
the literal padding both hang off the path's length. On the Windows runner the
temp directory is 80 chars, which pushed the template past the cap and failed
the test on its own guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Sweep both line endings and assert the footer arrives whole
The fixture was LF-only, so strlen(eol) was always 1 and a fix reserving one
byte rather than the eol's length passed while CRLF pages still aborted. The
emitted length was unchecked too, so a clipping fix read as success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Fail on a temp path too long, and name the length that died
A skip would red the Windows leg anyway, since it compares the skip set
exactly, while silently dropping the coverage on every other platform. The
wider path leaves more room before that bites. The abort message named the
template, identical at every step, rather than the expansion that aborted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 2dc30ab4907cf20a7f4e4d5397902c90da235c02
Author: Xavier Roche <roche@httrack.com>
Date: Fri Aug 14 02:55:41 2026 +0200
The default footer still uses the legacy %s form (#1240)
* Default footer uses named fields instead of %s
The named-field footer (-%F "{url}") has been the documented model since #667,
but the default template still carried the positional %s form, so any field a
user added to it was silently ignored: a "%s" anywhere selects legacy mode for
the whole string. WebHTTrack carried its own copy of that literal; it now takes
the engine macro.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Point the WebHTTrack footer test at the macro's own comment
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit d681929d1ac95c03ae37430ab8fc49ef8cc66af7
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 16:09:21 2026 +0200
An abandoned WebHTTrack server never stops, holding the macOS app open (#1236)
* An abandoned WebHTTrack server never stops on its own
The watchdog only started consulting the heartbeat after the launcher died,
and on macOS the launcher waits on "open -W", which returns when the whole
browser quits. Closing the window therefore left htsserver running with the
app bundle open, so the disk image could not be ejected without a force kill.
Bind the session to the user's attention when idle and to the work when busy:
the page now says goodbye on pagehide, a running mirror vetoes every exit so a
crawl is never lost, and the launcher waits on the server rather than on the
browser. /ping was also the one reply served without cache headers, and
smallserver() returned 0 unconditionally, so every clean quit was reported as
"Unable to create the server".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Keep the idle fallback at 120s and settle the test before judging survivors
A backgrounded tab has its timers throttled, so the fallback stays where it
was; promptness is the goodbye's job. The two servers that must stay alive
started after the one whose death ends the wait, so the test judged them
before either had outlived a timeout of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Report the killed server before the pings that failed because of it
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Count windows instead of racing a browser's throttled timers
A single pending-goodbye deadline cannot tell "the last window left" from "one
of several left", so closing one of two windows ended a session the other still
had open: a hidden tab's timers are throttled to as little as one wake-up a
minute, and its cancelling ping arrived after the countdown had fired. Each page
now carries an id, pings under it and drops it on pagehide, and the server exits
when the last one is gone.
Liveness is counted in watchdog ticks rather than wall clock, so a suspended
laptop no longer ages a session out from under the user. A bare connection no
longer cancels a departure, since any local peer can open one. And a failed
thread spawn left commandRunning latched, which the new mirror veto would have
turned into a server that never exits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Take a farewell only from a request holding the session id
/ping is a GET, so it clears neither the session-id gate nor the Origin check,
both of which are POST-only. That was harmless while a heartbeat could only
extend a session; ending one on an unauthenticated request handed every local
process, and every page the user visits, a way to close someone's WebHTTrack.
A full window table now refuses newcomers instead of evicting, so a flood of
ids cannot push the real window out either.
Also from review: query_alnum_value left a half-read value in the caller's
buffer when it returned false, the new predicates and flags were plain ints in
a tree that spells booleans hts_boolean, and --ping-timeout re-parsed with the
bare sscanf that #614 cost us on --port.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Prove a flood of window ids cannot evict the real one
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Name the launcher's pid with a type MSVC has
pid_t is POSIX, and the old code only ever named it inside an #ifndef _WIN32
block. Hoisting the parent check into a function signature put it where Windows
compiles it, and webhttrack.vcxproj stopped at seven errors on one line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
commit 1483289eec48fc503d81bf8c4bd9115e0af45c10
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 11:14:28 2026 +0200
debian: record the 3.49.21-2 upload (#1234)
The revision carried the hppa FTBFS fix to the buildds as a quilt patch against
the frozen orig. The patch itself stays out of git, where the fix is already in
the source; only the changelog entry belongs here, so the file stays continuous
for the next upload.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 1751c4b5935105a847c2a82fb013e0af512284cc
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 08:21:17 2026 +0200
mkdeb.sh reaches debsign before it discovers the key is wrong (#1233)
The key is only resolved when debsign runs, which on a release is after the
tarball has been built. Ask gpg for it at startup instead, and name the
0x-prefixed fingerprint form in the error, since a short id is what people reach
for first and it is the ambiguous one.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit bad28b629c2a7d684ba44a9f3772f5a9e8630d72
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 08:08:03 2026 +0200
A lost Windows runner leaves almost no telemetry behind (#1229)
The suite watchdog's commit statuses are the only evidence that survives a lost runner, and most of it was being thrown away: they went to the PR merge commit, whose SHA GitHub collects, leaving 3 of 146 recorded kills with any telemetry at all. They now go to the PR head, which costs us a line in the PR's checks list.
Each status carries more too. Loop lag and the number of posts that failed since one landed separate a box that stopped from a network that broke; `n=`/`f=`/`e=` come from one `GetTcpIPv4Statistics` call; `m=`/`c=`/`a=` are summed from the process array already enumerated for `p=` and `h=`. The cadence halves to 15s, and the in-flight test name is clipped to 30 characters to pay for the new fields. The widest real line measures 103 of the 140 characters GitHub keeps.
The rest closes two unsound spots in the Windows kill paths, neither of which explains the deaths. `stop_server` re-read `/proc/<pid>/winpid` after signalling its target, so the number it handed to `taskkill /F /T` could already belong to a stranger, and a hosted runner reissues a freed PID within a second rather than in theory. Nothing checked the image before firing either. Both the winpid and the image are now read while the target is alive and checked against `tasklist`, with a warning annotation when they disagree, which is also the meter for how often a tree kill would have left our own trees.
Refs #1228.
commit 6584fded15a7c8be1e03ee0f0e84e206d5724fbb
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 07:43:13 2026 +0200
mkdeb.sh regenerates an orig that debian/patches cannot apply to (#1232)
A quilt patch is written against the tarball it will be applied to. Once the fix
it backports is upstream, HEAD carries it too, so an orig regenerated from HEAD
makes the patch fail or apply with fuzz. Require --orig whenever the series file
is non-empty, unsigned builds included: this one breaks the build, not policy.
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit d8f0cec17d3fe71100dd5c1588c0361629ca7b2c
Author: Xavier Roche <roche@httrack.com>
Date: Thu Aug 13 01:01:27 2026 +0200
Three suite tests fail on an emulated buildd for being slow, not broken (#1231)
* Three suite tests fail on an emulated buildd for being slow, not broken
hppa reported 3.49.21-1 as Build-Attempted with 105, 151 and 269 red, all three
on wall clock under qemu-user: the guard's own diagnostics dump took 31s of a
30s bound, one configure run passed a hard 300s cap, and the pairwise header
sweep outran the 600s harness budget mid-batch.
Each now measures the property instead of the host. 105 times the guard to its
DUMP announcement, leaving the dump (minutes, emulated) out of the bound. 151
watches configure's output for silence rather than capping its total, and skips
when the budget runs out while it is still making progress. The sweep runs in
slices so it can be given up on, paces itself against what 269 hands it, and 269
declares the larger budget its n^2 compiles need, so the emulated leg still runs
it to completion instead of pacing out.
The emulated leg is the CI counterpart of that buildd and was green on the same
commit: it prints test-suite.log now, so a test that skipped rather than ran
stops reading as coverage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Fix the review's two skip-masking holes and the unbounded budget raise
A hung configure could reach the budget skip before the silence window closed, so
a wedge reported SKIP; the skip now needs a run that is still writing and enough
budget left for the detector to speak. 151 also paced inside run(), before
accept/reject read the verdict, so a configure that answered wrongly could exit 77
instead of 1: pacing moved to the callers, which judge first.
The per-test raise is bounded and normalized through one budget parser, since bash
test errors rather than compares past intmax and would have left the guard unarmed,
and a leading zero read as octal in arithmetic and decimal in test. Renamed
TEST_TIMEOUT_AT_LEAST so each use site carries the upwards-only rule, documented in
AGENTS.md, and the sweep takes an explicit --budget rather than sniffing the
environment, so the MSVC job cannot report a paced skip as a header break.
Tests for each: 151 drives run() through a configure_cmd seam with a child that
hangs and one that only crawls, 105 requires the DUMP announcement to be seen while
the guard runs (its fallback made the latency bound vacuous) and pins the hostile
budget values, and the sweep counts the units that reached the compiler.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* Two of the new assertions passed a mutant; make them bite
Mutation-testing the added tests found two that could not see the bug they were
written for. The verdict check stubbed run() out, so a pacer left inside the real
one stayed invisible: it now drives the real run() through a child that answers
wrongly with the budget spent. The announcement check only asked that the marker
appear while the guard ran, which a driver announcing after the dump still
satisfies: a slow ps widens the dump, and the marker must now precede it.
Writing the first exposed a third: reject takes one argument fewer than accept, so
the extra one reached run() as an env assignment and the child failed to exec,
which made the probe pass on the wrong answer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* The verdict probe left the pacer disarmed
It numbered its run past the case count, so the pacer it exists to catch declined
to fire on a negative "steps left", and the mutant that puts pacing back inside
run() survived. Keep cases ahead of the run number.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
* A killed configure left its child running and hung the macOS job
The macOS leg reported no test failure and then sat until its 20-minute step
timeout: bash 3.2 does not replace the subshell around the child with the child
itself, so killing that subshell left the run alive, and it outlived make check.
Give each run its own process group and kill the group.
The probe that exposed it now covers it: a child that spawns a child of its own,
and nothing of it left running afterwards.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Xavier Roche <roche@httrack.com>
---------
Signed-off-by: Xavier Roche <roche@httrack.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
![[bug history graph]](https://qa.debian.org/data/bts/graphs/h/httrack.png){kind=link}