[3.0][Testing] Add an integration test suite that runs against a real forum - #9345
Open
albertlast wants to merge 4 commits into
Open
[3.0][Testing] Add an integration test suite that runs against a real forum#9345albertlast wants to merge 4 commits into
albertlast wants to merge 4 commits into
Conversation
This was referenced Aug 2, 2026
The installer tells you to delete it and cannot do it itself: the ?delete link it offers is a GET, and command line arguments only ever reach $_POST, so nothing on the CLI path ever gets there. Leaving it behind is not cosmetic. Settings.php redirects every request back into the installer while the file exists, so the forum the script just built is unreachable, and SMF puts a "MAJOR SECURITY RISK: you have not removed install.php" box on every page it shows an administrator - which also lands in front of anything else a test or a person is trying to read on that page. Deleting it is safe for a reinstall because install_one() calls reset.sh first, and reset.sh clears Settings.php and then blocks until the entrypoint has staged a fresh copy. Adds a check in front of the two installer passes to say so out loud when it has not: without one, php reports "Could not open input file: install.php", which reads like a broken script rather than a stack that was never made installable. Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
Two forums side by side, each with its own administrator, and a password
chosen at install time is a combination that ends in hand written SQL
sooner or later - which is a poor way to answer a question as ordinary as
"is this the password?".
user.sh answers it. list shows the accounts, check says whether SMF would
accept a password and exits 0 or 1 so it can be used in a conditional,
and reset sets a new one. --engine reads the settings use-engine.sh saved
for the other engine, so the forum that is not currently live can be
looked at without switching to it and back.
Two details that stop it being a thin wrapper around an UPDATE:
- The hashing goes through Security::hashPassword() rather than being
written here, so what lands in the table is by construction what
Login2 reads back out. A script that hashes passwords its own way is
a script that eventually disagrees with the forum.
- reset clears passwd_flood too. SMF locks an account out for a while
after enough wrong guesses, and a new password behind a live lockout
behaves exactly like a password that did not take.
check also points out an account that is not activated, which fails to
log in with an entirely correct password.
The password is passed to the container through the environment rather
than in the argument list, which anything able to read the process table
can see. Also completes the file list in the README, which still only
described the image and had none of the scripts in it.
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
The unit suite is deliberately database-free, and says so: its bootstrap
notes that anything reaching Config::$modSettings, User::$me or Db::$db
"belongs in an integration suite running against a real install". There
was not one, so most of the forum had no automated proof of anything.
Adds tests/Integration/ as a second PHPUnit testsuite, and .docker/test.sh
to run it on one engine or both. composer test still runs everything;
when there is no forum to talk to the integration tests skip rather than
fail, so it stays useful without Docker.
IntegrationTestCase gives each test a transaction that is rolled back
afterwards, actingAs()/adminId() via User::setMe(), hook() registration
that lives only in $modSettings, and assertNoErrorsLogged() - which is
usually the point of the test, because SMF records most of what goes
wrong in log_errors rather than showing it.
Three tests to start:
HarnessTest checks the harness itself, including that the rollback
really happens and that assertNoErrorsLogged can fail
ModSettingsTest the counter regression: updateModSettings($x, true)
emitted SET value = value + 1 against a text column
SchemaTest compares Sources/Db/Schema/v3_0/ against the database in
both directions, which is the drift AGENTS.md warns only
ever shows up at runtime
ModSettingsTest is why running both engines matters rather than being
tidy: with the fix reverted it still passes on MySQL, which coerces text
to a number, and fails only on PostgreSQL, which refuses. Verified in
both directions before committing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
The banners in the new test classes were written by hand with the wrong number of asterisks, so SMF/section_comments did not recognise them and inserted its own alongside, leaving IntegrationTestCase with two "Internal properties" headings and two "Internal methods" ones. AGENTS.md says not to hand-write these. Removes them and takes what the fixer produces, along with the single_quote, native_function_invocation and no_unused_imports changes it wanted in the same pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: albertlast <mathiaspapealbert@hotmail.com>
albertlast
force-pushed
the
tests/integration
branch
from
August 28, 2026 22:12
f2fade7 to
9713156
Compare
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The unit suite from #9326 is deliberately database-free, and its own bootstrap
says what is missing: anything reaching
Config::$modSettings,User::$meorDb::$db"belongs in an integration suite running against a real install".There was not one, so most of the forum had no automated proof of anything.
This adds it.
tests/Integration/is a second PHPUnit testsuite that runsagainst a forum installed by #9344, on either engine:
.docker/test.sh # both engines .docker/test.sh --engine postgresqlcomposer teststill runs everything. When there is no forum to talk to theintegration tests skip rather than fail, so it stays useful on a machine with
no Docker — 108 unit tests pass, 20 integration tests skip, exit 0.
What the base class provides
IntegrationTestCasegives each test:tearDown(), so tests need not order themselvesaround each other.
$modSettingsis restored separately, because the rollbackreturns the table and not the static array;
actingAs($id)/adminId(), viaUser::setMe()— the same seamLogin2::DoLogin()uses once it has checked the password;hook($name, $function)withpermanent: false, so it lives in$modSettingsand disappears with the rollback;
assertNoErrorsLogged(), usually the point of the test. SMF records most ofwhat goes wrong in
log_errorsrather than showing it, so a page that returnedthe right thing while quietly logging an undefined index has still regressed;
queryRow()/rawSetting(), which read past$modSettingsand its cache.Two things the rollback does not cover, both documented in the class: DDL, since
MySQL commits implicitly on
CREATE/ALTER/DROP; and anything happening inanother process, such as a request made over HTTP.
The three tests
HarnessTestchecks the harness itself — that the rollback really happens(one test writes, the next asserts it is gone), that
$modSettingsisrestored, and that
assertNoErrorsLogged()is capable of failing. A suitewhose isolation quietly broke would not fail; it would start passing things it
should not.
ModSettingsTestpins the counter regression fixed in [3.0] Makes the settings counters increment on PostgreSQL too #9340.updateModSettings($x, true)emittedSET value = value + 1againstsettings.value, a text column.SchemaTestcomparesSources/Db/Schema/v3_0/against the live databasein both directions — declared tables and columns that are missing, and
columns present that nothing declares. This is the drift AGENTS.md warns about,
where a query naming a removed column fails at runtime only, and inside a
background task retries forever.
Why both engines, concretely
Not thoroughness for its own sake. I reverted #9340's fix and re-ran:
MySQL coerces text to a number and hides the bug completely. A suite that only
ever saw one engine would have proved nothing here. Worth knowing too: on
PostgreSQL a failed query poisons the rest of the transaction, so one swallowed
error turns every later query in that test into
false—queryRow()reportsthat as a readable assertion failure rather than a
TypeErrorabout argument #1.Notes
tests/bootstrap.phpgains one line registeringSMF\Tests\for autoloading.The unit tests are each self-contained so nothing needed it before; anything
sharing a base class does. Kept beside the two
setPsr4()calls already thererather than adding an
autoload-devsection only the suite would use.composer.jsongainstest-unitandtest-integrationalongsidetest.AGENTS.mdis updated: it previously told contributors thatDb::$dbwassimply out of scope.
Verified
.docker/test.sh— 128 tests, 196 assertions, green on both engines.Settings.phpremoved — 108 pass, 20 skip, exit 0.check-signed-off.php,check-smf-index.php,check-smf-license.php,check-smf-languages.phpandphplintall pass;shellcheckclean on allfive
.dockerscripts.Two commits that are not about testing
Lets the section comment fixer place its own banners— I wrote the sectionbanners by hand with the wrong asterisk count, so
SMF/section_commentsdid notrecognise them and added its own alongside. AGENTS.md already says not to
hand-write these; this takes what the fixer produces.
Removes the trailing tabs from a blank line in PM search— one character inSources/PersonalMessage/Search.php, which I have otherwise not touched.c344b5c23left a line holding nothing but three tabs. It has not turned CI redso far because the style workflow normally only checks the files a PR changes;
it checks everything when
composer.lockis in the diff, which this PR'sdoes. Any other branch touching a dependency will hit the same thing, so it
seemed better to fix than to work around. Happy to split it out if preferred.
Merge order
Merge #9326 and #9344 before this one. Both are contained in this branch, so the
diff shown here is theirs as well as its own; once they land and this is rebased on
release-3.0, what is left istests/Integration/and its bootstrap.Issues References (Fixes|Related|Closes)
.docker/install-forum.shto have a forum to test.