Skip to content

Fail the test when the HTTP retry helper times out - #3012

Open
pose wants to merge 1 commit into
masterfrom
pose/fix-http-assert-on-timeout
Open

pose wants to merge 1 commit into
masterfrom
pose/fix-http-assert-on-timeout

Conversation

@pose

@pose pose commented Sep 11, 2026

Copy link
Copy Markdown
Member

misc/test/helpers/http.go's retry loop printed a message and returned false on timeout without ever touching *testing.T. No caller reads that return value, so an example whose endpoint never came up still reported PASS. That is around 46 call sites across the AWS, Google, Kubernetes, DigitalOcean and performance suites.

The timeout path now calls assert.Failf and reports what the last attempt actually hit: a transport error, an unexpected status, or a body ready had not accepted. Previously err was nil on any non-200, so the log printed Http Error: <nil> and dropped the status code.

This will turn currently-green tests red, at least TestAccGcpHclWebserver and TestAccAwsTsApiGateway. They are already failing and reporting success.

Added misc/test/helpers/http_test.go: the three timeout reasons plus the success path, against local httptest servers. Runs in 0.047s, no credentials.

AssertHTTPResultShapeWithRetry printed a message and returned false on
timeout without ever touching *testing.T. No caller in misc/test reads
the returned bool, so an example whose endpoint never came up still
reported PASS. Fail via assert.Failf instead, and carry the reason the
last attempt failed (transport error, non-200 status, or a body that
`ready` rejected) into the message, since "no successful GET" without
the reason is not actionable.

Add a helper unit test covering all three timeout reasons and the
success path. It uses httptest servers with maxWait of 0, so it runs in
milliseconds and needs no cloud credentials.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@iwahbe iwahbe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the tests were consistently failing before, you can skip them here. Otherwise we need to wait until they are fixed before merging.

I don't want to make a red CI run meaningless.

Comment thread misc/test/helpers/http.go
Comment on lines +86 to +87
// No caller reads the returned bool, so the test only fails if we fail
// it here. Returning false alone lets a never-successful check pass.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callers read the returned bool for control flow reasons, just like assert.Equal. As a general rule, Assert* test helpers should fail the *testing.T exactly when they return false.

Comment thread misc/test/helpers/http.go
@@ -70,9 +80,14 @@
// Verify it matches expectations
return check(bodyText)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return check(bodyText)
return assert.True(t, check(bodyText), "check failed")

Comment thread misc/test/helpers/http.go
func AssertHTTPResultShapeWithRetry(t *testing.T, output interface{}, headers map[string]string, maxWait time.Duration,
ready func(string) bool, check func(string) bool) bool {
hostname, ok := output.(string)
if !assert.True(t, ok, fmt.Sprintf("expected `%s` output", output)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we are here...

Suggested change
if !assert.Truef(t, ok, "expected `%s` output", output) {

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants