Consider SingleFlight for concurrent token refresh deduplication

[elbichop]

Hi Duende team,

I'd like to propose considering a single-flight / request-coalescing mechanism for concurrent token refresh operations in "Duende.AccessTokenManagement".

The motivation is a concurrency scenario where multiple requests discover at approximately the same time that an access token needs to be refreshed.

Instead of allowing each request to independently perform the same refresh operation, concurrent requests could be coalesced so that:

• One request performs the refresh.
• Other concurrent requests await that same in-flight operation.
• Once the refresh completes, all waiting requests receive the resulting token.
• A different token/resource can have its own independent in-flight operation.

Conceptually:

flowchart LR
    A[Request A] --> F@{ shape: fork, label: "Fork or Join" }
    B[Request B] --> F
    C[Request C] --> F
    D[Request D] --> F
    E[Request E] --> F
    F --> G["one refresh operation"]
    G --> H([token])

Loading

This is slightly different from simply serializing access with a lock or "SemaphoreSlim".

The goal is specifically to share the result of the operation already in progress, rather than making every caller perform the operation sequentially.

Why I think this could be useful

Token refresh is a natural example of a single-flight operation.

Under load, several concurrent HTTP requests can reach the token-expiration/refresh path at almost exactly the same time. Without request coalescing, this can result in multiple callers attempting the same refresh operation.

A single-flight primitive makes the intended concurrency semantics explicit:

var token = await singleFlight.RunAsync(
    tokenKey,
    operationToken => RefreshTokenAsync(operationToken),
    cancellationToken);

The important part of this model is that caller cancellation is independent from the shared operation.

One caller cancelling its request should not cancel a refresh that other callers are already waiting for.

A small .NET implementation

I've been working on a small open-source implementation of this pattern for .NET:

• GitHub: https://github.com/elbichop/SingleFlight
• NuGet: https://www.nuget.org/packages/SingleFlight.Net

It intentionally has a very narrow scope: deduplicating concurrent asynchronous work by key within a single process.

It does not provide caching, distributed coordination, Redis integration, or other infrastructure.

The idea is simply:

«For a given key, if an operation is already in flight, concurrent callers await that same operation instead of starting duplicate work.»

The current API also separates the caller's cancellation token from the token supplied to the shared operation:

var token = await singleFlight.RunAsync(
    "access-token:client-a",
    operationToken => RefreshTokenAsync(operationToken),
    cancellationToken);

This means an individual HTTP request can stop waiting without cancelling the refresh operation that other callers depend on.

Possible integration

I'm not suggesting that "Duende.AccessTokenManagement" should necessarily take a dependency on this particular package.

Rather, I think the single-flight pattern itself could be worth considering as part of the token refresh implementation.

It could potentially be implemented internally, or the approach could be exposed through an appropriate abstraction if that fits the architecture better.

The important question, in my opinion, is whether token refresh should have explicit "one in-flight refresh per token/resource" semantics when multiple callers reach the refresh path concurrently.

I've also seen concurrency around token refresh discussed in the AccessTokenManagement ecosystem, which makes me think this may be worth exploring as a more general concurrency primitive rather than treating each individual synchronization case separately.

I'd be very interested in hearing the Duende team's thoughts on whether this is a problem worth addressing and, if so, what implementation direction would fit best with "Duende.AccessTokenManagement".

Thanks for taking the time to read this.

[elbichop]

Hi Duende team,

Just following up on this proposal after some time. I’d be interested to hear whether the team sees value in a single-flight/request-coalescing approach for concurrent token refresh operations in AccessTokenManagement.

Happy to provide a proof of concept or discuss the implementation details if useful.

Thanks!

[maartenba]

Hi @elbichop ! This is a cool approach, and I will move it to the "show and tell" section to inspire others.

[elbichop]

Thanks a lot, @maartenba ! I really appreciate it.

I built SingleFlight.Net to solve a problem I kept running into with concurrent requests for the same resource, and I thought it could be useful to others as well.

I'd be very happy to see it shared in the Show & Tell section. Any feedback from the community would be greatly appreciated!