1. connect2id Avatar connect2id
  2. Crypto
  3. Nimbus-JOSE-JWT
  4. Issues

RemoteJWKSet calls update URL more than once when refreshing the JWKSet

Issue #457 resolved
Andreas Huber created an issue

We see regular load spikes on our oauth2 servers. They are caused by the cache refresh in RemoteJWKSet.get(), which is done every five minutes. On a busy system it can happen that multiple threads fetch the JWKSet concurrently. This increases the load and it takes longer than necessary.

I suggest to add double checked locking to the cache refresh. I created pull request https://bitbucket.org/connect2id/nimbus-jose-jwt/pull-requests/90/use-double-checked-locking-for-cache

Comments (10)

  2. Vladimir Dzhuvinov

    I’m curious, what is the context of having this 5 min refresh of the the RemoteJWKSet?

  4. Vladimir Dzhuvinov

    I merged the RP into a branch (here 376a308 ), but I still want to give the suitability of the change some thought before it goes out in the next release.

    Do you use the RemoteJWKSet in the OAuth 2 server for the client jwks_uri’s? If that is the case I’m not sure the keeping of RemoteJWKSets around, one per registered client (if that is indeed the case) with a continuous refresh is ideal. One efficient strategy is to refresh the cache with a dedicated thread, and leave the RemoteJWKSets out of this.

  5. Andreas Huber reporter

    I think it is the other way around. (Maybe I am mixing up the terminology.) We use it to verify bearer tokens signed by an oauth2 server.

    I see how my description above could be misleading. It is not the oauth2 server that causes the load spike. The load on the server is caused by the cache refresh on the clients.

    We have a cluster of web-servers. They authenticate requests with the bearer tokens. When DefaultJWKSetCache.requiresRefresh() returns true, or we receive a bearer token with an unknown kid, the method RemoteJWKSet.updateJWKSetFromURL() is called to fetch the latest JWKSet from the oauth2 server. There can be hundreds of requests per second on the web-server, which leads to as many requests.

    I don’t think a dedicated thread would work in this case. We would not be able to verify requests with an unknown kid until the cache is updated sometime in the future.

  7. Vladimir Dzhuvinov

    PS: When you were working at this problem, did you have thoughts about some other more appropriate design for client-side RemoteJWKSets? i wonder if the API and multi-threading can be improved further.

    From your original post I thought you were dealing with OAuth 2.0 server side code, where JWTs can also be submitted, by clients that are registered for JWT-based authentication at the token endpoint, or extensions like JAR or CIBA. There the client JWK set will often get persisted when it’s cached. Registered clients can also be inactive for long periods of time.

  8. Andreas Huber reporter

    Thanks, @Vladimir Dzhuvinov .

    To answer your question. There are three reasons to update the cache.

    1. cache is empty
    2. cache does not contain the requested key
    3. remove stale keys

    We could move 3 into a dedicated thread that refreshes the cache asynchronously (as you proposed above). This would improve performance because we would not have to block every request while the cache is updated. But starting a new thread opens a can of worms. We would have to handle lifecycle operations for application servers (Java2EE, Tomcat, JBoss, …). For example, stop the thread when the web-context is stopped. Otherwise we would cause a resource leak.

    We would still have the synchronization for 1 and 2. And I don’t see a way around that.

  9. Vladimir Dzhuvinov

    Okay, thanks for chiming in. I always want to make sure crucial insights don’t get missed and design & API thinking is not neglected when plugging issues in the code :)

    ‌

  11. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Component
–
Milestone
–
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!