OAuth2 Share Principal Object with Multiple Gateway Instances - spring-security-oauth2

Question

I have integrated Spring Cloud Gateway with OAuth2 server. It works well with single instance gateway. here is my security config.
#EnableWebFluxSecurity
public class GatewaySecurityConfiguration {
#Bean
public SecurityWebFilterChain securityWebFilterChain(ServerHttpSecurity http) {
http
.authorizeExchange().pathMatchers("/user/v3/api-docs", "/actuator/**").permitAll()
.anyExchange().authenticated()
.and()
.oauth2Login()
.and()
.csrf().disable();
return http.build();
}
But, When I scale gateway to 2 instances, some requests works expected, however some requests return 401.
load balancer (kubernetes nodeport service)
/ \
gateway gateway
\ /
(microservice clusters)
When I logged in first instance of gateway, the principal object is created successfully and also assign session to redis. If next request comes to second instance, it returns 401 because it has not principal.
how can i solve this problem?
ps: i am using redis for web sessions to commonize session informations between gateways.

Answer 1

TL;DR
You can share session principal information on Redis through WebSession. But you can't share access token(JWT), because they are stored in-memory at servers.
Solution-1: Your requests should always go to the server where you are logged in. (details below)
Solution-2: Implement new ReactiveOAuth2AuthorizedClientService bean that stores sessions in redis. (details below too)
Long Answer
From Spring Cloud documentation (https://cloud.spring.io/spring-cloud-static/Greenwich.SR5/multi/multi__more_detail.html);
The default implementation of ReactiveOAuth2AuthorizedClientService
used by TokenRelayGatewayFilterFactory uses an in-memory data store.
You will need to provide your own implementation
ReactiveOAuth2AuthorizedClientService if you need a more robust
solution.
The first thing you know: When you login successfully, the access token(as jwt) is returned by oauth2 server, and server creates session and this session is mapped to access token on ConcurrentHashMap (authorizedClients instance InMemoryReactiveOAuth2AuthorizedClientService class).
When you request API Gateway to access microservices with your session id, the access token(jwt) is resolved by TokenRelayGatewayFilterFactory in gateway, and this access token is set in Authorization header, and the request is forwarding to microservices.
So, let me explain how TokenRelayGatewayFilterFactory works (assume that you use WebSession through Redis and you have 2 gateway instances and you logged in at instance-1.)
If your request goes to instance-1, the principal is get back by session id from redis, then authorizedClientRepository.loadAuthorizedClient(..) is called in filter. This repository is instance of AuthenticatedPrincipalServerOAuth2AuthorizedClientRepository object. The isPrincipalAuthenticated() method returns true, so the flow goes on authorizedClientService.loadAuthorizedClient(). this service is defined as ReactiveOAuth2AuthorizedClientService interface, and it has only one implementation ( InMemoryReactiveOAuth2AuthorizedClientService). This implementation has ConcurrentHashMap(key: principal object, value: JWT)
If your request goes to instance-2, all flow above are valid. But reminder that ConcurrentHashMap no access token to principal, because the access token is stored in instance-1's ConcurrentHashMap. So, the access token is empty, then your request downstreams without Authorization header. You will get 401 Unauthorized.
Solution-1
So, your requests should always go to the server where you are logged in to get valid access token.
If you use NGINX as load balancer, then use ip_hash in upstream.
If you use kubernetes service as load balancer, then use ClientIP in session affinity.
Solution-2
InMemoryReactiveOAuth2AuthorizedClientService is only implementation of ReactiveOAuth2AuthorizedClientService. So, create new implementation that uses Redis, and then do it primary bean.
#RequiredArgsConstructor
#Slf4j
#Component
#Primary
public class AccessTokenRedisConfiguration implements ReactiveOAuth2AuthorizedClientService {
private final SessionService sessionService;
#Override
#SuppressWarnings("unchecked")
public <T extends OAuth2AuthorizedClient> Mono<T> loadAuthorizedClient(String clientRegistrationId, String principalName) {
log.info("loadAuthorizedClient for user {}", principalName);
Assert.hasText(clientRegistrationId, "clientRegistrationId cannot be empty");
Assert.hasText(principalName, "principalName cannot be empty");
// TODO: When changed immutability of OAuth2AuthorizedClient, return directly object without map.
return (Mono<T>) sessionService.getSessionRecord(principalName, "accessToken").cast(String.class)
.map(mapper -> {
return new OAuth2AuthorizedClient(clientRegistration(), principalName, accessToken(mapper));
});
}
#Override
public Mono<Void> saveAuthorizedClient(OAuth2AuthorizedClient authorizedClient, Authentication principal) {
log.info("saveAuthorizedClient for user {}", principal.getName());
Assert.notNull(authorizedClient, "authorizedClient cannot be null");
Assert.notNull(principal, "principal cannot be null");
return Mono.fromRunnable(() -> {
// TODO: When changed immutability of OAuth2AuthorizedClient , persist OAuthorizedClient instead of access token.
sessionService.addSessionRecord(principal.getName(), "accessToken", authorizedClient.getAccessToken().getTokenValue());
});
}
#Override
public Mono<Void> removeAuthorizedClient(String clientRegistrationId, String principalName) {
log.info("removeAuthorizedClient for user {}", principalName);
Assert.hasText(clientRegistrationId, "clientRegistrationId cannot be empty");
Assert.hasText(principalName, "principalName cannot be empty");
return null;
}
private static ClientRegistration clientRegistration() {
return ClientRegistration.withRegistrationId("login-client")
.authorizationGrantType(AuthorizationGrantType.AUTHORIZATION_CODE)
.clientId("dummy").registrationId("dummy")
.redirectUriTemplate("dummy")
.authorizationUri("dummy").tokenUri("dummy")
.build();
}
private static OAuth2AccessToken accessToken(String value) {
return new OAuth2AccessToken(OAuth2AccessToken.TokenType.BEARER, value, null, null);
}
}
Notes:
SessionService is my custom class that interacts redis with reactivehashoperations instance.
The best way is to store OAuth2AuthorizedClient instead of access token. But, it is too hard now (https://github.com/spring-projects/spring-security/issues/8905)

Answer 2

The TokenRelayGatewayFilterFactory uses an in-memory data store to store the OAuth2AuthorizedClient which includes the (JWT) access token. This data store is not shared between multiple gateways.
To share the OAuth2AuthorizedClient information with Spring Session through Redis provide the following configuration:
#Bean
public OAuth2AuthorizedClientRepository authorizedClientRepository() {
return new HttpSessionOAuth2AuthorizedClientRepository();
}
For reactive WebSessions:
#Bean
public ServerOAuth2AuthorizedClientRepository authorizedClientRepository() {
return new WebSessionServerOAuth2AuthorizedClientRepository();
}
Further information for this configuration can be found at https://github.com/spring-projects/spring-security/issues/7889