Symfony 6 Custom login strange behavior - symfony

Question

I have a custom login system where using username/password against LDAP server. Based on the response i know that they are correct. The response is array with user details. I don't use the password hasher, the only information i store about the user is email/name which i recieve from the LDAP response.
My Authenticator
class CustomAuthenticator extends AbstractLoginFormAuthenticator
{
public function authenticate(Request $request): Passport
{
$uname = $request->request->get('uname', '');
$password = $request->request->get('password', '');
// check them against the LDAP server
$ldapData = $this->ldapChecker($uname, $password); // binds ldap server with username/password from login form. If the bind is success the ldap returns array containing user details (name, email etc)
if(is_array($ldapData)){
// if its array the submitted username/password are correct.
$email = $ldapData[0]['email'] // getting the user email from the ldap response
// here i'm checking if the username exist in the database if not i create new user
$checkUserExist = $this->user->findOneBy(['uname'=>$uname]);
if(!$checkUserExist){
// creating new user with the data from ldap response;
// storing name, uname, email
}
return new Passport(
new UserBadge($email, function($userIdentifier) {
$user = $this->user->findOneBy(['email' => $userIdentifier]);
if (!$user) {
throw new UserNotFoundException('passport fail');
}
return $user;
}),
new CustomCredentials(function($credentials, User $user) {
return true;
}, $password)
);
}
// here is logic to handle wrong username password
}
public function onAuthenticationSuccess(Request $request, TokenInterface $token, string $firewallName): ?Response
{
return new RedirectResponse($this->urlGenerator->generate('app_home'));
}
My security.yaml
security:
# enable_authenticator_manager: true
# https://symfony.com/doc/current/security.html#registering-the-user-hashing-passwords
password_hashers:
Symfony\Component\Security\Core\User\PasswordAuthenticatedUserInterface: 'auto'
# https://symfony.com/doc/current/security.html#loading-the-user-the-user-provider
role_hierarchy:
ROLE_ADMIN: ROLE_USER
providers:
# used to reload user from session & other features (e.g. switch_user)
app_user_provider:
entity:
class: App\Entity\User
property: email
firewalls:
dev:
pattern: ^/(_(profiler|wdt)|css|images|js)/
security: false
main:
lazy: true
provider: app_user_provider
custom_authenticator: App\Security\CustomAuthenticator
logout:
path: app_logout
# where to redirect after logout
# target: app_any_route
# activate different ways to authenticate
# https://symfony.com/doc/current/security.html#the-firewall
# https://symfony.com/doc/current/security/impersonating_user.html
# switch_user: true
# Easy way to control access for large sections of your site
# Note: Only the *first* access control that matches will be used
access_control:
- { path: ^/login$, roles: PUBLIC_ACCESS }
- { path: ^/logout$, roles: PUBLIC_ACCESS }
- { path: ^/admin, roles: ROLE_ADMIN }
- { path: ^/, roles: ROLE_USER }
when#test:
security:
password_hashers:
# By default, password hashers are resource intensive and take time. This is
# important to generate secure password hashes. In tests however, secure hashes
# are not important, waste resources and increase test times. The following
# reduces the work factor to the lowest possible values.
Symfony\Component\Security\Core\User\PasswordAuthenticatedUserInterface:
algorithm: auto
cost: 4 # Lowest possible value for bcrypt
time_cost: 3 # Lowest possible value for argon
memory_cost: 10 # Lowest possible value for argon
The problem is sometimes when i open the app it loads user details from another user / or after loggin it shows details of another user or when i open incognito tab and open app it shows that i'm logged. I disabled the twig cache because i thought is cache problem but the problem persist.