Issues getting kerberos/Windows AD login work for a web service - nginx

Question

I have been struggling with this for quite a while now, and I can't get it to work.
Here is the setup:
I have a nginx webserver serving a django app at mywebapp.k8s.dal1.mycompany.io
It has the SPNEGO plugin compiled in and I have the following endpoint in my config:
location /ad-login {
uwsgi_pass django;
include /usr/lib/mycompany/lib/wsgi/uwsgi_params;
auth_gss on;
auth_gss_realm BURNERDEV1.DAL1.MYCOMPANY.IO;
auth_gss_service_name HTTP/mywebapp.k8s.dal1.mycompany.io;
auth_gss_allow_basic_fallback off;
}
My AD Domain controller is at burnerdev1.dal1.mycompany.io and I have the following users configured:
rep_movsd
portal
I run the following commands on the DC server in an Admin prompt:
ktpass -out krb5.keytab -mapUser portal#BURNERDEV1.DAL1.MYCOMPANY.IO +rndPass -mapOp set +DumpSalt -crypto AES256-SHA1 -ptype KRB5_NT_PRINCIPAL -princ HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
C:\Users\myself\Documents\keytab>ktpass -out krb5.keytab -mapUser portal#BURNERDEV1.DAL1.MYCOMPANY.IO +rndPass -mapOp set +DumpSalt -crypto AES256-SHA1 -ptype KRB5_NT_PRINCIPAL -princ HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
Targeting domain controller: dal1devdc1.burnerdev1.dal1.mycompany.io
Using legacy password setting method
Failed to set property 'servicePrincipalName' to 'HTTP/mywebapp.k8s.dal1.mycompany.io' on Dn 'CN=portal,CN=Users,DC=burnerdev1,DC=dal1,
DC=mycompany,DC=io': 0x13.
WARNING: Unable to set SPN mapping data.
If portal already has an SPN mapping installed for HTTP/mywebapp.k8s.dal1.mycompany.io, this is no cause for concern.
Building salt with principalname HTTP/mywebapp.k8s.dal1.mycompany.io and domain BURNERDEV1.DAL1.MYCOMPANY.IO (encryption type 18)...
Hashing password with salt "BURNERDEV1.DAL1.MYCOMPANY.IOHTTPmywebapp.k8s.dal1.mycompany.io".
Key created.
Output keytab to krb5.keytab:
Keytab version: 0x502
keysize 110 HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO ptype 1 (KRB5_NT_PRINCIPAL) vno 3 etype 0x12 (AES256-SHA1) k
eylength 32 (0x632d9ca3356374e9de490ec2f7718f9fb652b20da40bd212a808db4c46a72bc5)
C:\Users\myself\Documents\keytab>setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
Checking domain DC=burnerdev1,DC=dal1,DC=mycompany,DC=io
Registering ServicePrincipalNames for CN=portal,CN=Users,DC=burnerdev1,DC=dal1,DC=mycompany,DC=io
HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
Updated object
C:\Users\myself\Documents\keytab>
Now in the "Active Directory Users and Computers" section, I rightclicked the user and selected "Properties"
Then on the "Delegation" tab I set "Trust this user for delegation to any service (Kerberos only)"
Next I copy the krb5.keytab file to my webserver and restart the nginx container
On the Windows workstation which is part of the domain, I log on as rep_movsd - when I run klist:
C:\Users\rep_movsd>klist
Current LogonId is 0:0x208d7
Cached Tickets: (2)
#0> Client: rep_movsd # BURNERDEV1.DAL1.MYCOMPANY.IO
Server: krbtgt/BURNERDEV1.DAL1.MYCOMPANY.IO # BURNERDEV1.DAL1.MYCOMPANY.IO
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
Start Time: 7/16/2020 2:05:51 (local)
End Time: 7/16/2020 12:05:51 (local)
Renew Time: 7/23/2020 2:05:51 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
#1> Client: rep_movsd # BURNERDEV1.DAL1.MYCOMPANY.IO
Server: HTTP/mywebapp.k8s.dal1.mycompany.io # BURNERDEV1.DAL1.MYCOMPANY.IO
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
Start Time: 7/16/2020 2:06:01 (local)
End Time: 7/16/2020 12:05:51 (local)
Renew Time: 7/23/2020 2:05:51 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
I setup Firefox to do SPENGO authentication
Then I hit mywebapp.k8s.dal1.mycompany.io/ad-login and I get a 403 Forbidden error
The nginx server debug log shows:
[debug] 16#16: *195 Client sent a reasonable Negotiate header
[debug] 16#16: *195 GSSAPI authorizing
[debug] 16#16: *195 Use keytab /etc/krb5.keytab
[debug] 16#16: *195 Using service principal: HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
[debug] 16#16: *195 my_gss_name HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
[debug] 16#16: *195 gss_accept_sec_context() failed: Cannot decrypt ticket for HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO using keytab key for HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO:
[debug] 16#16: *195 GSSAPI failed
[debug] 16#16: *195 http finalize request: 403, "/ad-login?" a:1, c:1
[debug] 16#16: *195 http special response: 403, "/ad-login?"
[debug] 16#16: *195 http set discard body
[debug] 16#16: *195 charset: "" > "utf-8"
[debug] 16#16: *195 HTTP/1.1 403 Forbidden
BTW while messing around earlier - I found that if I had set a fixed password for the "portal" user with ktpass and logged in as that account on the workstation, the login would succeed.
I was under the mistaken impression that I'd need to create a new keytab for every user and combine all of them.
Any help is greatly appreciated - I read so many conflicting docs its only confused me further and I've been losing sleep over this.
Thanks in advance!

Answer 1

I've read your problem statement carefully, and I think if you follow the steps I wrote below the issue will be solved.
On the DC server where you are creating the keytab, (1) UAC must be temporarily disabled. (2) The user creating the keytab must be a member of the Domain Admins group.
Ensure the SPN is not a duplicate, then remove the SPN from the Active Directory user account portal. This must be done before creating a new keytab using the same SPN against the same account. The below command is a one-liner, word-wrapping makes it look like two lines.
setspn -d HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
Re-create the keytab again exactly as you did before.
You do not need to run the command setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal because SPN was already set on the Active Directory user account by the ktpass command in step 3.
Replace the old keytab with the new keytab.
Restart the nginx webserver service.
Clear browser cache AND clear Kerberos case (klist purge).
Try it again.
You must do all these steps including the final step 7. Do not skip any.
You service account is named portal. A hash of this password is stored in both Active Directory and the keytab. Same hash is in both locations. The keytab on the nginix server is utilized to decrypt the inbound Kerberos service tickets to determine who the user is attempting to access the web app. More specifically, the GSS authentication does all the work, it uses the keytab to un-scramble the encrypted service tickets. The user rep_movsd does not have the service account credentials. It is part of the Active Directory domain, and when accessing the nginix web server, it gets it's own Kerberos service ticket and its identity is proven to the web server by simply being in possession of a service ticket that is decrypted by the keytab. If it wasn't part of the BURNERDEV1.DAL1.MYCOMPANY.IO domain, or had an expired password, or was a disabled account, it would not be able to get a service ticket and thus not prove its identity and fail authentication.
If you have time, please see my TechNet Wiki article on keytab creation and the logic behind it to help you better understand this complex subject.