Spring Cloud Data Flow Server OAuth2 with Azure AD - spring-security-oauth2

I am trying to setup OAuth authentication (OpenID Connect) with Azure AD and Spring Cloud Data Flow server. Following below documentation reference but no help.
https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#appendix-identity-provider-azure
https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-security-oauth2
When I push data flow server in open source cloud foundry, i get below error (if OAuth2 config is removed from application.yml, it gets deployed without issue):
redentials/instance.key and /etc/cf-instance-credentials/instance.crt
18:16:57.512: [HEALTH.0] Failed to make TCP connection to port 8080: connection refused
18:16:57.512: [CELL.0] Timed out after 1m0s: health check never passed.
18:16:57.515: [CELL/SSHD.0] Exit status 0
18:17:07.588: [APP/PROC/WEB.0] Exit status 137 (exceeded 10s graceful shutdown interval)
18:17:07.857: [API.3] Process has crashed with type: "web"
18:17:07.931: [API.3] App instance exited with guid ff60a149-d91f-4d9c-90b9-661c3bb8ad0f payload: {"instance"=>"e35f4a5d-a4f0-433d-6546-82ed", "index"=>0, "cell_id"=>"231ab214-d841-46ba-b20f-243aeac9bbfa", "reason"=>"CRASHED", "exit_description"=>"Instance never healthy after 1m0s: Failed to make TCP connection to port 8080: connection refused", "crash_count"=>3, "crash_timestamp"=>1622845027800626529, "version"=>"3a0686fb-a43a-4528-a425-21a544
From the logs, I do not see any hint of OAuth2.0 setup is taking effect by data flow server. Here is my application.yml entries related to OAuth2 config:
spring:
cloud:
dataflow:
security:
authorization:
provider-role-mappings:
dataflow-server:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
dataflow-server:
provider: azure
redirect-uri: 'https://data-flow-server/dashboard'
client-id: 977-95bc-4f3645d77f43
client-secret: ~02K-5pf182_E-x-PWn
authorization-grant-type: authorization_code
scope:
- openid
- profile
- email
- offline_access
- api://dataflow-server/dataflow.view
- api://dataflow-server/dataflow.deploy
- api://dataflow-server/dataflow.destroy
- api://dataflow-server/dataflow.manage
- api://dataflow-server/dataflow.modify
- api://dataflow-server/dataflow.schedule
- api://dataflow-server/dataflow.create
provider:
azure:
issuer-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/v2.0
user-name-attribute: name
access-token-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/token
token-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/token
user-authorization-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/authorize
authorization-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/authorize
resourceserver:
jwt:
jwk-set-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/discovery/v2.0/keys

Related

Artifactory doesn't work after restoring nuget packages

When a team member restores nuget packages using donet, artifactory enters a loop with the errors below until the memory overflows.
The problem only happens when he adds in his Nuget.Config the tag protocolVersion="3"
<add key="Company" value="https://repo.company.com/artifactory/api/nuget/v3/rdi-nuget-virtual" protocolVersion="3" />
In the tests we noticed that the use of the protocolversion=3 tag causes this.
This tag is necessary because the download of the file is much faster
Anyone have an idea what it could be?
Could you help me please?
logs below
Version jfrog: 7.29.8 rev 72908900
2022-01-11T14:00:57.263Z [jfrt ] [WARN ] [6cdecc92eaf486c7] [.r.ArtifactoryResponseBase:136] [ttp-nio-8081-exec-43] - Sending HTTP error code 403: Download request for repo:path 'nuget-remote-cache:.nuGetV3/feed.json' is forbidden for user: 'lcunha'.
2022-01-11T14:00:57.264Z [jfrt ] [ERROR] [6cdecc92eaf486c7] [etV3VirtualAndRemoteCommon:274] [ttp-nio-8081-exec-43] - Failed to download resource in repo: nuget-remote, at url: https://api.nuget.org/v3/index.json. HTTP STATUS CODE: 403
2022-01-11T14:00:57.264Z [jfrt ] [ERROR] [6cdecc92eaf486c7] [etV3VirtualAndRemoteCommon:133] [ttp-nio-8081-exec-43] - Failed to convert artifactory url (https://repostaging.companysoftware.com:443/artifactory/api/nuget/v3/company-nuget-virtual/registration-semver2) to original remote url for repo: nuget-remote, package: xunit.core
java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
2022-01-11T14:01:56.786Z [jfrou] [ERROR] [2016c910242342cc] [external_topology.go:82 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2022-01-11T14:02:04.871Z [jfrou] [WARN ] [7f3fb3a30ade9665] [local_topology.go:268 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy"
2022-01-11T14:02:09.877Z [jfrou] [ERROR] [7f3fb3a30ade9665] [local_topology.go:128 ] [main ] - periodic send heartbeat failed for 4 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out
java.lang.OutOfMemoryError: Java heap space
-XX:OnOutOfMemoryError="kill -9 %p"
Executing /bin/sh -c "kill -9 3974"...
2022-01-11T14:02:11.185Z [jfrou] [WARN ] [5a89519a8048b91d] [local_topology.go:268 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy"
2022-01-11T14:02:11.196Z [jfrou] [ERROR] [79bb63bc55c1ed15] [external_topology.go:82 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": read tcp 127.0.0.1:55970-127.0.0.1:8040: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022/01/11 14:02:11 httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:56788->127.0.0.1:8045: read: connection reset by peer
2022-01-11T14:02:11.208Z 35[jfob ] [WARN ] [1ed879c85a5af005] [access_join.go:70 ] [main ] - Refreshing platform config change events gRPC stream - target server is unavailable - if issue persists check communication with access [access_client]
/opt/jfrog/artifactory/app/bin/artifactory.sh: line 359: 3974 Killed $TOMCAT_HOME/bin/catalina.sh run
Can you share your setup?
-The application server is the CentOS Linux release 7.9.2009 (Core)
-MySQL as backend Database
-Also, we are using the Apache as reverse proxy to apply the SSL certificate.
How are you running Artifactory?
-The Artifactory is container based: releases-docker.jfrog.io/jfrog/artifactory-pro:7.29.8
How much resources to you give it? Memory and CPU
The server has allocated:
2 vCPU
16 GB RAM
SSD
What are the java memory settings? Mostly interested in heap settings (Xms and Xmx)?
We are using the standard configuration. No changes were made in the Xms and Xms parameters.
As you are using Artifactory v7.x? You may refer to our System Requirements wiki page for the recommended hardware based on your environment.
Also, from on the error message shared, it looks like the user does not have proper permissions hence we see 403 errors. Please do validate and assign the required permissions to the user and let us know the results.
Sending HTTP error code 403:
Download request for repo:
path 'nuget-remote-cache:.nuGetV3/feed.json' is forbidden for user: 'lcunha'.

Issues getting kerberos/Windows AD login work for a web service

I have been struggling with this for quite a while now, and I can't get it to work.
Here is the setup:
I have a nginx webserver serving a django app at mywebapp.k8s.dal1.mycompany.io
It has the SPNEGO plugin compiled in and I have the following endpoint in my config:
location /ad-login {
uwsgi_pass django;
include /usr/lib/mycompany/lib/wsgi/uwsgi_params;
auth_gss on;
auth_gss_realm BURNERDEV1.DAL1.MYCOMPANY.IO;
auth_gss_service_name HTTP/mywebapp.k8s.dal1.mycompany.io;
auth_gss_allow_basic_fallback off;
}
My AD Domain controller is at burnerdev1.dal1.mycompany.io and I have the following users configured:
rep_movsd
portal
I run the following commands on the DC server in an Admin prompt:
ktpass -out krb5.keytab -mapUser portal#BURNERDEV1.DAL1.MYCOMPANY.IO +rndPass -mapOp set +DumpSalt -crypto AES256-SHA1 -ptype KRB5_NT_PRINCIPAL -princ HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
C:\Users\myself\Documents\keytab>ktpass -out krb5.keytab -mapUser portal#BURNERDEV1.DAL1.MYCOMPANY.IO +rndPass -mapOp set +DumpSalt -crypto AES256-SHA1 -ptype KRB5_NT_PRINCIPAL -princ HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
Targeting domain controller: dal1devdc1.burnerdev1.dal1.mycompany.io
Using legacy password setting method
Failed to set property 'servicePrincipalName' to 'HTTP/mywebapp.k8s.dal1.mycompany.io' on Dn 'CN=portal,CN=Users,DC=burnerdev1,DC=dal1,
DC=mycompany,DC=io': 0x13.
WARNING: Unable to set SPN mapping data.
If portal already has an SPN mapping installed for HTTP/mywebapp.k8s.dal1.mycompany.io, this is no cause for concern.
Building salt with principalname HTTP/mywebapp.k8s.dal1.mycompany.io and domain BURNERDEV1.DAL1.MYCOMPANY.IO (encryption type 18)...
Hashing password with salt "BURNERDEV1.DAL1.MYCOMPANY.IOHTTPmywebapp.k8s.dal1.mycompany.io".
Key created.
Output keytab to krb5.keytab:
Keytab version: 0x502
keysize 110 HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO ptype 1 (KRB5_NT_PRINCIPAL) vno 3 etype 0x12 (AES256-SHA1) k
eylength 32 (0x632d9ca3356374e9de490ec2f7718f9fb652b20da40bd212a808db4c46a72bc5)
C:\Users\myself\Documents\keytab>setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
Checking domain DC=burnerdev1,DC=dal1,DC=mycompany,DC=io
Registering ServicePrincipalNames for CN=portal,CN=Users,DC=burnerdev1,DC=dal1,DC=mycompany,DC=io
HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
Updated object
C:\Users\myself\Documents\keytab>
Now in the "Active Directory Users and Computers" section, I rightclicked the user and selected "Properties"
Then on the "Delegation" tab I set "Trust this user for delegation to any service (Kerberos only)"
Next I copy the krb5.keytab file to my webserver and restart the nginx container
On the Windows workstation which is part of the domain, I log on as rep_movsd - when I run klist:
C:\Users\rep_movsd>klist
Current LogonId is 0:0x208d7
Cached Tickets: (2)
#0> Client: rep_movsd # BURNERDEV1.DAL1.MYCOMPANY.IO
Server: krbtgt/BURNERDEV1.DAL1.MYCOMPANY.IO # BURNERDEV1.DAL1.MYCOMPANY.IO
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
Start Time: 7/16/2020 2:05:51 (local)
End Time: 7/16/2020 12:05:51 (local)
Renew Time: 7/23/2020 2:05:51 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
#1> Client: rep_movsd # BURNERDEV1.DAL1.MYCOMPANY.IO
Server: HTTP/mywebapp.k8s.dal1.mycompany.io # BURNERDEV1.DAL1.MYCOMPANY.IO
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
Start Time: 7/16/2020 2:06:01 (local)
End Time: 7/16/2020 12:05:51 (local)
Renew Time: 7/23/2020 2:05:51 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
I setup Firefox to do SPENGO authentication
Then I hit mywebapp.k8s.dal1.mycompany.io/ad-login and I get a 403 Forbidden error
The nginx server debug log shows:
[debug] 16#16: *195 Client sent a reasonable Negotiate header
[debug] 16#16: *195 GSSAPI authorizing
[debug] 16#16: *195 Use keytab /etc/krb5.keytab
[debug] 16#16: *195 Using service principal: HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
[debug] 16#16: *195 my_gss_name HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO
[debug] 16#16: *195 gss_accept_sec_context() failed: Cannot decrypt ticket for HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO using keytab key for HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO:
[debug] 16#16: *195 GSSAPI failed
[debug] 16#16: *195 http finalize request: 403, "/ad-login?" a:1, c:1
[debug] 16#16: *195 http special response: 403, "/ad-login?"
[debug] 16#16: *195 http set discard body
[debug] 16#16: *195 charset: "" > "utf-8"
[debug] 16#16: *195 HTTP/1.1 403 Forbidden
BTW while messing around earlier - I found that if I had set a fixed password for the "portal" user with ktpass and logged in as that account on the workstation, the login would succeed.
I was under the mistaken impression that I'd need to create a new keytab for every user and combine all of them.
Any help is greatly appreciated - I read so many conflicting docs its only confused me further and I've been losing sleep over this.
Thanks in advance!
I've read your problem statement carefully, and I think if you follow the steps I wrote below the issue will be solved.
On the DC server where you are creating the keytab, (1) UAC must be temporarily disabled. (2) The user creating the keytab must be a member of the Domain Admins group.
Ensure the SPN is not a duplicate, then remove the SPN from the Active Directory user account portal. This must be done before creating a new keytab using the same SPN against the same account. The below command is a one-liner, word-wrapping makes it look like two lines.
setspn -d HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal
Re-create the keytab again exactly as you did before.
You do not need to run the command setspn -A HTTP/mywebapp.k8s.dal1.mycompany.io#BURNERDEV1.DAL1.MYCOMPANY.IO portal because SPN was already set on the Active Directory user account by the ktpass command in step 3.
Replace the old keytab with the new keytab.
Restart the nginx webserver service.
Clear browser cache AND clear Kerberos case (klist purge).
Try it again.
You must do all these steps including the final step 7. Do not skip any.
You service account is named portal. A hash of this password is stored in both Active Directory and the keytab. Same hash is in both locations. The keytab on the nginix server is utilized to decrypt the inbound Kerberos service tickets to determine who the user is attempting to access the web app. More specifically, the GSS authentication does all the work, it uses the keytab to un-scramble the encrypted service tickets. The user rep_movsd does not have the service account credentials. It is part of the Active Directory domain, and when accessing the nginix web server, it gets it's own Kerberos service ticket and its identity is proven to the web server by simply being in possession of a service ticket that is decrypted by the keytab. If it wasn't part of the BURNERDEV1.DAL1.MYCOMPANY.IO domain, or had an expired password, or was a disabled account, it would not be able to get a service ticket and thus not prove its identity and fail authentication.
If you have time, please see my TechNet Wiki article on keytab creation and the logic behind it to help you better understand this complex subject.

WSO2 API Manager Custom Domain error

I have configured my wso2 with custom name by setting
-->
secu.helomyl.in
<!--
Host name to be used for the Carbon management console
-->
<MgtHostName>secu.helomyl.in</MgtHostName>
It starts and i can access the url and get wso2.But the below error is in the logs.Can you please help?
[2017-02-17 14:46:32,513] INFO - QpidServiceComponent Successfully connected to AMQP server on port 5673
[2017-02-17 14:46:32,514] WARN - QpidServiceComponent MQTT Transport is disabled as per configuration.
[2017-02-17 14:46:32,514] INFO - QpidServiceComponent WSO2 Message Broker is started.
[2017-02-17 14:46:32,533] WARN - PropertiesFileInitialContextFactory Unable to create factory:Illegal character in query between indicies 66 and 1
amqp://admin:admin#clientid/carbon?brokerlist='tcp://15.100.133.77 :5673'
^
[2017-02-17 14:46:33,044] INFO - PassThroughHttpSSLListener Starting Pass-through HTTPS Listener...
[2017-02-17 14:46:33,047] INFO - PassThroughListeningIOReactorManager Pass-through HTTPS Listener started on 0.0.0.0
Check the api-manager.xml in wso2am-2.0.0/repository/conf location. There is space in the below configuration. That causes the issue.
tcp://15.100.133.77 :5673

ERROR when sending data to my KAA server

When using first kaa server application and sending data to my Kaa server from outside I get this error: CONNACK message + KAASYNC message.
My configuration for kaa server is:
transport host...=localhost=My PUBLIC ip
My config mongo on kaa server is:
host: MY PUBLIC IP port: 27017
THIS IS WHAT I GET WHEN I COMPILE MY SDK
[pool-2-thread-1] INFO org.kaaproject.kaa.client.channel.failover.DefaultFailoverManager - Server [BOOTSTRAP, -1835393002] failed
[pool-2-thread-1] WARN org.kaaproject.kaa.client.channel.impl.DefaultChannelManager - Attempt to connect to the next bootstrap service will be made in 2000 ms, according to failover strategy decision
[pool-1-thread-1] INFO FirstKaaDemo - Sampled Temperature: 34
[pool-4-thread-14] INFO org.kaaproject.kaa.client.logging.strategies.RecordCountLogUploadStrategy - Need to upload logs - current count: 14, threshold: 1
[Thread-2] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Can't sync. Channel [default_operation_tcp_channel] is waiting for CONNACK message + KAASYNC message
[pool-6-thread-1] INFO org.kaaproject.kaa.client.channel.impl.channels.AbstractHttpChannel - Processing sync all for channel default_bootstrap_channel
[pool-1-thread-1] INFO FirstKaaDemo - Sampled Temperature: 25
[pool-4-thread-15] INFO org.kaaproject.kaa.client.logging.strategies.RecordCountLogUploadStrategy - Need to upload logs - current count: 15, threshold: 1
[Thread-2] INFO org.kaaproject.kaa.client.channel.impl.channels.DefaultOperationTcpChannel - Can't sync. Channel [default_operation_tcp_channel] is waiting for CONNACK message + KAASYNC message
Step 1: Change Kaa Host/IP
To VM ip address. My case is 192.168.1.142
Step 2: Regenerate SDK and recompide and run. Done

WSO2 Identity Server and WSO2 API Manager integration - java.rmi.server.ExportException: Port already in use: 9999;

I have integrated WSO2 Identity server and WSO2 API Manager.
While starting the WSO2 Id Server , I am getting the below error in console..
ERROR {org.wso2.carbon.core.init.JMXServerManager} - Could not create the RMI local registry
java.rmi.server.ExportException: Port already in use: 9999; nested exception is:
java.net.BindException: Address already in use: JVM_Bind
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:341)
at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249)
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:208)
at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:152)
at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:137)
Can anyone help?
But its starting successfully with the following message
INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - Server : WSO2 Identity Server-5.2.0
[2016-12-27 15:31:13,744] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - WSO2 Carbon started in 57 sec
[2016-12-27 15:31:14,909] INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} - Mgt Console URL : https://localhost:9443/carbon/
[2016-12-27 15:31:14,948] INFO {org.wso2.carbon.identity.authenticator.x509Certificate.internal.X509CertificateServiceComponent} - X509 Certificate Servlet activated successfully..
Before i started wso2server.bat , there is nothing in the port..
My machine's JAVA_OPTS was being set to debug mode and this was causing WSO2 to start in debug mode . Thus it was listening to 9999 port.
I have removed the JAVA_OPTS and now able to start it properly.
related issue
WSO2 Identity Server listening to port 9999

Resources