Can't upload to shinyapps.io - r

I am getting an issue uploading to shinyapps.io. I had issues with the firewall before so I am wondering if maybe there are leftover issues there but I can't find anything that would indicate that in the error message.
Preparing to deploy application...DONE
Uploading bundle for application: 4222252...Error in force(code) : Could not upload file.
Calls: <Anonymous> -> withStatus -> force
Execution halted
The application is a very simple shinyapp that I used to test. There's really only a dummy data.table and a linechart created with the billboarder package. Nothing crazy going on there.
I tried to log it and this is what I got:
* Connection 2 seems to be dead!
* Closing connection 2
* schannel: shutting down SSL/TLS connection with api.shinyapps.io port 443
* Connection 3 seems to be dead!
* Closing connection 3
* schannel: shutting down SSL/TLS connection with api.shinyapps.io port 443
* Found bundle for host api.shinyapps.io: 0x1fd5bd908e0 [can pipeline]
* Trying 18.204.73.112...
* TCP_NODELAY set
* Connected to api.shinyapps.io (18.204.73.112) port 443 (#5)
> GET /v1/applications/?filter=account_id:932539&count=100&offset=0 HTTP/1.1
Host: api.shinyapps.io
User-Agent: rsconnect/0.8.17
Accept: */*
Accept-Encoding: deflate, gzip
Date: Thu, 03 Jun 2021 08:06:19 GMT
X-Auth-Token: EB5EF5C5A9A0A1AD8021108F4973EA4F
X-Auth-Signature: NDA5YzIwY2E3YTMxMTQ1Y2U4YzYwZjdhMTA2ODM2NzJmOWE1MDExNTI3NmFiODRmNzFkMjUzZmJlZDA3YWRkZg==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e
< HTTP/1.1 200 OK
< Date: Thu, 03 Jun 2021 08:06:20 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 5879
< Connection: keep-alive
< Etag: "36941c41b5fad4ebdec0fb9ed5ef1154e9a52e79"
< Server: TornadoServer/4.5.3
<
* Connection #5 to host api.shinyapps.io left intact
* Found bundle for host api.shinyapps.io: 0x1fd5bd908e0 [can pipeline]
* Hostname api.shinyapps.io was found in DNS cache
* Trying 18.204.73.112...
* TCP_NODELAY set
* Connected to api.shinyapps.io (18.204.73.112) port 443 (#6)
> GET /v1/applications/?filter=account_id:932539&count=100&offset=0 HTTP/1.1
Host: api.shinyapps.io
User-Agent: rsconnect/0.8.17
Accept: */*
Accept-Encoding: deflate, gzip
Date: Thu, 03 Jun 2021 08:06:31 GMT
X-Auth-Token: EB5EF5C5A9A0A1AD8021108F4973EA4F
X-Auth-Signature: MjdiYWUzZWYzODIzNmM5NzMxMGZhOGM5MTQ4Y2VjMTFlYjdlODFkMWRhNDU4ZGFkNjNmMGQ5YmVlMTllZGQ5Ng==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e
< HTTP/1.1 200 OK
< Date: Thu, 03 Jun 2021 08:06:32 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 5879
< Connection: keep-alive
< Etag: "36941c41b5fad4ebdec0fb9ed5ef1154e9a52e79"
< Server: TornadoServer/4.5.3
<
* Connection #6 to host api.shinyapps.io left intact
* Connection 4 seems to be dead!
* Closing connection 4
* schannel: shutting down SSL/TLS connection with api.shinyapps.io port 443
* Found bundle for host api.shinyapps.io: 0x1fd5bd908e0 [can pipeline]
* Trying 3.231.196.102...
* TCP_NODELAY set
* Connected to api.shinyapps.io (3.231.196.102) port 443 (#7)
> GET /v1/applications/?filter=account_id:932539&count=100&offset=0 HTTP/1.1
Host: api.shinyapps.io
User-Agent: rsconnect/0.8.17
Accept: */*
Accept-Encoding: deflate, gzip
Date: Thu, 03 Jun 2021 08:07:34 GMT
X-Auth-Token: EB5EF5C5A9A0A1AD8021108F4973EA4F
X-Auth-Signature: YjMzMzNlMDQ0MTEwZmZmYjQxOTMxMzA1N2JkMzc4ZmM0MTA5ZGE4ZmE5NGFmYmY5MGE0ZWZmNjAzOTFmZjc5Yg==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e
* schannel: failed to decrypt data, need more data
< HTTP/1.1 200 OK
< Date: Thu, 03 Jun 2021 08:07:36 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 5879
< Connection: keep-alive
< Etag: "36941c41b5fad4ebdec0fb9ed5ef1154e9a52e79"
< Server: TornadoServer/4.5.3
<
* Connection #7 to host api.shinyapps.io left intact
* Found bundle for host api.shinyapps.io: 0x1fd5bd908e0 [can pipeline]
* Hostname in DNS cache was stale, zapped
* Trying 3.231.196.102...
* TCP_NODELAY set
* Connected to api.shinyapps.io (3.231.196.102) port 443 (#8)
> GET /v1/applications/?filter=account_id:932539&count=100&offset=0 HTTP/1.1
Host: api.shinyapps.io
User-Agent: rsconnect/0.8.17
Accept: */*
Accept-Encoding: deflate, gzip
Date: Thu, 03 Jun 2021 08:09:51 GMT
X-Auth-Token: EB5EF5C5A9A0A1AD8021108F4973EA4F
X-Auth-Signature: MDhlNzE0MDFhMmJhNjU3NTA2Y2UyOTk3ZWY2OWQ1ZDgyNjMyMWRhYjcyOTNhOTMyMDFkMGY4YzIwZmJiZWM1MQ==; version=1
X-Content-Checksum: d41d8cd98f00b204e9800998ecf8427e
< HTTP/1.1 200 OK
< Date: Thu, 03 Jun 2021 08:09:51 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 5879
< Connection: keep-alive
< Etag: "36941c41b5fad4ebdec0fb9ed5ef1154e9a
GET /v1/applications/?filter=account_id:932539&count=100&offset=0 870ms
If anyone has any idea what's happening here I would love to hear it!

Related

Why curl returns different response from telnet from a web server

Hello I did run curl command and equal telnet command but response is different.
I try to get HTTP HEADER response code from a web server.
curl returns correct response, page indeed doesn't exist. But telnet with equal request return different response.
Please see CURL code:
$ curl -vI "https://www.abcanchorage.org/not-exist-page-con2uiu28itact"
* About to connect() to www.abcanchorage.org port 443 (#0)
* Trying 185.230.63.96...
* Connected to www.abcanchorage.org (185.230.63.96) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=abcanchorage.org
* start date: Nov 09 17:32:52 2020 GMT
* expire date: Feb 07 17:32:52 2021 GMT
* common name: abcanchorage.org
* issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
> HEAD /not-exist-page-con2uiu28itact HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.abcanchorage.org
> Accept: */*
>
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Date: Mon, 14 Dec 2020 14:06:25 GMT
Date: Mon, 14 Dec 2020 14:06:25 GMT
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8
< Connection: keep-alive
Connection: keep-alive
< content-language: en
content-language: en
< x-wix-request-id: 1607954785.82658627446080423994
x-wix-request-id: 1607954785.82658627446080423994
< Age: 0
Age: 0
< Server-Timing: cache;desc=miss, varnish;desc=miss, dc;desc=84
Server-Timing: cache;desc=miss, varnish;desc=miss, dc;desc=84
< X-Seen-By: r5KTLwzxoi1C+SXup0UeuQ==,sHU62EDOGnH2FBkJkG/Wx8EeXWsWdHrhlvbxtlynkVg1leqePNkwQ1H/U0dj1nMN,2d58ifebGbosy5xc+FRalthBvkD9LHf+OIny1MrcDwiq6+5o8xu+S2UUMqtKcebjUMQY1HiUPm50af1ZAsGkCw==,2UNV7KOq4oGjA5+PKsX47Gzh5saLoQp8TIRIohc0Wac=,m0j2EEknGIVUW/liY8BLLuvhI/meCohDY7RevwAJ7JU=,w4q8mm9FnmU4emOs6psVXTk8VV0IpMOWw3zRTVWAPzKTzRA6xkSHdTdM1EufzDIPWIHlCalF7YnfvOr2cMPpyw==,8OhaUUQpIrZVCQED4XmuQ40tjCf+ijnQjDvVBuKrqCUYXyC7l2oN0jRBomMsk5tniLmOBCJX9PwPq0FFNfh5cw==
X-Seen-By: r5KTLwzxoi1C+SXup0UeuQ==,sHU62EDOGnH2FBkJkG/Wx8EeXWsWdHrhlvbxtlynkVg1leqePNkwQ1H/U0dj1nMN,2d58ifebGbosy5xc+FRalthBvkD9LHf+OIny1MrcDwiq6+5o8xu+S2UUMqtKcebjUMQY1HiUPm50af1ZAsGkCw==,2UNV7KOq4oGjA5+PKsX47Gzh5saLoQp8TIRIohc0Wac=,m0j2EEknGIVUW/liY8BLLuvhI/meCohDY7RevwAJ7JU=,w4q8mm9FnmU4emOs6psVXTk8VV0IpMOWw3zRTVWAPzKTzRA6xkSHdTdM1EufzDIPWIHlCalF7YnfvOr2cMPpyw==,8OhaUUQpIrZVCQED4XmuQ40tjCf+ijnQjDvVBuKrqCUYXyC7l2oN0jRBomMsk5tniLmOBCJX9PwPq0FFNfh5cw==
< Cache-Control: no-cache
Cache-Control: no-cache
< Vary: Accept-Encoding
Vary: Accept-Encoding
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
<
* Connection #0 to host www.abcanchorage.org left intact
Please see TELNET code:
$ telnet www.abcanchorage.org 80
Trying 35.246.6.109...
Connected to www.abcanchorage.org.
Escape character is '^]'.
HEAD /not-exist-page-con2uiu28itact HTTP/1.1
User-Agent: curl/7.29.0
Host: www.abcanchorage.org
Accept: */*
HTTP/1.1 301 Moved Permanently
Date: Mon, 14 Dec 2020 14:08:50 GMT
Content-Length: 0
Connection: keep-alive
location: https://www.abcanchorage.org/not-exist-page-con2uiu28itact
Age: 1496
Server-Timing: cache;desc=hit, varnish;desc=hit, dc;desc=euw2
X-Seen-By: sHU62EDOGnH2FBkJkG/Wx8EeXWsWdHrhlvbxtlynkVgmNySqidgeEPHXBvm3U9iS,2d58ifebGbosy5xc+FRalhEF/w8i5SZb0AgDKvjhMuBVlHGeDiRYk3btR6xZR3zaGgqFbFMYwiXnFojPwdof6LPb61OjwxlOgfM3AWuO4IQ=,2UNV7KOq4oGjA5+PKsX47LZ7Kls+1whC/C/a0aUIqJE=
Cache-Control: no-cache
Expires: -1
X-Wix-Request-Id: 1607954930.2811039255969122439
Server: Pepyaka/1.19.0
^]quit
telnet> quit
Connection closed.

Use nginx-proxy and wordpress in docker

I try to host some wordpress with an nginx proxy above:
This is my docker-compose.yml
proxy:
image: jwilder/nginx-proxy:0.4.0
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
# SSL Certificates
- /etc/letsencrypt/live/mysupersite.com/cert.pem:/etc/nginx/certs/site.com.crt:ro
- /etc/letsencrypt/live/mysupersite.com/privkey.pem:/etc/nginx/certs/site.com.key:ro
wordpress:
image: wordpress:4.8.1-php5.6-apache
expose:
- 80
volumes:
- /var/www/html:/var/www/html
restart: always
environment:
- WORDPRESS_DB_HOST=mysql
- WORDPRESS_DB_USER=dev
- WORDPRESS_DB_PASSWORD=passwd
- WORDPRESS_DB_NAME=db
- VIRTUAL_HOST=mysupersite.com,www.mysupersite.com
Everything seems to start well. I have 2 running containers.
But I'm not able to visit my website on https://mysupersite.com in my browser.
What part am I missing? Do I need extra configuration somewhere?
I see this error in my browser:
The page isn’t redirecting properly
output o cur -L -v https://www.mysupersite.com is the following:
* Connection #0 to host www.mysupersite.com left intact
* Issue another request to this URL: 'https://www.mysupersite.com/'
* Found bundle for host www.mysupersite.com: xx [can pipeline]
* Re-using existing connection! (#0) with host www.mysupersite.com
* Connected to www.mysupersite.com (IP) port 443 (#0)
> GET / HTTP/1.1
> Host: www.mysupersite.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.9.12
< Date: Tue, 22 Aug 2017 07:33:22 GMT
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Connection: keep-alive
< X-Powered-By: PHP/5.6.31
< Set-Cookie: PHPSESSID=xxx; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< X-Pingback: http://www.mysupersite.com/xmlrpc.php
< Location: https://www.mysupersite.com/
< Strict-Transport-Security: max-age=31536000
<
* Connection #0 to host www.mysupersite.com left intact
* Issue another request to this URL: 'https://www.mysupersite.com/'
* Found bundle for host www.mysupersite.com: xxxx [can pipeline]
* Re-using existing connection! (#0) with host www.mysupersite.com
* Connected to www.mysupersite.com (IP) port 443 (#0)
> GET / HTTP/1.1
> Host: www.mysupersite.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.9.12
< Date: Tue, 22 Aug 2017 07:33:23 GMT
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Connection: keep-alive
< X-Powered-By: PHP/5.6.31
< Set-Cookie: PHPSESSID=xxxx; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< X-Pingback: http://www.mysupersite.com/xmlrpc.php
< Location: https://www.mysupersite.com/
< Strict-Transport-Security: max-age=31536000
<
* Connection #0 to host www.mysupersite.com left intact
* Maximum (50) redirects followed
curl: (47) Maximum (50) redirects followed

HTTP negotiation fails from Windows to Linux

I am trying to authenticate my local Windows 10 machine to a web service running inside a docker container. To be more specific, this container is running Hadoop services and a MIT Kerberos KDC. I have installed MIT Kerberos for Windows on my local machine and have successfully gotten a ticket hadoop/quickstart.cloudera#CLOUDERA from the KDC. When I authenticate with the same principal inside my container and call this command: curl -i --negotiate -u : "http://quickstart.cloudera:50070/webhdfs/v1/?op=GETFILESTATUS" I get a valid response. However, the same command run from my Windows machine returns this error:
Error 403 GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)
Can anyone familiar with SSPI/GSSAPI give me some insight on what the problem here could be?
I have the environment variable KRB5CCNAME=<path to ccache> set correctly. This is the cURL info from my Windows command prompt:
curl 7.52.1 (x86_64-w64-mingw32) libcurl/7.52.1 WinSSL zlib/1.2.8 WinIDN libssh2/1.7.0_DEV
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz
Let me know if you need any more information about my environment setup.
Update:
Here is the full HTTP response from my Windows machine when I run the command with the -v flag added for verbosity.
* timeout on name lookup is not supported
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to quickstart.cloudera (127.0.0.1) port 50070 (#0)
> GET /webhdfs/v1/?op=GETFILESTATUS HTTP/1.1
> Host: quickstart.cloudera:50070
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 401 Authentication required
HTTP/1.1 401 Authentication required
< Cache-Control: must-revalidate,no-cache,no-store
Cache-Control: must-revalidate,no-cache,no-store
< Date: Fri, 10 Feb 2017 19:41:27 GMT
Date: Fri, 10 Feb 2017 19:41:27 GMT
< Pragma: no-cache
Pragma: no-cache
< Date: Fri, 10 Feb 2017 19:41:27 GMT
Date: Fri, 10 Feb 2017 19:41:27 GMT
< Pragma: no-cache
Pragma: no-cache
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1
< WWW-Authenticate: Negotiate
WWW-Authenticate: Negotiate
< Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
< Content-Length: 1404
Content-Length: 1404
< Server: Jetty(6.1.26.cloudera.4)
Server: Jetty(6.1.26.cloudera.4)
<
* Ignoring the response-body
* Curl_http_done: called premature == 0
* Connection #0 to host quickstart.cloudera left intact
* Issue another request to this URL: 'http://quickstart.cloudera:50070/webhdfs/v1/?op=GETFILESTATUS'
* Found bundle for host quickstart.cloudera: 0x817220 [can pipeline]
* Re-using existing connection! (#0) with host quickstart.cloudera
* Connected to quickstart.cloudera (127.0.0.1) port 50070 (#0)
* Server auth using Negotiate with user ''
> GET /webhdfs/v1/?op=GETFILESTATUS HTTP/1.1
> Host: quickstart.cloudera:50070
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAFopAAAADw==
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 403 GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)
HTTP/1.1 403 GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)
< Cache-Control: must-revalidate,no-cache,no-store
Cache-Control: must-revalidate,no-cache,no-store
< Date: Fri, 10 Feb 2017 19:41:27 GMT
Date: Fri, 10 Feb 2017 19:41:27 GMT
< Pragma: no-cache
Pragma: no-cache
< Date: Fri, 10 Feb 2017 19:41:27 GMT
Date: Fri, 10 Feb 2017 19:41:27 GMT
< Pragma: no-cache
Pragma: no-cache
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1
< Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
< Content-Length: 1546
Content-Length: 1546
< Server: Jetty(6.1.26.cloudera.4)
Server: Jetty(6.1.26.cloudera.4)
<
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 403 GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /webhdfs/v1/. Reason:
<pre> GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
</body>
</html>
* Curl_http_done: called premature == 0
* Closing connection 0
In contrast, this is the response from running the same command inside my container:
* About to connect() to quickstart.cloudera port 50070 (#0)
* Trying 172.18.0.2... connected
* Connected to quickstart.cloudera (172.18.0.2) port 50070 (#0)
> GET /webhdfs/v1/?op=GETFILESTATUS HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: quickstart.cloudera:50070
> Accept: */*
>
< HTTP/1.1 401 Authentication required
HTTP/1.1 401 Authentication required
< Cache-Control: must-revalidate,no-cache,no-store
Cache-Control: must-revalidate,no-cache,no-store
< Date: Fri, 10 Feb 2017 21:15:39 GMT
Date: Fri, 10 Feb 2017 21:15:39 GMT
< Pragma: no-cache
Pragma: no-cache
< Date: Fri, 10 Feb 2017 21:15:39 GMT
Date: Fri, 10 Feb 2017 21:15:39 GMT
< Pragma: no-cache
Pragma: no-cache
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1
< WWW-Authenticate: Negotiate
WWW-Authenticate: Negotiate
< Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
< Content-Length: 1404
Content-Length: 1404
< Server: Jetty(6.1.26.cloudera.4)
Server: Jetty(6.1.26.cloudera.4)
<
* Ignoring the response-body
* Connection #0 to host quickstart.cloudera left intact
* Issue another request to this URL: 'http://quickstart.cloudera:50070/webhdfs/v1/?op=GETFILESTATUS'
* Re-using existing connection! (#0) with host quickstart.cloudera
* Connected to quickstart.cloudera (172.18.0.2) port 50070 (#0)
* Server auth using GSS-Negotiate with user ''
> GET /webhdfs/v1/?op=GETFILESTATUS HTTP/1.1
> Authorization: Negotiate YIICkQYJKoZIhvcSAQICAQBuggKAMIICfKADAgEFoQMCAQ6iBwMFAAAAAACjggGBYYIBfTCCAXmgAwIBBaEKGwhDTE9VREVSQaImMCSgAwIBA6EdMBsbBEhUVFAbE3F1aWNrc3RhcnQuY2xvdWRlcmGjggE8MIIBOKADAgESoQMCAQKiggEqBIIBJnWbXp9WlAVk0nGIqD7T25On1+OCPzXDX/aoH01FTjJbEwrYj4cMML7Tf6jaKDIANEh57kTJOvPknL3CWHI1c3LeNpt1Ir8H2M3Zvk91HpbWXzv5WJTeUOK9L6zTaKFEs/dKQgD7VzmHKDJtMyVKQWLLVU8JuyKAV6iM4FvxfZ+WDF8QCk7pxwjgX1OT7jv9jR28MPpsIweqUYYnneJxVTsxgmsHdOvj5wpMGy9RA9R8jtR+Wh3l5r3a3zcUTmGwAqY+NXhBkviSTw+DgitnipYh5tXBRhNqGfk86qWAdodGgL+SdkwwGsq91PyYQiMCLXjWx90aBOEFeZLDyqBaXlMIZ3TT3urUQEuB206+8KNw1n6N6+u+ZY4QT7NJyVZqHbnOR5V5maSB4TCB3qADAgESooHWBIHTyaBY3PormkycaX9uf/lLe6ISYnItZikJqslAGpJVnla2HXYvhFjqn5yr8td1pw3zzdnDEZx3a9EylIrRQD5IoIvHCzd0mlJhHFj4xxISM5hxlMiL8DewMjGsVcDveqpHw1SyxIsrEPOhe62HuRXS7c1Z9kYkP6KldzyAJOttOVYCuL36hOxwEFqJtbWk1/f9gfTdzmxQmEASM3/wsj2Q/WYCZY/hazDIz6dmHsyla/F6NXGK0BwRnHHBCqSHe7GdWmBDNjHiuo6R0/YvqTl5Uvf0Rw==
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: quickstart.cloudera:50070
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Cache-Control: no-cache
Cache-Control: no-cache
< Expires: Fri, 10 Feb 2017 21:15:39 GMT
Expires: Fri, 10 Feb 2017 21:15:39 GMT
< Date: Fri, 10 Feb 2017 21:15:39 GMT
Date: Fri, 10 Feb 2017 21:15:39 GMT
< Pragma: no-cache
Pragma: no-cache
< Expires: Fri, 10 Feb 2017 21:15:39 GMT
Expires: Fri, 10 Feb 2017 21:15:39 GMT
< Date: Fri, 10 Feb 2017 21:15:39 GMT
Date: Fri, 10 Feb 2017 21:15:39 GMT
< Pragma: no-cache
Pragma: no-cache
< Content-Type: application/json
Content-Type: application/json
< Set-Cookie: hadoop.auth="u=hadoop&p=hadoop/quickstart.cloudera#CLOUDERA&t=kerberos&e=1486797339425&s=BqBHGJ+/FxxeSR0ayBXHOrfPkwU="; Path=/; HttpOnly
Set-Cookie: hadoop.auth="u=hadoop&p=hadoop/quickstart.cloudera#CLOUDERA&t=kerberos&e=1486797339425&s=BqBHGJ+/FxxeSR0ayBXHOrfPkwU="; Path=/; HttpOnly
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
< Server: Jetty(6.1.26.cloudera.4)
Server: Jetty(6.1.26.cloudera.4)
<
* Connection #0 to host quickstart.cloudera left intact
* Closing connection #0
{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":5,"fileId":16385,"group":"supergroup","length":0,"modificationTime":1459909590753,"owner":"hdfs","pathSuffix":"","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}
The answer is very easy in this case. Curl on Windows is compiled with SSPI. When SSPI is requested to perform SPNEGO, it tries Kerberos, which fails here. Likely "Server not found in database" (use Wireshark) and then falls back to NTLM. It sends a raw NTLM token to your JGSS-backed server which rejects the token because
This is not a SPNEGO wrapped token, but a raw NTLM token
Java does not support NTLM
Here is sample code how to intercept this and respond with a meaningful message. Raise an issue with Hadoop.
Moreover, your Curl version 7.19.7 on Linux is extremely old and unsecure, you should upgrade immediately AND the SPNEGO Authenticator on Jetty is broken because it does not respond with a context-completion token. To sum up, the entire authentication should not be trusted because it is faulty. See RFC 7546.

RCurl (with digest authentication) not setting realm correctly on Windows

I've been working on an R interface to a HTTP API with digest authentication and I've been running into a problem wherein the request will work absolutely fine on my non-Windows OSs, but I always get 401 status when running exactly the same code on Windows.
I'm currently trying to do it with RCurl, but the same thing was happening with httr when I tried that.
Also the API is unfotunately proprietary so I've had to change all the URLs, sorry.
On my non-Windows OSs I get the following behaviour:
rprompt> getURL('http://demo.someapi.net/some/url', userpwd="demo:demo", httpauth=1L, verbose=TRUE)
* Trying 195.224.16.34...
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* TLS 1.0 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate: *.someapi.net
* Server certificate: RapidSSL SHA256 CA - G3
* Server certificate: GeoTrust Global CA
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo0NTozMw", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:44:33 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host demo.someapi.net left intact
* Issue another request to this URL: 'https://demo.someapi.net/some/url'
* Found bundle for host demo.someapi.net: 0x7f96c8d55af0
* Re-using existing connection! (#0) with host demo.someapi.net
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* Server auth using Digest with user 'demo'
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Authorization: Digest username="demo", realm="company.product", nonce="MTEvMDcvMjAxNiAwODo0NTozMw", uri="/some/url", cnonce="YjRkMDQxYmM4MDFkYTMxOWZhNTViNGNmYTM5YzQyNGI=", nc=00000001, qop=auth, response="5d9643d083b2380f12d71855a98ceac3", opaque="0000000000000000", algorithm="MD5"
Accept: */*
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Length: 981
< Content-Type: application/json; charset=utf-8
< Server: Microsoft-IIS/7.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:44:33 GMT
<
* Connection #0 to host demo.someapi.net left intact
and evertything works exactly as we expect it to. On Windows however we get this:
rprompt> getURL('http://demo.someapi.net/some/url', userpwd="demo:demo", httpauth=1L, verbose=TRUE)
* Trying 195.224.16.34...
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: C:/Users/username/Documents/R/win-library/3.3/RCurl/etc/ca-bundle.crt
CApath: none
* SSL connection using TLSv1.0 / ECDHE-RSA-AES256-SHA
* Server certificate:
* subject: OU=GT56411961; OU=See www.rapidssl.com/resources/cps (c)15; OU=Domain Control Validated - RapidSSL(R); CN=*.someapi.net
* start date: 2015-01-26 09:31:11 GMT
* expire date: 2018-03-28 16:30:51 GMT
* subjectAltName: demo.someapi.net matched
* issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA - G3
* SSL certificate verify ok.
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo1MjowOA", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:51:07 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host demo.someapi.net left intact
* Issue another request to this URL: 'https://demo.someapi.net/some/url'
* Found bundle for host demo.someapi.net: 0xaa60b80
* Re-using existing connection! (#0) with host demo.someapi.net
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* Server auth using Digest with user 'demo'
> GET /some/url HTTP/1.1
Authorization: Digest username="demo",realm="",nonce="MTEvMDcvMjAxNiAwODo1MjowOA",uri="/some/url",cnonce="553f542ddef0e3c265e50539297bad81",nc=00000001,algorithm=MD5,response="1ec58793bb1d8142f09af112b905fa36",qop="auth",opaque="0000000000000000"
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
* Authentication problem. Ignoring this.
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo1MjowOA", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:51:07 GMT
< Content-Length: 1293
<
* Connection #0 to host demo.someapi.net left intact
which just returns a 401 landing page HTML.
The issue seems to be that the realm field is empty, but I have no idea how to fix this or even how to work around it.
It should be noted that both .NET's webclient and Python's requests library handles things fine, but unfortunately this has to be done in R.
I'm happy to use any R packages that are needed to help solve this.
Thanks.
For anyone else who ends up with a similar problem, you can get around it by using httr and using your own handles.
make.request <- function (url, user, pass) {
handle <- httr::handle(url)
response <- GET(url=NULL, authenticate(user, pass, type="digest"), handle=handle)
# error checking and stuff...
}
I'm not sure what the issue is with RCurl though.

Opengraph HTTP 301 redirect wrong location

Open the opengraph debugger page and try it with this url: http://www.jetradar.com/?marker=12345 - I get this as a result: http://screencloud.net/v/rHlW - for some reason it tries this obviously wrong redirect url "http://www.jetradar.com/,%20http://www.jetradar.com/" though ask the server via curl - you get nothing suspicious.
$ curl -v http://www.jetradar.com/?marker=12345
* Adding handle: conn: 0x20e9b20
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x20e9b20) send_pipe: 1, recv_pipe: 0
* About to connect() to www.jetradar.com port 80 (#0)
* Trying 5.10.84.53...
* Connected to www.jetradar.com (5.10.84.53) port 80 (#0)
> GET /?marker=12345 HTTP/1.1
> User-Agent: curl/7.32.0
> Host: www.jetradar.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
* Server nginx/1.5.12 is not blacklisted
< Server: nginx/1.5.12
< Date: Fri, 07 Nov 2014 12:43:03 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Status: 301 Moved Permanently
< Location: http://www.jetradar.com/
< X-UA-Compatible: IE=Edge,chrome=1
< Set-Cookie: marker=12345; path=/; expires=Sun, 07-Dec-2014 12:43:03 GMT
< X-Request-Id: ac8d046ea03191f637cbdf8d9129a1a0
< X-Runtime: 0.011112
< X-Rack-Cache: miss
< X-Powered-By: Phusion Passenger 4.0.19
< Location: http://www.jetradar.com/
< X-Page-Speed: 1.8.31.4-4009
< Cache-Control: max-age=0, no-cache
<
* Connection #0 to host www.jetradar.com left intact
<html><head/><body>You are being redirected.</body></html>
Any ideas?
Hello from jetradar team to stackoverflow community:)

Resources