I would like to setup a proxy to cache microsoft sharepoint online. However, even getting some progresses, I still could not understand why it is still TCP_MISS.
OS: Centos 7
Squid: initially 3.3.8 (centos default) and upgraded to 3.5.22 later
ssl_bump is utilised for https interception
refresh pattern:
refresh_pattern -i .(xz|rpm) 14400 50% 18000 override-expire override-lastmod ignore-reload ignore-no-cache ignore-private ignore-auth
the test file for downloading is kernel-plus-3.10.0-327.10.1.el7.centos.plus.x86_64.rpm which was uploaded to sharepoint.
with "debug_options 22,70,9", I am getting messages in cache.log as follows:
2016/12/08 16:36:10.383 kid1| ctx: exit level 0
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(291) refreshCheck: checking freshness of 'https ://xxx.sharepoint.com/sites/its/itst/int/kernel-plus-3.10.0-327.10.1.el7.centos.plus.x86_64.rpm'
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(312) refreshCheck: Matched '.(xz|rpm) 864000 50%% 1080000'
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(314) refreshCheck: age: 2225
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(316) refreshCheck: check_time: Thu, 08 Dec 2016 21:36:10 GMT
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(318) refreshCheck: entry->timestamp: Thu, 08 Dec 2016 20:59:05 GMT
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(173) refreshStaleness: STALE: expires 1481230745 < check_time 1481232970
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(338) refreshCheck: Staleness = 2225
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(362) refreshCheck: YES: Must revalidate stale object (origin set no-cache or private)
2016/12/08 16:36:10.384 kid1| 22,3| refresh.cc(654) getMaxAge: getMaxAge: 'https ://xxx.sharepoint.com/sites/its/itst/int/kernel-plus-3.10.0-327.10.1.el7.centos.plus.x86_64.rpm'
2016/12/08 16:36:12.247 kid1| ctx: enter level 0: 'https ://xxx.sharepoint.com/sites/its/itst/int/kernel-plus-3.10.0-327.10.1.el7.centos.plus.x86_64.rpm'
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(291) refreshCheck: checking freshness of 'https ://xxx.sharepoint.com/sites/its/itst/int/kernel-plus-3.10.0-327.10.1.el7.centos.plus.x86_64.rpm'
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(312) refreshCheck: Matched '.(xz|rpm) 864000 50%% 1080000'
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(314) refreshCheck: age: 63
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(316) refreshCheck: check_time: Thu, 08 Dec 2016 21:37:12 GMT
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(318) refreshCheck: entry->timestamp: Thu, 08 Dec 2016 21:36:09 GMT
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(173) refreshStaleness: STALE: expires 1481232969 < check_time 1481233032
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(338) refreshCheck: Staleness = 63
2016/12/08 16:36:12.247 kid1| 22,3| refresh.cc(497) refreshCheck: NO: Serving from cache - even though explicit expiry has passed, we enforce Min value (override-expire option)
2016/12/08 16:36:12.247 kid1| 22,3| http.cc(482) cacheableReply: YES because HTTP status 200
and TCP_MISS in access.log.
my questions are:
1. why did squid refreshcheck twice on one downloading request?
2. I guess that the squid decides to "Serving from cache" in the second refreshcheck, why still TCP_MISS in access.log?
if offline_mode is on, the cache has served the download happily with TCP_OFFLINE_HIT.
Please advise how could I fix the issue.
Thanks in advance.
Related
Im trying to send a JSON using the angular 8 HttpClient to an ASP.net core backend using the following code:
import { HttpClient, HttpHeaders} from '#angular/common/http';
import { User } from '#/_models';
login(username, password) {
return this.http.post<any>(`https://${config.apiUrl}/api/User/LoginUser`,
JSON.stringify({
"username": username,
"password": password
})
).pipe(map(user => {
// store user details and jwt token in local storage to keep user logged in between page refreshes
localStorage.setItem('currentUser', JSON.stringify(user));
console.log(user);
this.currentUserSubject.next(user);
return user;
}));
}
}
However, when trying to send the JSON when logging in to the login page which is hosted using the command npm start, so
localhost:8080/login
is sending the request.
The request headers are:
POST /api/User/LoginUser HTTP/1.1
Host: (Checked_and_correct_API_URL)
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: http://localhost:8080/login
Content-Type: text/plain
Content-Length: 49
Origin: http://localhost:8080
DNT: 1
Connection: keep-alive
the response headers are:
HTTP/2.0 415 Unsupported Media Type
content-type: application/problem+json; charset=utf-8
server: Microsoft-IIS/10.0
x-powered-by: ASP.NET
date: Thu, 19 Dec 2019 13:10:01 GMT
content-length: 147
X-Firefox-Spdy: h2
and the sent parameters are:
{"username":"jeboimarc2","password":"KUTminor1!"}
Which is also correct
Can anyone tell me why i am getting the unsupported media type error? Thanks in advance.
Pass raw json data, don't convert it to string.
Try like this:
return this.http.post<any>(`https://${config.apiUrl}/api/User/LoginUser`,{"username": username, "password": password })
I have a problem with my squid v3.5.27.
squid.conf
acl auth proxy_auth REQUIRED
acl basicauth proxy_auth_regex -i service
http_access deny !auth
http_access deny basicauth
http_access allow all
http_port 0.0.0.0:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/etc/squidCA.pem
ssl_bump bump all
sslproxy_cert_error deny all
sslproxy_flags DONT_VERIFY_PEER
sslcrtd_program /usr/lib64/squid/ssl_crtd -s /etc/squid/ssl_db -M 10MB
Iam authentificated by USERTEST, but get TCP_DENIED/403, when I try to open any sites
access.log:
17/May/2018:12:35:18 +0300 10.0.5.177 USERTEST TCP_DENIED/403 https://rbc.ru "-"
cache.log:
2018/05/17 12:35:18.484 kid1| 28,3| Checklist.cc(70) preCheck: 0x38457c8 checking slow rules
2018/05/17 12:35:18.484 kid1| 28,5| Acl.cc(138) matches: checking http_access
2018/05/17 12:35:18.484 kid1| 28,5| Checklist.cc(400) bannedAction: Action 'DENIED/0' is not banned
2018/05/17 12:35:18.484 kid1| 28,5| Acl.cc(138) matches: checking http_access#1
2018/05/17 12:35:18.484 kid1| 28,5| Acl.cc(138) matches: checking !auth
2018/05/17 12:35:18.484 kid1| 28,5| Acl.cc(138) matches: checking auth
2018/05/17 12:35:18.484 kid1| 28,5| Acl.cc(36) AuthenticateAcl: SslBumped request: It is an encapsulated request do not authenticate
2018/05/17 12:35:18.484 kid1| 28,3| Acl.cc(158) matches: checked: auth = 1
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: !auth = 0
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: http_access#1 = 0
2018/05/17 12:35:18.485 kid1| 28,5| Checklist.cc(400) bannedAction: Action 'ALLOWED/0' is not banned
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(138) matches: checking http_access#2
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(138) matches: checking basicauth
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(36) AuthenticateAcl: SslBumped request: It is an encapsulated request do not authenticate
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: basicauth = 1
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(138) matches: checking sites1
2018/05/17 12:35:18.485 kid1| 28,3| DomainData.cc(108) match: aclMatchDomainList: checking 'www.rbc.ru'
2018/05/17 12:35:18.485 kid1| 28,3| DomainData.cc(113) match: aclMatchDomainList: 'www.rbc.ru' NOT found
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: sites1 = 0
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: http_access#2 = 0
2018/05/17 12:35:18.485 kid1| 28,5| Checklist.cc(400) bannedAction: Action 'DENIED/0' is not banned
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(138) matches: checking http_access#3
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(138) matches: checking basicauth
2018/05/17 12:35:18.485 kid1| 28,5| Acl.cc(36) AuthenticateAcl: SslBumped request: It is an encapsulated request do not authenticate
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: basicauth = 1
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: http_access#3 = 1
2018/05/17 12:35:18.485 kid1| 28,3| Acl.cc(158) matches: checked: http_access = 1
2018/05/17 12:35:18.485 kid1| 28,3| Checklist.cc(63) markFinished: 0x38457c8 answer DENIED for match
2018/05/17 12:35:18.485 kid1| 28,3| Checklist.cc(163) checkCallback: ACLChecklist::checkCallback: 0x38457c8 answer=DENIED
2018/05/17 12:35:18.485 kid1| 28,5| Gadgets.cc(83) aclIsProxyAuth: aclIsProxyAuth: called for basicauth
2018/05/17 12:35:18.485 kid1| 28,9| Acl.cc(99) FindByName: ACL::FindByName 'basicauth'
2018/05/17 12:35:18.485 kid1| 28,5| Gadgets.cc(88) aclIsProxyAuth: aclIsProxyAuth: returning 1
2018/05/17 12:35:18.485 kid1| 28,8| Gadgets.cc(51) aclGetDenyInfoPage: got called for basicauth
2018/05/17 12:35:18.485 kid1| 28,8| Gadgets.cc(70) aclGetDenyInfoPage: aclGetDenyInfoPage: no match
As you can see I hit in acl basicauth = 1 and http_access#3 = 1 despite the fact that I have a different login. If I disable SSLbump - all works fine.
Any ideas?
Thanks
You can do something like this.
For Basic Authentication:
auth_param basic program /usr/bin/python /home/test/auth.py
auth_param basic realm Please enter username and password
auth_param basic children 100
auth_param basic credentialsttl 1 second
acl AuthUsers proxy_auth REQUIRED
In the above, test.py represents you can write your custom python script for the authentication or you can also use any other method.
and for ssl-bump you can do:
http_port 3128 ssl-bump cert=/etc/squid/ssl/myca.pem generate-host-
certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/spool/squid_ssldb -M 4MB
sslcrtd_children 5
acl tcp_level at_step SslBump1
ssl_bump peek tcp_level all
ssl_bump bump all
and for allowing the access
http_access allow AuthUsers
http_access deny all
i can connect to my freePbx server with jssip.
but registeration got faild with wrong password in Asterisk Logs.
i can connect and register with none WebRtc and WebSocket clients with same password for my PjSip Extension.
it's work in .net library Ozeki with the same user, pass and etc but in nodejs and webRtc it's not working.
the Asterisk Log:
ERROR[24788]: res_http_websocket.c:506 ws_safe_read: Error readng from webScoket: Connection rest by peer.
NOTICE[33279]: chan_sip.c28486 handle_request_register: Registeration from '<sip:170#192.168.0.210>' faild for 192.168.0.250:43751 - wrong password
jssip conf:
var socket = new JsSIP.WebSocketInterface('ws://192.168.0.210:8089/ws');
var configuration = {
sockets : [ socket ],
authorization_user: '170',
uri : 'sip:170#192.168.0.210',
ws_servers : 'ws://192.168.0.210:8089/ws',
password : '856589',
realm : '192.168.0.210',
display_name : '170',
contact_uri : 'sip:170#192.168.0.210'
};
jssip log:
olden#golden-client:~/LocalFiles/projects/jssip$ node ./index.js JsSIP version 3.0.21 +0ms jssip-node-websocket new() [url:"ws://192.168.0.210:8088/ws", options:undefined] +0ms
JsSIP:UA new() [configuration:{ sockets: [ NodeWebSocket { _url: 'ws://192.168.0.210:8088/ws', _options: {}, _sipUri: 'sip:192.168.0.210:8088;transport=ws', _viaTransport: 'WS', _ws: null } ], authorization_user: '170', uri: 'sip:170#192.168.0.210:5060', ws_servers: 'ws://192.168.0.210:8088/ws', password: '170170', realm: '192.168.0.210', display_name: '170', contact_uri: 'sip...#192.168.0.210' }] +0ms
JsSIP:Transport new() +0ms
JsSIP:UA configuration parameters after validation: +15ms
JsSIP:UA - authorization_user: "170" +0ms
JsSIP:UA - password: NOT SHOWN +0ms
JsSIP:UA - realm: "192.168.0.210" +0ms
JsSIP:UA - ha1: NOT SHOWN +0ms
JsSIP:UA - display_name: "170" +0ms
JsSIP:UA - uri: sip:170#192.168.0.210:5060 +0ms
JsSIP:UA - contact_uri: {"_parameters":{},"_headers":{},"_scheme":"sip","_user":"170","_host":"192.168.0.210"} +0ms
JsSIP:UA - instance_id: "f3a58a94-b426-4a8d-8b15-9f7208a42f9b" +0ms
JsSIP:UA - use_preloaded_route: false +0ms
JsSIP:UA - session_timers: true +0ms
JsSIP:UA - no_answer_timeout: 60000 +0ms
JsSIP:UA - register: true +0ms
JsSIP:UA - register_expires: 600 +0ms
JsSIP:UA - registrar_server: sip:192.168.0.210:5060 +0ms
JsSIP:UA - connection_recovery_max_interval: null +0ms
JsSIP:UA - connection_recovery_min_interval: null +0ms
JsSIP:UA - via_host: "192.168.0.210" +0ms
JsSIP:UA start() +0ms
JsSIP:Transport connect() +1ms
connecting
jssip-node-websocket connect() +18ms
jssip-node-websocket WebSocket connecting [url:"ws://192.168.0.210:8088/ws"] +0ms
jssip-node-websocket WebSocket open [url:"ws://192.168.0.210:8088/ws"] +18ms
connected
JsSIP:Transport send() +22ms
JsSIP:Transport sending message:
JsSIP:Transport
JsSIP:Transport REGISTER sip:192.168.0.210:5060 SIP/2.0
JsSIP:Transport Via: SIP/2.0/WS 192.168.0.210;branch=z9hG4bK160956
JsSIP:Transport Max-Forwards: 69
JsSIP:Transport To: <sip:170#192.168.0.210:5060>
JsSIP:Transport From: "170" <sip:170#192.168.0.210:5060>;tag=s9bkng5emf
JsSIP:Transport Call-ID: 6ra4f68tegdmqllfoa7kja
JsSIP:Transport CSeq: 1 REGISTER
JsSIP:Transport Contact: <sip...#192.168.0.210>;+sip.ice;reg-id=1;+sip.instance="<urn:uuid:f3a58a94-b426-4a8d-8b15-9f7208a42f9b>";expires=600
JsSIP:Transport Expires: 600
JsSIP:Transport Allow: INVITE,ACK,CANCEL,BYE,UPDATE,MESSAGE,OPTIONS,REFER,INFO
JsSIP:Transport Supported: path,gruu,outbound
JsSIP:Transport User-Agent: JsSIP 3.0.21
JsSIP:Transport Content-Length: 0
JsSIP:Transport
JsSIP:Transport
JsSIP:Transport +0ms
jssip-node-websocket send() +3ms
jssip-node-websocket WebSocket message received +3ms
JsSIP:Transport received text message:
JsSIP:Transport
JsSIP:Transport SIP/2.0 401 Unauthorized
JsSIP:Transport Via: SIP/2.0/WS 192.168.0.210;branch=z9hG4bK160956;received=192.168.0.3;rport=35832
JsSIP:Transport From: "170" <sip:170#192.168.0.210:5060>;tag=s9bkng5emf
JsSIP:Transport To: <sip:170#192.168.0.210:5060>;tag=as0f727a9b
JsSIP:Transport Call-ID: 6ra4f68tegdmqllfoa7kja
JsSIP:Transport CSeq: 1 REGISTER
JsSIP:Transport Server: FPBX-13.0.192.19(13.12.1)
JsSIP:Transport Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
JsSIP:Transport Supported: replaces, timer
JsSIP:Transport WWW-Authenticate: Digest algorithm=MD5, realm="asterisk", nonce="078cc3e4"
JsSIP:Transport Content-Length: 0
JsSIP:Transport
JsSIP:Transport
JsSIP:Transport +3ms
JsSIP:DigestAuthentication authenticate() | response generated +0ms
JsSIP:Transport send() +10ms
JsSIP:Transport sending message:
JsSIP:Transport
JsSIP:Transport REGISTER sip:192.168.0.210:5060 SIP/2.0
JsSIP:Transport Via: SIP/2.0/WS 192.168.0.210;branch=z9hG4bK4224730
JsSIP:Transport Max-Forwards: 69
JsSIP:Transport To: <sip:170#192.168.0.210:5060>
JsSIP:Transport From: "170" <sip:170#192.168.0.210:5060>;tag=s9bkng5emf
JsSIP:Transport Call-ID: 6ra4f68tegdmqllfoa7kja
JsSIP:Transport CSeq: 2 REGISTER
JsSIP:Transport Authorization: Digest algorithm=MD5, username="170", realm="asterisk", nonce="078cc3e4", uri="sip:192.168.0.210:5060", response="26576b92bf68c3d315d480b02d53783c"
JsSIP:Transport Contact: <sip...#192.168.0.210>;+sip.ice;reg-id=1;+sip.instance="<urn:uuid:f3a58a94-b426-4a8d-8b15-9f7208a42f9b>";expires=600
JsSIP:Transport Expires: 600
JsSIP:Transport Allow: INVITE,ACK,CANCEL,BYE,UPDATE,MESSAGE,OPTIONS,REFER,INFO
JsSIP:Transport Supported: path,gruu,outbound
JsSIP:Transport User-Agent: JsSIP 3.0.21
JsSIP:Transport Content-Length: 0
JsSIP:Transport
JsSIP:Transport
JsSIP:Transport +0ms
jssip-node-websocket send() +11ms
jssip-node-websocket WebSocket message received +1ms
JsSIP:Transport received text message:
JsSIP:Transport
JsSIP:Transport SIP/2.0 403 Forbidden
JsSIP:Transport Via: SIP/2.0/WS 192.168.0.210;branch=z9hG4bK4224730;received=192.168.0.3;rport=35832
JsSIP:Transport From: "170" <sip:170#192.168.0.210:5060>;tag=s9bkng5emf
JsSIP:Transport To: <sip:170#192.168.0.210:5060>;tag=as0f727a9b
JsSIP:Transport Call-ID: 6ra4f68tegdmqllfoa7kja
JsSIP:Transport CSeq: 2 REGISTER
JsSIP:Transport Server: FPBX-13.0.192.19(13.12.1)
JsSIP:Transport Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE
JsSIP:Transport Supported: replaces, timer
JsSIP:Transport Content-Length: 0
JsSIP:Transport
JsSIP:Transport
JsSIP:Transport +3ms
{ response:
IncomingResponse {
data: 'SIP/2.0 403 Forbidden\r\nVia: SIP/2.0/WS 192.168.0.210;branch=z9hG4bK4224730;received=192.168.0.3;rport=35832\r\nFrom: "170" <sip:170#192.168.0.210:5060>;tag=s9bkng5emf\r\nTo: <sip:170#192.168.0.210:5060>;tag=as0f727a9b\r\nCall-ID: 6ra4f68tegdmqllfoa7kja\r\nCSeq: 2 REGISTER\r\nServer: FPBX-13.0.192.19(13.12.1)\r\nAllow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH, MESSAGE\r\nSupported:replaces, timer\r\nContent-Length: 0\r\n\r\n',
headers:
{ Via: [Array],
From: [Array],
To: [Array],
'Call-ID': [Array],
CSeq: [Array],
Server: [Array],
Allow: [Array],
Supported: [Array],
'Content-Length': [Array] },
method: 'REGISTER',
via:
{ protocol: 'SIP',
transport: 'WS',
host_type: 'IPv4',
host: '192.168.0.210',
branch: 'z9hG4bK4224730',
received: '192.168.0.3' },
via_branch: 'z9hG4bK4224730',
call_id: '6ra4f68tegdmqllfoa7kja',
cseq: 2,
from: NameAddrHeader { _uri: [Object], _parameters: [Object], _display_name: '170' },
from_tag: 's9bkng5emf',
to:
NameAddrHeader {
_uri: [Object],
_parameters: [Object],
_display_name: undefined },
to_tag: 'as0f727a9b',
body: '',
sdp: null,
status_code: 403,
reason_phrase: 'Forbidden' },
cause: 'Rejected' }
The REGISTRATION process follows, as would be expected, the procedure specified in RFC3261, that's to say, a first REGISTER is answered by a 401 error response that causes a second REGISTER with authorization information. In this case, the server rejects those data.
170 (REGISTER) ---------------------> Asterisk
170 <----------(401 - Not authorized) Asterisk
170 (REGISTER with Auth info) ------> Asterisk
170 <----------------(407 - Rejected) Asterisk
So, asterisk is rejecting the register because it does not like the second REGISTER message.
Authorization: Digest algorithm=MD5, username="170", realm="asterisk", nonce="078cc3e4", uri="sip:192.168.0.210:5060", response="26576b92bf68c3d315d480b02d53783c"
Well, realm "asterisk" is not the same configured in jssip, and it does not follow what it is specified in the RFC:
Operators of user agents or proxy servers that will authenticate received requests MUST adhere to the following guidelines for creation of a realm string for their server:
Realm strings MUST be globally unique. It is RECOMMENDED that a realm string contain a hostname or domain name, following the recommendation in Section 3.2.1 of RFC 2617 [17].
Realm strings SHOULD present a human-readable identifier that can be rendered to a user.
So, you should configure sip.conf to define the realm string in asterisk to follow the above rules. Anyway, note that even if jssip was configured with a different "realm", in the second REGISTER it is used the field coming in the first response. So, I do not think it is the problem.
Other thing is uri="sip:192.168.0.210:5060" in the second REGISTER that does not include the user name. If it followed the next example, taken from the "sip:170:192.168.0.210:5060"
Authorization: Digest username="bob",..., uri="sip:bob#biloxi.com",..."
It is not clear to me why uri does not contain the username as it is specified in jssip.conf. I would remove "authorization_user" from the configuration, as it is an optional parameter and test.
Disable chan_sip (Settings > Advanced Settings - Dialplan and Operational - SIP Channel Driver=pjsip). And then it worked!
I am hitting an API to get a cookie for authenticating subsequent requests. I need to get the cookie from this collection of response headers:
{
"Server": "Cowboy",
"Connection": "close",
"Set-Cookie": "heroku-session-affinity=ADaDaANoA24IAXtpQOD///8HYgAAKehiAA6V12ECbAAAAAJtAAAABXdlYi4ybQAAAAV3ZWIuMWrNqkap0u4H0uG3Btrtlamaq2nQ5w__; Version=1; Expires=Wed, 11-Jan-2017 01:12:09 GMT; Max-Age=86400; Domain=xxxxxxxx.xxxx.xx; Path=/; HttpOnly",
"X-Powered-By": "Express",
"Vary": "Origin",
"Access-Control-Allow-Credentials": "true",
"Content-Type": "application/json; charset=utf-8",
"Content-Length": "30914",
"Etag": "W/\"78c2-veX2kgrO3zh118nYsOP80A\"",
"Set-Cookie": "connect.sid=s%3ACX3tfhPk9nlRtU8e8gtouQIx1kpP07h9.iExm%2F96Dgzuh289nmjwbYO49E0Bq0WwUHmN539IkudI; Path=/; Expires=Thu, 12 Jan 2017 01:12:09 GMT; HttpOnly",
"Date": "Tue, 10 Jan 2017 01:12:09 GMT",
"Via": "1.1 vegur"
}
I need to get the following cookie:
"connect.sid=s%3ACX3tfhPk9nlRtU8e8gtouQIx1kpP07h9.iExm%2F96Dgzuh289nmjwbYO49E0Bq0WwUHmN539IkudI; Path=/; Expires=Thu, 12 Jan 2017 01:12:09 GMT; HttpOnly"
I want to parse the headers and return that cookie, but the following code only displays the cookie that I don't want:
internal final class CookieParser {
internal static func cookie(from response: HTTPURLResponse) -> HTTPCookie? {
print("\n====> Response: \(response)\n")
guard let url = response.url else { return nil }
guard let headerFields = response.allHeaderFields as? HTTPHeaders else { return nil }
print("\n====> Headers: \(headerFields)\n")
let cookies = HTTPCookie.cookies(withResponseHeaderFields: headerFields, for: url)
print("\n====> Cookies: \(cookies)\n")
return nil
}
}
The code above produces the following output:
====> Response: <NSHTTPURLResponse: 0x60800002e160> { URL: http://xxxxxxxx.xxxx.xx/xxx/xx/xxxxx } { status code: 200, headers {
"Access-Control-Allow-Credentials" = true;
Connection = close;
"Content-Length" = 30914;
"Content-Type" = "application/json; charset=utf-8";
Date = "Tue, 10 Jan 2017 01:12:09 GMT";
Etag = "W/\"78c2-veX2kgrO3zh118nYsOP80A\"";
Server = Cowboy;
"Set-Cookie" = "heroku-session-affinity=ADaDaANoA24IAXtpQOD///8HYgAAKehiAA6V12ECbAAAAAJtAAAABXdlYi4ybQAAAAV3ZWIuMWrNqkap0u4H0uG3Btrtlamaq2nQ5w__; Version=1; Expires=Wed, 11-Jan-2017 01:12:09 GMT; Max-Age=86400; Domain=partners.flye.co; Path=/; HttpOnly";
Vary = Origin;
Via = "1.1 vegur";
"X-Powered-By" = Express;
} }
====> Headers: ["Content-Type": "application/json; charset=utf-8", "Content-Length": "30914", "Connection": "close", "Set-Cookie": "heroku-session-affinity=ADaDaANoA24IAXtpQOD///8HYgAAKehiAA6V12ECbAAAAAJtAAAABXdlYi4ybQAAAAV3ZWIuMWrNqkap0u4H0uG3Btrtlamaq2nQ5w__; Version=1; Expires=Wed, 11-Jan-2017 01:12:09 GMT; Max-Age=86400; Domain=partners.flye.co; Path=/; HttpOnly", "Vary": "Origin", "Etag": "W/\"78c2-veX2kgrO3zh118nYsOP80A\"", "Access-Control-Allow-Credentials": "true", "X-Powered-By": "Express", "Date": "Tue, 10 Jan 2017 01:12:09 GMT", "Via": "1.1 vegur", "Server": "Cowboy"]
====> Cookies: [<NSHTTPCookie version:1 name:"heroku-session-affinity" value:"ADaDaANoA24IAXtpQOD///8HYgAAKehiAA6V12ECbAAAAAJtAAAABXdlYi4ybQAAAAV3ZWIuMWrNqkap0u4H0uG3Btrtlamaq2nQ5w__" expiresDate:2017-01-12 00:41:08 +0000 created:2017-01-11 00:41:08 +0000 sessionOnly:FALSE domain:".partners.flye.co" partition:"none" path:"/" isSecure:FALSE>]
The unwanted Set-Cookie value must be replacing the wanted Set-Cookie value in the dictionary of returned headers. Is there a way that I can retrieve the wanted Set-Cookie value before it is replaced?
When I do a conditional GET and get a 304 response with a Cache-Control max-age header, I expect the freshness of the resource to be updated and future requests to get the representation from the local cache, assuming it is fresh. However, I cannot get this to work using HttpClient.
This is my reproduction, as a simple .net 4.5 Console application using the Nuget dependencies Microsoft.AspNet.WebApi.OwinSelfHost and Microsoft.Net.Http.
using System;
using System.Net;
using System.Net.Cache;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading;
using System.Web.Http;
using Microsoft.Owin.Hosting;
using Owin;
namespace TestEtag
{
class Program
{
static void Main(string[] args)
{
// Run as administrator to allow web server to start.
// Clear browser cache to see initial 200 response when running multiple times
var host = String.Format("http://{0}:8080/", Environment.MachineName);
var server = WebApp.Start(host, app =>
{
var config = new HttpConfiguration();
config.MapHttpAttributeRoutes();
app.UseWebApi(config);
});
Console.WriteLine("Started Web server");
var clientHandler = new WebRequestHandler();
clientHandler.CachePolicy = new RequestCachePolicy(RequestCacheLevel.Default);
var client = new HttpClient(clientHandler) { BaseAddress = new Uri(host) };
var response = client.GetAsync("/resource").Result; // Should return response
Console.WriteLine("returned response " + response.StatusCode);
var response2 = client.GetAsync("/resource").Result; // Should get from local cache (and does)
Console.WriteLine("returned response " + response2.StatusCode);
Console.WriteLine("Pause for resource to expire");
Thread.Sleep(7000);
var response3 = client.GetAsync("/resource").Result; // Should make conditional get and get 304 (and does)
Console.WriteLine("returned response " + response3.StatusCode);
var response4 = client.GetAsync("/resource").Result; // Should get from local cache (but doesn't)
Console.WriteLine("returned response " + response4.StatusCode);
Console.ReadLine();
server.Dispose();
}
}
[Route("resource")]
public class ResourceController : ApiController
{
public HttpResponseMessage Get()
{
var etag = new EntityTagHeaderValue("\"XYZPQR\"");
if (Request.Headers.IfNoneMatch != null)
{
if (Request.Headers.IfNoneMatch.Contains(etag))
{
var notModifiedresponse = new HttpResponseMessage(HttpStatusCode.NotModified);
notModifiedresponse.Headers.CacheControl = new CacheControlHeaderValue() { MaxAge = new TimeSpan(0, 0, 0, 5) };
notModifiedresponse.Headers.ETag = etag;
return notModifiedresponse;
}
}
var response = new HttpResponseMessage()
{
Content = new StringContent("This is cached content with an etag")
};
response.Headers.CacheControl = new CacheControlHeaderValue() { MaxAge = new TimeSpan(0, 0, 0, 5) };
response.Headers.ETag = etag;
return response;
}
}
}
I believe that the last HTTP request should come directly from the local WinInet cache, however, you can see from fiddler it is making a network request and returning a 304.
Anyone have any ideas what I am doing wrong?
Update
I tried using Expires instead of Max-age but I get the same behaviour. A subset of that trace shows the following,
System.Net.Cache Verbose: 0 : [15524] RequestCacheProtocol#2086040::GetRevalidateStatus(HttpWebRequest#10920143)
System.Net.Cache Information: 0 : [15524] StatusCode=304 NotModified System.Net.Cache Information: 0 : [15524] ValidateResponse(), result = Continue.
System.Net.Cache Information: 0 : [15524] After Response Cache Validation.
System.Net.Cache Information: 0 : [15524] Updating headers on 304 response.
System.Net.Cache Information: 0 : [15524] RevalidateCache(), result = ReturnCachedResponse.
System.Net.Cache Verbose: 0 : [15524] Exiting RequestCacheProtocol#2086040::GetRevalidateStatus() -> result = ReturnCachedResponse
System.Net.Cache Verbose: 0 : [15524] RequestCacheProtocol#2086040::GetUpdateStatus()
System.Net.Cache Information: 0 : [15524] Response Status = NotModified.
System.Net.Cache Information: 0 : [15524] Valid based on Status Code: 304.
System.Net.Cache Information: 0 : [15524] Response==304 or Request was HEAD, updating cache entry.
System.Net.Cache Information: 0 : [15524] Update Cached Headers.
As you can see, the log claims that the cache entry has been updated.
System.Net.Http Information: 0 : [15524] HttpClient#17533994 - Request for HttpRequestMessage#55917598 completed successfully. Returning response HttpResponseMessage#39055747: StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Date: Tue, 07 Jan 2014 18:12:18 GMT
ETag: "XYZPQR"
Server: Microsoft-HTTPAPI/2.0
Age: 0
Content-Length: 35
Content-Type: text/plain; charset=utf-8
Expires: Tue, 07 Jan 2014 18:12:23 GMT }
The Expires date of this response shows 18:12:23.
The last request shows that the cached response is being found,
System.Net.Cache Information: 0 : [10644] WinInetCache.Retrieve() -> Filename = C:\Users\darrel.TAVIS\AppData\Local\Microsoft\Windows\INetCache\IE\P6AEJQC4\resource[1].cache, Status = Success.
System.Net.Cache Verbose: 0 : [10644] Exiting WinInetCache.Retrieve()() Status = Success System.Net.Cache Information: 0 : [10644] ...Dumping Cache Context...
System.Net.Cache Information: 0 : [10644] IsPrivateEntry = True
System.Net.Cache Information: 0 : [10644] IsPartialEntry = False
System.Net.Cache Information: 0 : [10644] StreamSize = 35
System.Net.Cache Information: 0 : [10644] Expires = Tue, 07 Jan 2014 18:12:13 GMT
System.Net.Cache Information: 0 : [10644] LastAccessed = Tue, 07 Jan 2014 18:12:19 GMT
System.Net.Cache Information: 0 : [10644] LastModified =
System.Net.Cache Information: 0 : [10644] LastSynchronized = Tue, 07 Jan 2014 18:12:08 GMT
System.Net.Cache Information: 0 : [10644] MaxStale(sec) = 0
System.Net.Cache Information: 0 : [10644] HitCount = 4
System.Net.Cache Information: 0 : [10644] UsageCount = 2
System.Net.Cache Information: 0 : [10644] EntryMetadata:
System.Net.Cache Information: 0 : [10644] HTTP/1.1 200 OK System.Net.Cache Information: 0 : [10644] Content-Length: 35 System.Net.Cache Information: 0 : [10644] Content-Type: text/plain; charset=utf-8 System.Net.Cache Information: 0 : [10644] Date: Tue, 07 Jan 2014 18:12:08 GMT System.Net.Cache Information: 0 : [10644] Expires: Tue, 07 Jan 2014 18:12:13 GMT System.Net.Cache Information: 0 : [10644] ETag: "XYZPQR"
System.Net.Cache Information: 0 : [10644] Server: Microsoft-HTTPAPI/2.0 System.Net.Cache Information: 0 : [10644] --- System.Net.Cache Information: 0 : [10644] SystemMetadata:
System.Net.Cache Information: 0 : [10644] Entry Status Line = HTTP/1.1 200 OK.
System.Net.Cache Information: 0 : [10644] Cache Cache-Control = .
System.Net.Cache Information: 0 : [10644] [Now Time (UTC)] = Tue, 07 Jan 2014 18:12:19 GMT.
System.Net.Cache Information: 0 : [10644] [Age1] NowTime-Date Header = 11, Date Header: Tue, 07 Jan 2014 18:12:08 GMT.
System.Net.Cache Information: 0 : [10644] [MaxAge] Cache Expires - Date = 5, Expires: Tue, 07 Jan 2014 18:12:13 GMT.
System.Net.Cache Information: 0 : [10644] ValidateFreshness(), result = Stale.
System.Net.Cache Information: 0 : [10644] Cache Age = 11, Cache MaxAge = 5.
System.Net.Cache Information: 0 : [10644] Request Condition = If-None-Match:"XYZPQR".
System.Net.Cache Information: 0 : [10644] ValidateCache(), result = Continue.
However, the Expires entry of the cache is still showing as 18:12:13 and not 18:12:23 from the 304 response. The result is that the cached entry is deemed stale and we pay for the round trip back to the server to get another 304.