A SyncEvolution user has reported problems with mirroring a local calendar in Google Calendar: when refreshing Google Calendar via CalDAV (= DELETE all events and recreate with PUT), some PUT commands fail with "404 Not Found".
It looks like this happens for VEVENTs which have an ORGANIZER (i.e., meetings). It works for simpler VEVENTs that have no ORGANIZER. Example of the failure below.
I believe this is a known limitation of Google CalDAV. The question is: what can be done to support this use case, either in the client or the server?
I can't think of any workaround in the client. The 404 error is too unspecific to trigger a workaround, and even if the client could detect the root cause for it, there's no other way of recreating the event than the PUT, which is failing.
PUT /caldav/v2/patrick.ohly.estamos%40googlemail.com/events/4fbbdbd8-c9de-4ce0-810a-01e3a438a35d-pohly-xyz.ics HTTP/1.1
Connection: TE
TE: trailers
Host: apidata.googleusercontent.com
Content-Length: 928
Content-Type: text/calendar; charset=utf-8
User-Agent: SyncEvolution
Authorization: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Sending request-line and headers:
Sending request body:
Body block (928 bytes):
[BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Synthesis AG//NONSGML SyncML Engine V3.4.0.47//EN
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
DTSTART:20071104T020000
RRULE:FREQ=MONTHLY;INTERVAL=12;BYDAY=1SU
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:HNP
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20070311T020000
RRULE:FREQ=MONTHLY;INTERVAL=12;BYDAY=2SU
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:HAP
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
STATUS:CONFIRMED
LAST-MODIFIED:20120813T184814Z
DTSTAMP:20120813T184814Z
UID:4fbbdbd8-c9de-4ce0-810a-01e3a438a35d-pohly-xyz
SEQUENCE:0
CLASS:PUBLIC
TRANSP:OPAQUE
SUMMARY:test event
DTSTART;TZID=America/Los_Angeles:20120815T103000
DTEND;TZID=America/Los_Angeles:20120815T113000
ATTENDEE;CN=John Doe;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;RSVP=TRUE:
mailto:john.doe#example.com
ORGANIZER;CN=Joan Doe:mailto:joan.doe#example.com
END:VEVENT
END:VCALENDAR
]
Request sent; retry is 1.
[status-line] < HTTP/1.1 201 Created
...
DELETE /caldav/v2/patrick.ohly.estamos%40googlemail.com/events/4fbbdbd8-c9de-4ce0-810a-01e3a438a35d-pohly-xyz.ics HTTP/1.1
Connection: TE
TE: trailers
Host: apidata.googleusercontent.com
Content-Length: 0
User-Agent: SyncEvolution
Authorization: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Sending request-line and headers:
Request sent; retry is 1.
[status-line] < HTTP/1.1 204 No Content
...
PUT /caldav/v2/patrick.ohly.estamos%40googlemail.com/events/4fbbdbd8-c9de-4ce0-810a-01e3a438a35d-pohly-xyz.ics HTTP/1.1
Connection: TE
TE: trailers
Host: apidata.googleusercontent.com
Content-Length: 928
Content-Type: text/calendar; charset=utf-8
User-Agent: SyncEvolution
Authorization: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Sending request-line and headers:
Sending request body:
Body block (928 bytes):
[BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Synthesis AG//NONSGML SyncML Engine V3.4.0.47//EN
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
DTSTART:20071104T020000
RRULE:FREQ=MONTHLY;INTERVAL=12;BYDAY=1SU
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:HNP
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20070311T020000
RRULE:FREQ=MONTHLY;INTERVAL=12;BYDAY=2SU
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:HAP
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
STATUS:CONFIRMED
LAST-MODIFIED:20120813T184814Z
DTSTAMP:20120813T184814Z
UID:4fbbdbd8-c9de-4ce0-810a-01e3a438a35d-pohly-xyz
SEQUENCE:0
CLASS:PUBLIC
TRANSP:OPAQUE
SUMMARY:test event
DTSTART;TZID=America/Los_Angeles:20120815T103000
DTEND;TZID=America/Los_Angeles:20120815T113000
ATTENDEE;CN=John Doe;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;RSVP=TRUE:
mailto:john.doe#example.com
ORGANIZER;CN=Joan Doe:mailto:joan.doe#example.com
END:VEVENT
END:VCALENDAR
]
Request sent; retry is 1.
[status-line] < HTTP/1.1 404 Not Found
(I don't know about SyncEvolution, but perhaps some recent experience I have with google calendars can shed some light. I'm not sure there is a satisfactory answer.)
This can be a problem (even without a DELETE):
Suppose user U PUTs an event that has attendees but U is not the organizer. What happens is that U "steals" the event (becomes the organizer) and inserts a (probably) read-only copy of the event on everyone else's calendar. Note: for an event with ATTENDEES Google calendar makes the owner/creator/insertor the organizer. The UID is then "held" by the first ATTENDEE who mentions it. Since there can be no duplicate UIDs on google calendars, the error occurs on subsequent PUTs by other ATTENDEES.
This becomes even worse when events are deleted from one calendar (but not all) and the UID is still "held" by someone. Or the delete somehow does not actually remove the UID (which can occur on google calendars).
Mitigation strategies:
For events with an organizer, only PUT into the organizer's calendar.
Somehow unique-ify the UUID if you can not accomplish the above (with the result that the events become unconnected duplicates)
Related
I have an application that has been successfully using HTTP batch requests to insert, edit, and delete events via the Google Calendar API. In the last couple of days, the individual requests within the batches have started returning 404 errors (although the batch itself gets a 200 success response). Making those same requests as individual requests using the same authorization header is still working.
I'm pretty sure that this isn't related to the forthcoming shutdown of Google's global HTTP batch endpoints because we're using https://www.googleapis.com/batch/calendar/v3 as our endpoint.
Here's an example of what I'm trying to do:
https://www.googleapis.com/batch/calendar/v3
Authorization: Bearer your_auth_token
Content-Type: multipart/mixed; boundary=batch_google_calendar
--batch_google_calendar
Content-Type: application/http
Content-ID: <item-0-batchevent#example.com>
POST calendar/v3/calendars/your_calendar_id#group.calendar.google.com/events
Content-Type: application/json
{"summary":"batch API test","start":{"date":"2020-07-31"},"end":{"date":"2020-07-31"}}
--batch_google_calendar--
And the response is:
--batch_3J6sfuPtVQbjZLcpUe06245gKlO31YnC
Content-Type: application/http
Content-ID: <response-item-0-batchevent#example.com>
HTTP/1.1 404 Not Found
Vary: Origin
Vary: X-Origin
Vary: Referer
Content-Type: application/json; charset=UTF-8
[{
"error": {
"code": 404,
"message": "URL path: /v3/calendars/your_calendar_id#group.calendar.google.com/events could not be resolved. Maybe there is an error parsing the batch item.",
"status": "NOT_FOUND"
}
}
]
--batch_3J6sfuPtVQbjZLcpUe06245gKlO31YnC--
And here's an example of an individual request that's working:
https://www.googleapis.com/calendar/v3/calendars/your_calendar_id#group.calendar.google.com/events
Authorization: Bearer your_auth_token
Content-Type: application/json
{"summary":"API test","start":{"date":"2020-07-31"},"end":{"date":"2020-07-31"}}
Why might the individual request be succeeding but the batch request fail?
Google gave a helpful reply via their issue tracker: there was an error in the way that batch entry paths were specific in my application. This had worked without errors until last week, so I think something must have changed at their end to make it less tolerant of mistakes.
The error we had made was omitting the leading slash in the path in each batch entry. Here's what we were doing:
POST calendar/v3/calendars/your_calendar_id#group.calendar.google.com/events
And here's what we should have been doing:
POST /calendar/v3/calendars/your_calendar_id#group.calendar.google.com/events
I hope that this might be helpful to anyone else who ever finds themselves in a similar situation!
I'm trying to send AT commands to my ESP32* module and am not getting any response back.
I need to perform a POST request that contains the username and password and other requests later on. I am not structuring these correctly and there is not a lot of good documentation for this.
NOTE: because I cannot share my complete url due to privacy I will use something with the same length ********connected.com:443
Send login information to ********connected.com/login (POST) body{"email":"myemail.ca", "password":"xxxxx"}
once I get the token I will make other requests.
get information regarding user profile ********connected.com/getRoutine ( GET) query param username="bob"
I really want to understand how these requests are structured so if someone can explain it to me elegantly that would be great!
Here is what I have tried..
AT
OK
AT+CIPSTART="TCP","********connected.com",443
CONNECT
OK
AT+CIPSEND=48
> "GET ********connected.com:443/getUsersOnline"
OK
>
Recv 48 bytes
SEND OK
CLOSED
REQUESTED POST REQUEST I HAVE USED
AT+CIPSEND=177 “POST \r Host: ********connected.com\r\n Accept: application/json\r\n Content-Length: 224r\n Content-Type: application/jsonr\n { "email":"myemail.com", "password":"myPassword" } “
There are actually several parts of your system that might be the cause of the malfunctioning:
The AT commands sent (it is not clear how you check for server responses. Responses could proviede clues about what's wrong)
The server side app seems to be a custom implementation that might have bugs as well
The POST request might be malformed
Let's focus on the last one.
POST are described in RFC 7231, and though it is an obscure description without examples, it makes one thing clear: there's not actually a well defined standard... because it is strictly application dependant!
I also quote the relevant part of this brilliant answer to an old SO question:
When receiving a POST request, you should always expect a "payload", or, in HTTP terms: a message body. The message body in itself is pretty useless, as there is no standard.
For this reason, all we can do is to build a POST request as accurate as possible and then to debug the system as a whole thing making sure that the request matches what expected by the server side application.
In order to do this, let's check another external link I found: POST request examples. We found this request:
POST /test HTTP/1.1
Host: foo.example
Content-Type: application/x-www-form-urlencoded
Content-Length: 27
field1=value1&field2=value2
Now let's compare this example to your request:
POST
Host: ********connected.com
Accept: application/json
Content-Length: 224
Content-Type: application/jsonr
{ "email":"myemail.com", "password":"myPassword" }
You are saying to the server that you want to pass a resource to an unspecified application (no path), and that this resource is 224 bytes long (wrong! Message body is shorter).
For these reasons, at least these things can be improved:
POST /path/invic18app.php HTTP/1.1 //The path to the application and HTTP version are missing
Content-Length: 48 //This must be the length of the message body, without the header. I also would write it as the last option, just before message body
Write TWO empty lines before message body, otherwise the server will interpret it as further (wrong) options
I hope this helps you, even if it is a tentative answer (I cannot try this request myself). But, again, you definitely need to sniff packets a TCP levels, in order to avoid debugging the server if you are not sure that data is actually received! If you cannot install Wireshark, also tcpdump will be ok.
I am currently working on development of a caldav synchronization server layer for my calendar application. I am able to answer all the initial requested by the calendar client and currently stuck with the REPORT method.
When PROPFIND is done on Calendar, it asks for CTag and Sync-Token. I do answer this query by providing a CTag and Sync-Token (currently to mock the server, I dynamically generate these value and serve the client).
In the next query, the requested method is REPORT on the calendar as shown below:
Request from client:
REPORT URI /users/admin%40a.de/calendar/ PROTOCOL HTTP/1.1
----------------------------------------
Accept-encoding gzip, deflate
Accept */*
Connection keep-alive
Prefer return=minimal
Host **************
Brief t
User-agent Mac+OS+X/10.10.5 (14F27) CalendarAgent/316.1
Depth 1
Authorization Basic YWRtaW5AYS5kZTpwYXNz
Accept-language en-us
Content-type text/xml
Content-length 260
Request body: <?xml version="1.0" encoding="UTF-8"?>
<A:sync-collection xmlns:A="DAV:">
<A:sync-token>http://calserver.org/ns/sync-token/1</A:sync-token>
<A:sync-level>1</A:sync-level>
<A:prop>
<A:getcontenttype/>
<A:getetag/>
</A:prop>
</A:sync-collection>
Response from the Server:
Response header
Content-type: text/calendar; charset=UTF-8
Connection: keep-alive
Date Thu, 17 Dec 2015 19:35:40 GMT
Transfer-encoding chunked
Http/1.1 207 Multi-Status
Response body
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<D:multistatus xmlns:D=\"DAV:\" xmlns:C=\"urn:ietf:params:xml:ns:caldav\" xmlns:E=\"urn:ietf:params:xml:ns:carddav\">
<D:response>
<D:propstat>
<D:href>/calendar/2601ddd19c1001.ics</D:href>
<D:prop>
<D:getcontenttype>text/calendar</D:getcontenttype>
<D:getetag>"334411222s12"</D:getetag>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>
Now my question is, In the server-response, I answer the REPORT method that a new event has been created by providing a new ETag value, but the data is not requested by the Client ??
How and when should I serve the calendar data of the new event and what would be the request from client ???
Content-type should not be text/xml, It should be text/calendar.
The etag needs to be surrounded by double-quotes.
I'm pretty sure it's a bad idea to use a + in the uri the way you did. If you want to encode a space, use %20 instead, but it's probably even better to completely avoid any sort of special encoding.
The response to a sync-collection report also must have the current sync-token in the response body. See https://www.rfc-editor.org/rfc/rfc6578#section-6.4
Rather confused by the multiple edits but the content-type in the response header should definitely be text/xml
Content-type: text/xml; charset=UTF-8
By referring the below forum
Removing/Hiding/Disabling excessive HTTP response headers in Azure/IIS7 without UrlScan
I could easily hide the http headers like server information but still i'm getting below informtaion.
Is there a possibility to hide Allow and Public methods in the headers? Please share
HTTP/1.1 200 OK
**Allow: OPTIONS, TRACE, GET, HEAD, POST**
Date: Thu, 09 Jan 2014 09:37:00 GMT
**Public: OPTIONS, TRACE, GET, HEAD, POST**
Content-Length: 0
Connection: keep-alive
Like I said in my comments, you don't want to remove those headers from a response to an OPTIONS request, because that request asks the server what other methods can be called on the given resource, in this case /Main.aspx.
The response contains the allowed methods in the Allow and Public headers. If you remove those headers, the response becomes meaningless.
If you want to disable the OPTIONS request altogether, see Disable HTTP OPTIONS, TRACE, HEAD, COPY and UNLOCK methods in IIS.
Headers should be hidden anyway... Not quite sure what's happening here. Have you refreshed the main reference file.
PLEASE HELP!! Can't figure out why this simple code given by MSDN doesn't work....
I am using the following code in GetAccessToken() as given in the this MSDN article to get the access token to be used in windows notifications, but it returns "Bad Request 400"
PACKAGE_SECURITY_IDENTIFIER, CLIENT_SECRET are the values obtained when the app was registered with the Windows Store Dashboard
string urlEncodedSid = HttpUtility.UrlEncode(PACKAGE_SECURITY_IDENTIFIER);
string urlEncodedSecret = HttpUtility.UrlEncode(CLIENT_SECRET);
string body = String.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=notify.windows.com", urlEncodedSid, urlEncodedSecret);
string response;
using (WebClient client = new WebClient())
{
client.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
response = client.UploadString("https://login.live.com/accesstoken.srf", body);
}
Any help would be highly appreciated.......
I suspect the problem has to do with either an incorrect package identifier, and / or incorrect client secret.
From the MSDN page Push notification service request and response headers:
RESPONSE DESCRIPTION
--------------- --------------------------
200 OK The request was successful.
400 Bad Request The authentication failed.
Update - I ran the code from the question, using FAKE credentials.
Here is the RAW HTTP request:
POST https://login.live.com/accesstoken.srf HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: login.live.com
Content-Length: 88
Expect: 100-continue
Connection: Keep-Alive
grant_type=client_credentials&client_id=test&client_secret=test&scope=notify.windows.com
Here is the server's RAW response:
HTTP/1.1 400 Bad Request
Cache-Control: no-store
Content-Length: 66
Content-Type: application/json
Server: Microsoft-IIS/7.5
X-WLID-Error: 0x80045A78
PPServer: PPV: 30 H: BAYIDSLGN2A055 V: 0
Date: Thu, 21 Mar 2013 12:34:19 GMT
Connection: close
{"error":"invalid_client","error_description":"Invalid client id"}
You will note that the response is a 400. There is also some json that indicates the type of error. In my case, the error is Invalid client id. You probably want to take a look at your response - it will give you an indication of what happened.
I used Fiddler to debug the request/ response.
I found the reason for the error response. In fact it is the wrong PACKAGE_SECURITY_IDENTIFIER and CLIENT_SECRET.
DO NOT type the values. Because associated ASCII values differ. Therefore it is always better to copy and paste directly.
You will probably will get the access token with the simple code snippet.
Cheers
If you're using the new HttpClient API and you're sure you've copied and pasted the SID/secret values correct, you might be experiencing this issue because of encoding, provided you're using the FormUrlEncodedContent class as the content of your POST operation.
Contrary to the examples in the MSDN documentation, you don't want to URL encode the SID and secret values before adding them to the KeyValuePair collection. This is because encoding is implied by the FormUrlEncodedContent class, though I'm not seeing any documentation for this behavior. Hopefully this saves someone some time because I've been wrestling with this all night...