I'm processing M365 mailbox messages via MS Graph. I'm using .Net5 and the latest version of MSGraph SDK for .NET; (particularly the PageIterator for processing email messages) - but i'm actually experiencing the issue even via a pure call via Postman: in some cases the response is just truncated abnormally (hence the response JSON could not be parsed).
One example: ~56k messages are processed successfully, then during trying to get a next page by the iterator (for me seemengly randomly; some mailbox around 56k some at 78k, but almost always 50k+) i got a JSON parsing exception (sometimes unclosed string, sometimes unexpected char).
If i take the actual next page link from the iterator while catching the exception, i can reproduce the issue in Postman; the response is truncated.
In case i query the single message that is truncated separatelly via its id then the full message is available in the response.
An example call which fails has the response payload JSON truncated (but the call actually succeeds with HTTP 200):
https://graph.microsoft.com/v1.0/me/messages?$orderby=receivedDateTime+ASC&$select=ToRecipients,CcRecipients,Subject,From,Body,HasAttachments,ReceivedDateTime&$expand=attachments($select=name)&$top=32&$skip=57454
The end of the result json:
"#odata.etag": "someetaghere",
"id": "someiidhere",
"receivedDateTime": "2017-03-19T09:15:42Z",
"hasAttachments": false,
"subject": "Fwd: Contrat Morval",
"body": {
"contentType": "text",
"content":"Some text just an example which ends somewhere in the middle of the text
Some UPDATES for this particular case:
In Postman
if i remove the "Body" param from the $select list of the above query, it constantly fails with "503 Service Unavailable" after long (<~20sec) response times
unless if i set the $top param to 31 or lower, then everything works OK, regardless if "Body" is iuncluded in the $Select list or not
if i use $top>31 with "Body" inlcuded, the response payload is truncated always at the same position of the 24th item in the result array, regardless of the value of $top
I hoped if i use 30 as page size running my Graph SDK code then i could forget this bug :), but unfortunatelly there i receive "503 Service Unavailable" for the same query that succeeds in Postman, with message
Code: generalException
Message: Unexpected exception returned from the service.
ClientRequestId: 610103aa-ac07-4b8b-b7af-0aa7bdbcce0e
The Timestamp form the response headers: Thu, 03 Dec 2020 10:49:36 GMT
Any help would be appreciated, how could i ensure that the message is loaded correctly? I tought about some throttling, quota/message size limit or restriction, but i could not find anything - and now I can reproduce the issue in postman anytime.
Thanks
Related
How do I get the response object from call-with-input-request when the HTTP status is >= 300?
The docs say this about call-with-input-request:
Returns three values: The result of the call to reader (or #f if there is no message body in the response), the request-uri of the last request and the response object. If the response code is not in the 200 class, it will raise a condition of type (exn http client-error), (exn http server-error) or (exn http unexpected-server-response), depending on the response code. This includes 404 not found (which is a client-error).
This means that call-with-input-request signals a non-continuable condition, which (as far as I understand) means that the function does not return, and I cannot get access to the response object that would otherwise be returned. Therefore I don't see how I can actually get access to the response object corresponding to this request.
I still want to be able to inspect the response, even if its status is in the 30x-50x range. For example, I want to be able to print the HTTP reason string, or log it for debugging later. How can I achieve this?
If you trigger the exception from the REPL, you can inspect it with the ,exn comma-command. Then you'll notice the condition has a response property which is a contains the status code, headers etc.
The docs could be improved in this regard I'm sure. Perhaps you have a suggestion where to put this? The problem is that the exact contents of the condition object depend on where the condition was thrown, so not all properties will always be available.
I'm trying to use delta query to get teams channel messages updates according this documentation: HERE
This is the request url:
https://graph.microsoft.com/beta/teams/<teamId>/channels/<channelId>/messages/delta
However, calling the returned nextLinks one after another never returns a deltaLink. There're too many pages of results and it causes my app to be throttled before ever getting a deltaLink from it.
In other delta query endpoints, $top is supported to limit the number of results returned. Usually I'm able to get a deltaLink after calling the nextLinks once or twice. But $top doesn't seem to have an effect in the channel messages endpoint.
So I tried appending another queryString ?odata.maxpagesize=10 to the request instead, and it seemed to work a week ago. I was able to get the deltaLink after 2 pages. But it looks like Microsoft might have changed the API and this workaround no longer works.
I also tried adding Prefer: odata.maxpagesize=10 in my request header according to this documentation: HERE
But the nextLink this generates is too long and it gives me this error instead:
HTTP Error 414. The request URL is too long.
Has anyone been able to use this delta for channel messages? Or have I done something wrong?
I have quite a few REST end-points that process a request and simply return 200. I noticed it is an error to map the result with json(). If I try not to do any sort of mapping whatsoever, I see a browser warning that it could not parse XML. As returning nothing is pretty common, I am curious how I should be handling the response.
Here is a basic code example:
await this.http.put("/api/some_url", data).toPromise();
Here is the warning that shows up in Firefox:
XML Parsing Error: no root element found Location: http://localhost/api/some_url Line Number 1, Column 1:
If I try to map with json(), it blows up entirely. If I do nothing or try to map to text(), I just get the warning.
In all cases, the response headers contain Content-Length: 0, so I am surprised Angular2/Firefox is trying to parse anything. I don't know if this warning is coming from Angular2 or Firefox. I have tried this in Chrome and IE and there is no warning. So I am wondering if it is specific to Firefox.
When no content is returned, e.g. return Ok() (HTTP status code 200) in ASP.NET Core, content-type is (naturally) not specified, but Firefox assumes the response to be XML and tries to parse an empty document, which results in the following console error:
XML Parsing Error: no root element found Location ...
Specifying return NoContent(); (HTTP status code 204) makes Firefox happy (and also works for other browsers).
The issue is reported at Bugzilla#Mozilla:
and a parsing failure will simply be logged to the web console instead.
I'm doing performance testing on our FCM notification server with fake device tokens and should be getting an error like the following:
{
"multicast_id":6782339717028231855,
"success":0,
"failure":1,
"canonical_ids":0,
"results":[{"error":"InvalidRegistration"}
]
}
But about 3-5 requests out of 600K requests, we see a 400 error with a response body like this:
to
We know the json is formed correctly since we log the json body if we get any sort of error.
Has anyone seen any strange error/response body similar to this?
Error 400 means Invalid Json request and the response body with only to means the problem is an invalid registration_ids or to field.
In my case I was emitting a Downstream message to every user, including the ones that haven't yet registered with FCM, so some of those had an empty recipient value.
I am trying to make a get request to retrieve a string
When I use
retrieve : Task.Task Http.Error String
retrieve = getString "http://api.endpoint.com"
everything works fine.
On the other hand if I use
retrieve : Task.Task Http.Error String
retrieve = get Json.Decode.string "http://api.endpoint.com"
the http request gets done, but the chained tasks are not executed.
My question is: what is the difference between the two approaches above? Am I doing something wrong with the second one? How to debug it?
getString returns the response of the get request as a String. get take a JSON decoder and runs that over the response of the get request. So if you provide Json.Decode.string, it will expect the response to be have a Json encoded string in it. So it expects extra double quotes in the response.
If your http request fails the best way to debug is to look at what kind of error you get. In this case you'll probably get an UnexpectedPayload because the request succeeds, but the decoder fails.