We have a REST API implemented as a Cloud Service, that sends telemetry to Application Insights. And we use commands like
POST /api/groups/GRP_75e0b852-ee21-45fb-b943-13aa465c62da/members.
POST /api/groups/GRP_75e0b852-ee21-45fb-b943-13aa465c62da/folders/FLD_080af364-ad37-4351-837e-4fb1d5f02e50/discussions
The sections of the command preceded by GRP_ and FLD_ are parameters.
This makes looking at the breakdown of requests in Application Insights difficult since those requests show up individually.
I’ve implemented an ITelemetryInitializer that “normalizes” the Context.Operation.Name (and the Request URL) in our requests. But I see that those request are showing up bucketed as “Other Values”.
Requests with Other Values
Is there any way to reset the "bucketing" of the top-level list, or do I need to get a new AppInsights instance?
Standard dimentions like request name should be reset after a week. So if you stopped collecting parameters in the names it should clear up after a week. Current limit is 1000.
Related
Just trying out Paw for the first time … I think this is a basic question but trying to understand how to use the various concepts:
Documents
Groups
Requests
Environments
Sessions
For this example, let's say I am wanting to work with the Sonos API, so I create a new document (⌘N) and I name it "Sonos".
Next, I create my first request which returns my households. When I created this request, I had to configure OAuth 2.0 auth. So far, so good.
For my second request, I want to return the groups in a household. How do I set this second request (and subsequent requests) so that it inherits the auth config created in the first request? Groups seems to be arbitrarily folders, environments seems to be more for specifying dev/prod etc and I am unclear what sessions are (but wondering if that's what I want).
This is a long way of asking "how do I nest requests so they share the same auth config"?
My team and I have been at this for 4 full days now, analyzing every log available to us, Azure Application Insights, you name it, we've analyzed it. And we can not get down to the cause of this issue.
We have a customer who is integrated with our API to make search calls and they are complaining of intermittent but continual 502.3 Bad Gateway errors.
Here is the flow of our architecture:
All resources are in Azure. The endpoint our customers call is a .NET Framework 4.7 Web App Service in Azure that acts as the stateless handler for all the API calls and responses.
This API app sends the calls to an Azure Service Fabric Cluster - that cluster load balances on the way in and distributes the API calls to our Search Service Application. The Search Service Application then generates and ElasticSearch query from the API call, and sends that query to our ElasticSearch cluster.
ElasticSearch then sends the results back to Service Fabric, and the process reverses from there until the results are sent back to the customer from the API endpoint.
What may separate our process from a typical API is that our response payload can be relatively large, based on the search. On average these last several days, the payload of a single response can be anywhere from 6MB to 12MB. Our searches simply return a lot of data from ElasticSearch. In any case, a normal search is typically executed and returned in 15 seconds or less. As of right now, we have already increased our timeout window to 5 minutes just to try to handle what is happening and reduce timeout errors for the fact their searches are taking so long. However, we increased the timeout via the following code in Startup.cs:
services.AddSingleton<HttpClient>(s => {
return new HttpClient() { Timeout = TimeSpan.FromSeconds(300) };
});
I've read in some places that you actually have to do this in the web.config file as opposed to here, or at least in addition to it. Not sure if this is true?
So The customer who is getting the 502.3 errors have significantly increased the volumes they are sending us over the last week, but we believe we are fully scaled to be able to handle it. They are still trying to put the issue on us, but after many days of research, I'm starting to wonder if the problem is actually on their side. Could it be possible that they are not equipped to take the increased payload on their side. Can it be that their integration architecture is not scaled enough to take the return payload from the increased volumes? When we observe our resources usages (CPU/RAM/IO) on all of the above applications, they are all normal - all below 50%. This also makes me wonder if this is on their side.
I know it's a bit of a subjective question, but I'm hoping for some insight from someone who may have experienced this before, but even more importantly, from someone who has experience with a .Net API app in Azure which return large datasets in it's responses.
Any code blocks of our API app, or screenshots from Application Insights are available to post upon request - just not sure what exactly anyone would want to see yet as I type this.
While using google cloudVision services we have a need to track individual requests made and the response. Is there a way I can generate an ID as a memo field to every request I make to the CloudVision API & read it in the usage logs/ billing statement?
I can get close to this with timestamps- but with concurrent usages and latencies, mapping can get challenging. Along with reconciliation, this will also help track error codes and reasons
I am wondering whether you can log every request which has error yourself?
You may sending request in parallel in multiple thread, but in each thread you can just write the request and its result to a GCS bucket to record the log.
I have an application where requests to a controller will take a while to process. The controller starts a thread per request and eventually returns some data to a database. I need to limit how many requests can be processed. So let's say our limit is 100, if the controller is already processing 100 requests, the 101st request will return 503 status till at least one request is completed.
I could use an application wide static counter to keep count of current processes but is there a better way to do this ?
EDIT:
The reason why the controller takes a while to respond is because the controller calls another API, which is a large database spanning several TB of geostationary data. Even if I could optimize this in theory, its not something I have control over. To make matters worse, the third party API simply times out if I have more than 10 concurrent requests. I am already dropping incoming requests to a servicebus queue. I just need a good way on my api controller to keep a global count of how many requests are coming in and returning 503 whenever it exceeds a set number of requests.
The requests to the API controller should not be limited. An idea would be to take requests and store the list of processes that need completing (database, queue etc)
Then create something outside the web request that processes this work, this is where you can manage how many are processed at once using parallel processing/multi-threading etc. (using windows service /Worker Role / Hangfire etc)
Once processed, you could communicate back to the page via SignalR to then get the data required to display once processed, or show status.
The benefit of this is that you can always go back to the page or refresh and get some kind of status, without re-running the whole process.
I am using JBoss JMX Console to monitor my Web Application.
How can i find what http requests are being processed at any point in time?
For eg: I see 25 busy theads - i want to know what http requests are these threads processing.
I am not really sure if there is an ability to map a specific request to a thread but you can certainly see what HTTP requests are made to Tomcat using AccessLogValve. You can probably use the timestamps to map those requests if need be.
Jasper;
The arduous way to do this is to examine every instance of the MBeans that have this pattern:
jboss.web:name=HttpRequest1,type=RequestProcessor,worker=http-0.0.0.0-18080
They are MBeans that represent the web request serving threads and they have an attribute called currentQueryString which is the query string of the request currently being processed. There are also attributes for currentUri and method. You could script the collection of this data as well.
An easier way, which is enabled in JBoss servers by default, is to use the available at:
http://localhost:8080/web-console/status
It handily aggregates those same MBean's and reports them in one page.
There is also options for a fuller report
http://localhost:8080/web-console/status?full=true
and an XML formatted output
http://localhost:8080/web-console/status?XML=true