Can we insert checkpoints in the scripts. For example In load tests. we get results for each URL. If site is using same URL for many different pages, that pages lumped together into single URL in the result. In QALoad, it was possible to create "checkpoints" in the script, and that is how we were able to see the performance for each page, regardless of URL. Can we achieve this by VSTS ? Please help.
There are at least two possible mechanisms.
On each request you can set a "reporting name". This can be set and updated via the properties panel of the request; but that can be tedious to use. The "Set request details" panel can be opened with one of the web test icons and that allows reporting names (and also think times and response time goals) to be set for all the requests in the test. The reporting names are used in the various reports from Visual Studio. Some people recommend that reporting names be given to all requests and that they include sequence numbers that make it clear how the various requests relate to the flow through the test.
Another option is to put requests into "transactions". However, transactions are more useful for grouping small numbers of requests that are to be reported as a group as well as individually.
Related
We built a React webapp that makes fetch calls to WebAPI2 services hosted on the same website. I've added Application Insights to the application and the server code. There are some external web requests that run in the WebAPI services and I wanted to track the timings of those calls and compare them to the overall request duration.
I can see Fetches getting populated in the requests data. I also see customEvents being recorded. The problem is that I can't seem to correlate these two datasets. None of the calls in the requests have an operation_Id that match the operation_Id or operation_ParentId in the customEvents. I had thought that the whole purpose of these properties was to associate the calls with each other.
I saw this article that talks about some new W3C distributed tracing that can be used for correlation (https://learn.microsoft.com/en-us/azure/azure-monitor/app/correlation). I think that's for a different issue of dealing with server farms but even so, I've tried enabling those parameters without any luck either.
I also enabled the enableCorsCorrelation on the javascript config without that affecting anything I could tell. But I think that setting is only useful to correlate across different AI resources.
I am using a current version of AI's SDK. I notice that the source of these entries are coming from the different parts of the SDK. Our customEvents are written by dotnet:2.8.1-22898. The Fetch requests are written by web:2.8.1-19196.
Could the issue be sampling? I've tried to open the firehouse... I have 100% Data Sampling on the Dashboard. I have left the defaults for javascript config and applicationInsights.config on the server.
Has anyone had success correlating data in a customEvents dataset with other datasets?
I have finally given up on this and I'm looking for some help on this. Here is what I have found so far.
First of all, web performance tests and/or load tests in visual studio do NOT use the browser (during playback it's not used, but it is used during the recording of the test) which is when/where the ASPSessionId is stored in a cookie or form post parameter.
I have web performance tests that have extraction rules to get the ASPSessionID from the server which I try to set in a later request as header/form post parameter, however this doesn't seem to help and it appears that I am just using the same one over and over causing the server to respond differently (presents different pages)
On the system I am testing a user will go to the site and fill out an application. If the user is in the same session the user can fill out multiple subsequent applications and have the ability to re-use some data. If that is true, the user is presented a page to select the re-usable data. If the session is new the user does not get to do this.
If I play the web test over and over manually, it works as expected (new session ID, no re-use data page presented). However, if I play that same test over and over in a load test, the first time it will pass and each time after it fails, because the session is kept open and then the server provides different pages than the ones that exist in my web performance test. The failures on the subsequent applications includes fails like (expected response URL, extraction rules...etc)
So I was using an extraction rule to get the ASPSessionID from the server and store it in a cookie and/or web form post parameters and then set it, but it is not working.
What can I do in the web performance test to successfully close the ASPSessionID so that the test runs like it is running for the first time in the load test?
In the LoadTest Test Mix, set the "Percentage of New Users" to 100. That completely solved it for me.
Omniture/SiteCatalyst's code is integrated onto the webpage to collect the analytics in our firm.
Current process: SiteCatalyst id deployed by pasting HTML code onto each page of the website. This HTML code contains variables and other identifiers that facilitate the data collection process. These variables may be dynamically populated with server or application variables. The code snippet also calls the JavaScript library file, which contains SiteCatalyst-specific JavaScript functions used during metrics collection.
We use Add-on's like Charlie, HTTP Post, DigitalPulse Debugger to Test if the code inserted has accurate values corresponding to it. This process is time consuming and tedious.
How to Automate this process? Any help would be appreciated!
Example 1:
Click here to send a page view
s.pageName="New Page"
s.prop1="some value"
void(s.t());
Example 2:
s=s_gi('myreportsuiteid');
s.linkTrackVars="prop1,eVar1,events"; s.linkTrackEvents="event1";
s.prop1="some value"; s.eVar1="another value"; s.events="event1";
s.tl(this,'o','My Link Name');
There are a few different ways to automate testing. I've been looking into it lately myself. So far I'm looking into Selenium, Zombiejs and Phantomjs. You can search for "headless testing" which basically let's run code as a browser and test conditions on the page you visit.
Here's a good place to start https://github.com/ariya/phantomjs/wiki/Headless-Testing
Using these platforms, you could easily set pages to automatically validate if the SiteCatalyst code is firing, page names are correct, click events happen etc.
Selenium is an enterprise product whereas the JS frameworks would be more of a development effort.
we usually do this using a more customizable proxy application called Fiddler which we use to capture all the traffic sent from our brower.
Fiddler has an internal scripting language that let you make any type of check on the data passing in the Adobe Analytics call and highlight in the interface any bad call.
What are some good automated tools for load testing (stress testing) web applications, that do not use record and replay of HTTP network packets?
I am aware that there are numerous load testing tools on the market that record and replay HTTP network packets. But these are unsuitable for my purpose, because of this:
The HTTP packet format changes very often in our application (e.g. when
we optimize an AJAX call). We do not want to adapt all test scripts just because
there is a slight change in HTTP packet format.
Our test team shall not need to know any internals about our application
to write their test scripts. A tool that replays HTTP packets, however, requires
the team to know the format of HTTP requests and responses, such that they
can adapt details of the replayed HTTP packets (e.g. user name).
The automated load testing tool I am looking for should be able to let the test team write "black box" test scripts such as:
Invoke web page at URL http://... .
First, enter XXX into text field XXX.
Then, press button XXX.
Wait until response has been received from web server.
Verify that text field XXX now contains the text XXX.
The tool should be able to simulate up to several 1000 users, and it should be compatible with web applications using ASP.NET and AJAX.
JMeter I've found to be pretty helpful, it also has a recording functionality to record use cases so you don't have to specify each GET/POST manually but rather "click" the use case once and then let JMeter repeat it.
http://jmeter.apache.org/
A license can be expensive for it (if you dont have MSDN), but Visual Studio 2010 Ultimate edition has a great set of load and stress testing tools that do what you describe. You can try it out for free for 90 days here.
TestMaker by PushToTest.com can run recorded scripts such as Selenium as well as many different languages like HTML, Java, Ruby, Groovy, .Net, VB, PHP, etc. It has a common reporting infrastructure and you can create load in your test lab or using cloud testing environments like EC2 for virtual test labs.
They provide free webinars on using open source testing tools on a monthly basis and there is one next Tuesday.
http://www.pushtotest.com
There are a few approaches; I've been in situations, however, where I've had to roll my own load generating utilities.
As far as your test script is concerned it involves:
sending a GET request to http://form entry page (only checking if a 200 response is given)
sending a POST request to http://form submit page with pre-generated key/value pairs for text XXX and performing a regexp check on the response
Unless your web page is complex AJAX there is no need to "simulate a button press" - this is taken care of by the POST request.
Given that your test consists of just a 2-step process there should be several automated load packages that could do this.
I've previously used httperf for load testing a large website: it can simulate a session consisting of several requests and can simulate a large number of users (i.e. sessions) simultaneously. For example, if your website generated a session cookie from the home page you could make that the first request, httperf would then use that cookie for subsequent requests, until it had finished doing the list of requests supplied.
What about http://watin.sourceforge.net/ ?
In Joel's article for Inc. entitled How Hard Could It Be?: The Unproven Path, he wrote:
...it turns out that Jeff and his
programmers were so good that they
built a site that could serve 80,000
visitors a day (roughly 755,000 page
views)
How would I go about figuring out the maximum load my server(s) can handle?
Benchmarking your software is often a lot harder than it seems. Sure, it's easy to produce some numbers that say something about the performance of your software, but unless it was calculated using a very accurate representation of the actual usage patterns of your end users, it might be completely different from the actual results you will get in the wild. Websites are notoriously hard to benchmark correctly. Sure, you can run a script that measures the time it takes to generate a page but it will be a very different number from what you will see under real world usage.
Inorder to create a solid benchmark of what your servers can handle, you first need to figure out what the usage patterns of your users is. If your site is already running, you can easily collect this data from your logs. Next, you need to create a simulation that will emulate exactly the same patterns as your real users exhibit... that is - view front page, login, view status page and so forth. Different pages will create a different load on the servers requiring that you actually fetch correct set of pages when simulating load on your servers. Finally, you need to figure out which resources are cached by your users, you can do this again by looking through your access log or using a tool such as firebug.
JMeter, ab, or httperf
You can create several "stress tests" and run them as the other posters are telling.
Apache has a tool called JMeter where you can create these tests and run them several times.
http://jmeter.apache.org/
Greetings.
Jason, Have you looked at the Load Test built in to Visual Studio 2008 Team System? Check out this video to see a demo.
Edit: Here's another video that has better resolution.
Apache has a tool called ab that you can use to benchmark a server. It can simulate loads requests and concurrency situations for you.
Basically you need to mimic the behavior of a user and keep ramping up the number of users being mimiced until the server response is no longer acceptable.
There are a variety of tools that can do this but essentially you want to record a few sessions activity on your site and then play those sessions back (adding some randomisation to reflect real user behaviour) lots of times.
You will want to log the performance of each session and keep increasing the load until the the performance becomes unacceptable.