About 50% of our data points logged are "Dependency (AJAX)".
Where does this come from ?
I think these "Dependency (AJAX)" began to show up after I enabled Web sockets in the Application Settings of my web app (we are using signalr).
The Property "Command" of these data points have a value of "/signalr/ping", so this definitely has something to do with signalr.
I would like to exclude those, as it's using too much of my free plan available data points.
Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule is already disabled in my ApplicationInsights.config.
EDIT
I tried to disable Web sockets, but still getting those logs.
EDIT 2016-02-24
As suggested by Alex, I set disableAjaxTracking:true, but it didn't help.
Here is the appInsights code in the <head> of my html.
<script type="text/javascript">
var appInsights = window.appInsights || function (config) {
function r(config) { t[config] = function () { var i = arguments; t.queue.push(function () { t[config].apply(t, i) }) } } var t = { config: config }, u = document, e = window, o = "script", s = u.createElement(o), i, f; for (s.src = config.url || "//az416426.vo.msecnd.net/scripts/a/ai.0.js", u.getElementsByTagName(o)[0].parentNode.appendChild(s), t.cookie = u.cookie, t.queue = [], i = ["Event", "Exception", "Metric", "PageView", "Trace"]; i.length;) r("track" + i.pop()); return r("setAuthenticatedUserContext"), r("clearAuthenticatedUserContext"), config.disableExceptionTracking || (i = "onerror", r("_" + i), f = e[i], e[i] = function (config, r, u, e, o) { var s = f && f(config, r, u, e, o); return s !== !0 && t["_" + i](config, r, u, e, o), s }), t
}({
instrumentationKey: "#Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active.InstrumentationKey",
disableAjaxTracking: true
});
window.appInsights = appInsights;
appInsights.trackPageView();
</script>
#if (Request.IsAuthenticated)
{
<script>
appInsights.setAuthenticatedUserContext("#User.Identity.Name".replace(/[,;=| ]+/g, "_"));
</script>
}
Starting from the end of December 2015 Application Insights JavaScript SDK is automatically collecting AJAX requests. Here's a blog post about it.
Since its enablement we also introduced a maximum cap of AJAX requests that can be logged per page view, the default is 500, but you can change this setting by adding this property to your snippet (more on this in the blog and linked documentation):
maxAjaxCallsPerView: <number>
You can also disable AJAX request collection alltogether by using this setting:
disableAjaxTracking: true
EDIT: looks like disableAjaxTracking is broken as of 2/24/2016, until it is fixed, the mitigation is to use
maxAjaxCallsPerView: 0
Related
This question seems to be more or less a duplicate of this one, but that one received no answers and is over 2 years old so I don't know what the protocol is (I tried to find out).
Anyway, I've written an ASP.NET MVC5 web app and it all works fine in debug. After publishing to the server with the Release configuration I started seeing the error "Uncaught ReferenceError: Cannot access 'n' before initialization".
After many hours of trawling through the code I think I've isolated the issue. I have this small function (it's a Knockout view model, but that's irrelevant):
eventIndexViewModel = function (params) {
let self = this;
// Map values from params object to properties of self
for (const [key, value] of Object.entries(params)) {
self['_' + key] = value;
}
self.eventsView = ko.observable(self._eventsView);
self.calendarToggleButtonClass = ko.pureComputed(function () {
return self.eventsView() === "calendar" ? "active" : "";
});
self.tableToggleButtonClass = ko.pureComputed(function () {
return self.eventsView() === "table" ? "active" : "";
});
};
After being minified and published to the server, if I view the source in the dev tools console it looks like this:
eventIndexViewModel = function(n) {
let t = this;
for (const [n,i] of Object.entries(n))
t["_" + n] = i;
t.eventsView = ko.observable(t._eventsView);
t.calendarToggleButtonClass = ko.pureComputed(function() {
return t.eventsView() === "calendar" ? "active" : ""
});
t.tableToggleButtonClass = ko.pureComputed(function() {
return t.eventsView() === "table" ? "active" : ""
})
}
It is overkill to map the properties for the params object in this way in this particular instance, but I have much larger view models with many more properties in the same project and I want to keep them code consistent, so go with it.
Unless I'm misunderstanding something, the minified version has renamed both the params variable and the key variable in the for statement to n, and I think that is what is causing my error. Certainly, that line with the for statement is where the error is thrown.
Am I understanding the cause of this problem correctly? If so, is it a bug in the minification process? And either way, how can I get around it?
I am working on scraping websites, I have tried many technologies to scrape websites.
First of all I used PHP cURL as a scraping tool, and went up to some extent to scrape websites, but then I faced a problem, that was; the PHP cURL couldn't scrape websites that used Ajax to load the website contents/data. And that's what stopped me scraping through PHP.
After a decent research I have found another solution to scrape websites, that were beyond the limitation of Ajax loaded websites etc, and was very powerful and cool to use, they were indeed Phantom JS and Casper JS. I have scraped lot of sites with it.
The problem I faced with these tools was that, these tools works/controlled through the command line interface, for example when you want to run the Phantom/Casper JS code, you need to run it through the command line. And this is my basic problem. What I need is, to write the code in Phantom/Casper JS and I want to have a webpage with admin panel, where I can control these scripts. Currently I am scraping career/jobs listings websites, and I want to automate these tools, to scrape these sites automatically after a given time, to stay updated with the employers sites, who posts new jobs.
For instance, I have code for each website separately and I manually execute each file through the command line and then wait for it to finish scraping and then I continue with second one and so on. What I want to have is, I write a script in JavaScript (preferably in Node JS - but not compulsory) which will execute the scraper code after a specific instance, and then will start scraping all of the websites in the background.
I can do the automation, its not a problem, but the problem is, I am unable to connect the Phantom/Casper JS with the website, even I tried Spooky JS which connects Phantom/Casper JS with Node JS, but unfortunately it doesn't work for me, and its alot messy.
Is there any other tool that's powerful like these two, and I can easily interact with them through a webpage ?
Continuing my own research for scrapping sites, I was unable to find any perfect solution. But the powerful solution I came up with is to use Phantom JS module with Node JS. You can find this module here.
For installation Guide follow this documentation. Phantom JS is used asynchronously in node JS and then its alot easier to get the results, and really easy to interact with it using, express JS on server side and Ajax or Socket.io on client side to enhance the functionality.
Below is my code which I came up with :
const phantom = require('phantom');
const ev = require('events');
const event = new ev.EventEmitter();
var MAIN_URL,
TOTAL_PAGES,
TOTAL_JOBS,
PAGE_DATA_COUNTER = 0,
PAGE_COUNTER = 0,
PAGE_JOBS_DETAILS = [],
IND_JOB_DETAILS = [],
JOB_NUMBER = 1,
CURRENT_PAGE = 1,
PAGE_WEIGHT_TIME,
CLICK_NEXT_TIME,
CURRENT_WEBSITE,
CURR_WEBSITE_LINK,
CURR_WEBSITE_NAME,
CURR_WEBSITE_INDEX,
PH_INSTANCE,
PH_PAGE;
function InitScrap() {
// Initiate the Data
this.init = async function(url) {
MAIN_URL = url;
PH_INSTANCE = await phantom.create(),
PH_PAGE = await PH_INSTANCE.createPage();
console.log("Scrapper Initiated, Please wait...")
return "success";
}
// Load the Basic Page First
this.loadPage = async function(pageLoadWait) {
var status = await PH_PAGE.open(MAIN_URL),
w;
if (status == "success") {
console.log("Page Loaded . . .");
if (pageLoadWait !== undefined && pageLoadWait !== null && pageLoadWait !== false) {
let p = new Promise(function(res, rej) {
setTimeout(async function() {
console.log("Page After 5 Seconds");
PH_PAGE.render("new.png");
TOTAL_PAGES = await PH_PAGE.evaluate(function() {
return document.getElementsByClassName("flatten pagination useIconFonts")[0].textContent.match(/\d+/g)[1];
});
TOTAL_JOBS = await PH_PAGE.evaluate(function() {
return document.getElementsByClassName("jobCount")[0].textContent.match(/\d+/g)[0];
});
res({
p: TOTAL_PAGES,
j: TOTAL_JOBS,
s: true
});
}, pageLoadWait);
})
return await p;
}
}
}
function ScrapData(opts) {
var scrap = new InitScrap();
scrap.init("https://www.google.com/").then(function(init_res) {
if (init_res == "success") {
scrap.loadPage(opts.pageLoadWait).then(function(load_res) {
console.log(load_res);
if (load_res.s === true) {
scrap.evaluatePage().then(function(ev_page_res) {
console.log("Page Title : " + ev_page_res);
scrap.evaluateJobsDetails().then(function(ev_jobs_res) {
console.log(ev_jobs_res);
})
})
}
return
})
}
});
return scrap;
}
module.exports = {
ScrapData
};
}
I have angular-meteor app that needs Material md-autocomplete from a collection with 53,296 documents with angularUtils.directives.dirPagination but this amount of data make my browser hang.
I'm publishing the collection with:
Meteor.publish('city', function (options, searchString) {
var where = {
'city_name': {
'$regex': '.*' + (searchString || '') + '.*' ,
'$options': 'i'
}
};
return City.find(where, options);
});
I subscribe with:
subscriptions: function () {
Meteor.subscribe('city');
this.register('city', Meteor.subscribe('city'));
}
and have pagination on controller :
$scope.currentPage = 1;
$scope.pageSize = 100;
$scope.sort = {city_name_sort : 1};
$scope.orderProperty = '1';
$scope.helpers({
city: function(){
return City.find({});
}
});
but it takes a long time to load and its make chrome stop working.
You already have most of the server-side searching done because your search is running inside a subscription. You should make sure that the city_name field is indexed in mongo! You should only return that field to minimize data transfer. You can also simplify your regex.
Meteor.publish('city', function (searchString) {
const re = new RegExp(searchString,'i');
const where = { city_name: { $regex: re }};
return City.find(where, {sort: {city_name: 1}, fields: {city_name: 1}});
});
What I've found helps with server-side auto-complete is:
Don't start searching until the user has typed 3 or 4 characters. This drastically narrows down the search results.
Throttle the search to only run every 500ms so that you're not sending every character to the server because then it has to keep re-executing the search. If the person is typing fast the search might only run every 2 or 3 characters.
Run the same .find() on the client that you're running on the server (instead of just querying for {}). That's just good practice since the client-side collection is the union of all subscriptions on that collection, there might be documents there that you don't want to list.
Lastly I don't know why you're subscribing twice here:
subscriptions: function () {
Meteor.subscribe('city');
this.register('city', Meteor.subscribe('city'));
}
only one of those Meteor.subscribe('city') calls is necessary.
I'm not having much luck so far in loading the filepicker package in my Meteor project.
What I did:
$cd ~/myMeteorProject
$mrt add filepicker
>>>Done installing smart packages
$head smart.json
>>>{
"packages": {
"router": {},
"filepicker": {}
}
}
$mrt
>>>Stand back while Meteorite does its thing
Done installing smart packages
Ok, everything's ready. Here comes Meteor!
[[[[[ ~/myMeteorProject ]]]]]
=> Meteor server running on: http://localhost:3000/
So at this point all looks like I'd expect. ( I even double checked the contents of the filepicker package and it contains all of what i'd expect, the source to load has the same URL as is found on the filepicker.io site, etc.)
However, when I try to run the following (compiled from coffeescript):
if (Meteor.isClient) {
Meteor.startup(function() {
return filepicker.setKey('A9FiXXdu5RB^GYujfDPwlz'); //not my actual key, don't worry
});}
I get: Uncaught ReferenceError: filepicker is not defined
So, that's kind of a bummer. Any ideas? I've tried removing and re-adding both coffeescript and filepicker. is there some load-order issue? I note that filepicker-load.js has an alert if the script fails to load, which I'm not seeing...
I've had the exact same problem and for me, neither wrapping it in a setTimeout() nor a setInterval() worked.
Ink themselves provide a great non-blocking script in their docs, that queues all filepicker calls until its fully loaded and then executes them in the order they were called. This is how it works:
mrt remove filepicker (if added)
mrt remove loadpicker (if added)
create new file /lib/filepicker.js
/lib/filepicker.js
if (Meteor.isClient) {
(function(a) {
if (window.filepicker) {
return
}
var b = a.createElement("script");
b.type = "text/javascript";
b.async = !0;
b.src = ("https:" === a.location.protocol ? "https:" : "http:") + "//api.filepicker.io/v1/filepicker.js";
var c = a.getElementsByTagName("script")[0];
c.parentNode.insertBefore(b, c);
var d = {};
d._queue = [];
var e = "pick,pickMultiple,pickAndStore,read,write,writeUrl,export,convert,store,storeUrl,remove,stat,setKey,constructWidget,makeDropPane".split(",");
var f = function(a, b) {
return function() {
b.push([a, arguments])
}
};
for (var g = 0; g < e.length; g++) {
d[e[g]] = f(e[g], d._queue)
}
window.filepicker = d
})(document);
filepicker.setKey(YOUR_FILEPICKER_KEY);
}
This client code is a vanilla version of the non-blocking loader taken from https://developers.inkfilepicker.com/docs/web/. Using this, you can set the key once and then use filepicker as you like without worrying about if it's already loaded or not.
Of course you could also modify loadpicker with this, just paste in this code in your loadpicker.js
loadpicker.js
loadPicker = function(key) {
(function(a) {
if (window.filepicker) {
return
}
var b = a.createElement("script");
b.type = "text/javascript";
b.async = !0;
b.src = ("https:" === a.location.protocol ? "https:" : "http:") + "//api.filepicker.io/v1/filepicker.js";
var c = a.getElementsByTagName("script")[0];
c.parentNode.insertBefore(b, c);
var d = {};
d._queue = [];
var e = "pick,pickMultiple,pickAndStore,read,write,writeUrl,export,convert,store,storeUrl,remove,stat,setKey,constructWidget,makeDropPane".split(",");
var f = function(a, b) {
return function() {
b.push([a, arguments])
}
};
for (var g = 0; g < e.length; g++) {
d[e[g]] = f(e[g], d._queue)
}
window.filepicker = d
})(document);
filepicker.setKey(key)
};
Hope this works for you!
your syntax looks good and matches what i use in my app - i'm thinking that since filepicker loads the script by injecting a script element, you have a timing issue (calling setKey before the script is loaded)
maybe hold off on setting the key until the user does something, like
filepicker.setKey("KEY");
filepicker.pickAndStore({...},function(error){...});
or set it with a timeout
Meteor.setTimeout(function(){
filepicker.setKey("KEY");
},1000);
you could also use an interval to check if desired (more tolerant of long load times) - you could adjust this to give up after a certain amount of time
var filepickerInterval = Meteor.setInterval(function(){
if(filepicker){
Meteor.clearInterval(filepickerInterval);
filepicker.setKey("KEY");
}
}, 100);
You also can use the asynchronous javascript snippet to auto-queue calls to the filepicker object until the script loads. See https://developers.inkfilepicker.com/docs/web/#javascript
I have a website which is using Google Analytics newer asynchronous tracking method (_gaq). The problem I've run into is that I want to institute some specific link tracking and am worried that I will be creating a race condition.
Basically, it's a news website so it has headlines which link to stories all over the place. A headline for a story might appear in 3 different places on a page, and appear on hundreds of other pages. Thus, in order to understand how our audience is interacting with the site we have to track how each specific headline block is used, and not just the destination. Because of those two stipulations tracking individual pages, nor tracking referred pages won't be enough, we have to track individual links.
So if I have a link.
Here
Because _gaq.push() is an asynchronous call, isn't it possible that the page change will occur prior to Google's completion of the click tracking? If so is there a way to prevent that, or do I have a misunderstanding about the way that Google Analytics Async functions (http://code.google.com/apis/analytics/docs/tracking/asyncUsageGuide.html).
You're right. If the browser leaves the page before it sends the GA tracking beacon (gif hit) for the event, the event will not be recorded. This is not new to the async code however, because the process of sending the tracking beacon is asynchronous; the old code worked the same way in that respect. If tracking is really that important, you could do something like this:
function track(link) {
if (!_gat) return true;
_gaq.push(['_trackEvent', 'stuff']);
setTimeout(function() {location.href=link.href'}, 200);
return false;
}
...
This will stop the browser from going to the next page when the link is clicked if GA has been loaded already (it's probably best to not make the user wait that long). Then it sends the event and waits 200 milliseconds to send the user to the href of the link they clicked on. This increases the likelihood that the event will be recorded. You can increase the likelihood even more by making the timeout longer, but that also may be hurting user-experience in the process. It's a balance you'll have to experiment with.
I've got this problem too, and am determined to find a real solution.
What about pushing the function into the queue?
// Log a pageview to GA for a conversion
_gaq.push(['_trackPageview', url]);
// Push the redirect to make sure it happens AFTER we track the pageview
_gaq.push(function() { document.location = url; });
From Google's documentation for universal analytics (new version since most other answers for this question). You can now easily specify a callback.
var trackOutboundLink = function(url) {
ga('send', 'event', 'outbound', 'click', url, {'hitCallback':
function () {
document.location = url;
}
});
}
For clarity I'd recommend using this syntax, which makes it clearer which properties you're sending and easier to add more :
ga('send', 'event', {
'eventCategory': 'Homepage',
'eventAction': 'Video Play',
'eventLabel': label,
'eventValue': null,
'hitCallback': function()
{
// redirect here
},
'transport': 'beacon',
'nonInteraction': (interactive || true ? 0 : 1)
});
[Here's a complete list of parameters for all possible ga calls.]
In addition I've added the transport parameter set to beacon (not actually needed because it's automatically set if appropriate):
This specifies the transport mechanism with which hits will be sent.
The options are 'beacon', 'xhr', or 'image'. By default, analytics.js
will try to figure out the best method based on the hit size and
browser capabilities. If you specify 'beacon' and the user's browser
does not support the navigator.sendBeacon method, it will fall back
to 'image' or 'xhr' depending on hit size.
So when using navigator.beacon the navigation won't interrupt the tracking . Unfortunately Microsoft's support for beacon is non existent so you should still put the redirect in a callback.
In event handler you should setup hit callback:
_gaq.push(['_set', 'hitCallback', function(){
document.location = ...
}]);
send you data
_gaq.push(['_trackEvent'
and stop event event processing
e.preventDefault();
e.stopPropagation();
I'm trying out a new approach where we build the URL for utm.gif ourselves, and request it, then only once we've received the response (the gif) we send the user on their way:
Usage:
trackPageview(url, function() { document.location = url; });
Code (CrumbleCookie from: http://www.dannytalk.com/read-google-analytics-cookie-script/)
/**
* Use this to log a pageview to google and make sure it gets through
* See: http://www.google.com/support/forum/p/Google%20Analytics/thread?tid=5f11a529100f1d47&hl=en
*/
function trackPageview(url, fn) {
var utmaCookie = crumbleCookie('__utma');
var utmzCookie = crumbleCookie('__utmz');
var cookies = '__utma=' + utmaCookie + ';__utmz=' + utmzCookie;
var requestId = '' + (Math.floor((9999999999-999999999)*Math.random()) + 1000000000);
var hId = '' + (Math.floor((9999999999-999999999)*Math.random()) + 1000000000);
var utmUrl = 'http://www.google-analytics.com/__utm.gif';
utmUrl += '?utmwv=4.8.9';
utmUrl += '&utmn=' + requestId;
utmUrl += '&utmhn=' + encodeURIComponent(window.location.hostname);
utmUrl += '&utmhid=' + hId;
utmUrl += '&utmr=-';
utmUrl += '&utmp=' + encodeURIComponent(url);
utmUrl += '&utmac=' + encodeURIComponent(_gaProfileId);
utmUrl += '&utmcc=' + encodeURIComponent(cookies);
var image = new Image();
image.onload = function() { fn(); };
image.src = utmUrl;
}
/**
* #author: Danny Ng (http://www.dannytalk.com/read-google-analytics-cookie-script/)
* #modified: 19/08/10
* #notes: Free to use and distribute without altering this comment. Would appreciate a link back :)
*/
// Strip leading and trailing white-space
String.prototype.trim = function() { return this.replace(/^\s*|\s*$/g, ''); }
// Check if string is empty
String.prototype.empty = function() {
if (this.length == 0)
return true;
else if (this.length > 0)
return /^\s*$/.test(this);
}
// Breaks cookie into an object of keypair cookie values
function crumbleCookie(c)
{
var cookie_array = document.cookie.split(';');
var keyvaluepair = {};
for (var cookie = 0; cookie < cookie_array.length; cookie++)
{
var key = cookie_array[cookie].substring(0, cookie_array[cookie].indexOf('=')).trim();
var value = cookie_array[cookie].substring(cookie_array[cookie].indexOf('=')+1, cookie_array[cookie].length).trim();
keyvaluepair[key] = value;
}
if (c)
return keyvaluepair[c] ? keyvaluepair[c] : null;
return keyvaluepair;
}
Using onmousedown instead of onclick may also help. It doesn't eliminate the race condition, but it gives GA a head start. There's also the concern of someone clicking on a link and dragging away before letting go of the mouse button, but that's probably a negligible case.