I have a few questions regarding this, as I don't want to submit the application form without knowing how it works. I have a turn based online game with around 150k DAU. So that means there are alot of writes, reads and listeners. My (RTDB) load peaks at about 30% atm.
Default calls per day is 10,000. What happens when those run out, will the calls beyond that return errors?
Would it cost me alot more given the amount of users?
In regards to performance, is there something like a "known average percent increase" in read/write operations? It's described in the tutorial video as "instant" but everything takes a little time.
How do I know what amount to apply for? I'm guessing it's 1M-10M or 10M-100M but it's hard to know..
Thankful for any response :)
I've been trying to use sequenceCount in order to process a funnel analysis on a website data. We want to track how many user can reach from point A to point B. sequenceCount is a perfect fit for this need as we have trackers that follows the user's interaction on the website. However, let's say that there is not only one tracker but many that trigger the point A in our funnel analysis. This would mean that currently we would have to use sequenceCount for EACH triggers A and make them point to B.
This problem might lead to situation where we have to write plenty of sequenceCount and could kill all performance.
I've been looking on the web for answers but I didn't find any about that.
I'm looking for thoughts that could solve the problem or avoid it
Background
I'm writing an Alexa Skill and looking to get pieces of information from the user.
The following conversation for example:
Alexa: What month were you born at?
User: April
Alexa: Good. And what was your favorite movie?
User: April
The problem
Given the following utterances:
GetMonthIntent {month}
GetMovieIntent {movie}
Once a user answers April for the second time, the GetMonthIntent might be triggered.
What I have tried
Asking the user to specify which piece of information is giving by using the following utterances:
GetMonthIntent Month {month}
GetMovieIntent Movie {movie}
The question
What is the right way to make Alexa wait for a single term answer based on the current context?
In the same vein as the other answers here, you should take a look at the newest Node.JS library here, which handles state out of the box:
https://github.com/alexa/alexa-skills-kit-sdk-for-nodejs#making-skill-state-management-simpler
You could define:
State_Launch
State_Month
State_Movie
And then return the proper error response if anything other than the GetMovieIntent or GetMonthIntent (etc.) intents are called in the wrong state.
You would have to do data validation on the server side to make sure the "month" is a valid one, and movies are even harder to validate unless you have a list of expected values. That is, if you care to parse them for use beyond repeating back.
Unfortunately, there is no solution. There is no way to specify the 'context' in which a user reply should be interpreted, so you have to tell the user "what was your favorite movie? Please say 'my favorite movie is' and then the name of the movie".
Here are two ASK feature requests that I think would address your issue:
https://forums.developer.amazon.com/content/idea/41062/creating-something-to-help-with-more-structured-qu.html
https://forums.developer.amazon.com/content/idea/55525/allow-a-response-to-specify-a-set-of-expected-inte.html
Personally I think this is fairly important so I voted for those, but they are not near the top.
I ran into this same problem when I created the "Who's on First? Baseball Skit" skill. I handled this by:
Create a sequence number for each response given by Alexa
Write this number to the "session" in the response.
The session is then passed back to your skill by Alexa in the next request.
Read the sequence number from the request to know what the previous question was.
If a given intent could be the answer to multiple questions (eg. month and movie in your case) then use the sequence number to determine which it is.
This should give you ideas on how to deal with repeated answers. The session is quite easy to use. Other options include writing the userId and status to a database like DynamoDB, but I find that the session works in most cases.
I can't find anything on the web about how to sample Adobe Analytics data? I need to integrate Adobe Analytics into a new website with a ton of traffic so the stakeholders want to sample the data to avoid exorbitant server calls. I'm using DTM but not sure if that will help or be a non-factor? Can anyone either point me to some documentation or give me some direction on how to do this?
Adobe Analytics does not have any built-in method for sampling data, neither on their end nor in the js code.
DTM doesn't offer anything like this either. It doesn't have any (exposed) mechanisms in place to evaluate all requests made to a given property (container); any rules that extend state beyond "hit" scope are cookie based.
Adobe Target does offer ability to output code based on % of traffic so you can achieve sampling this way, but really, you're just trading one server call cost for another.
Basically, your only solution would be to create your own server-side framework for conditionally outputting the Adobe Analytics (or DTM) tag, to achieve sampling with Adobe Analytics.
Update:
#MichaelJohns comment below:
We have a file that we use as a boot strap file to serve the DTM file.
What I think we are going to do is use some JS logic and cookies
around that to determine if a visitor should be served the DTM code.
Okay, well maybe i'm misunderstanding what your goal here is (but I don't think I am) but that's not going to work.
For example, if you only want to output tracking for 50% of visitors, how would you use javascript and cookies alone to achieve this? In order to know that you are only filtering 50%, you need to know the total # of people in play. By itself, javascript and cookies only know about ONE browser, ONE person. It has no way of knowing anything about those other 99 people unless you have some sort of shared state between all of them, like keeping track of a count in a database server-side.
The best you can do solely with javascript and cookies is that you can basically flip a coin. In this example of 50%, basically you'd pick a random # between 1 and 100 and lower half gets no tracking, higher half gets tracking.
The problem with this is that it is possible for the pendulum to swing 100% one way or the other. It is the same principle as flipping a coin 100 times in a row: it is entirely possible that it can land on tails all 100 times.
In theory, the trend over time should show an overall average of 50/50, but this has a major flaw in that you may go one month with a ton of traffic, another month with few. Or you could have a week with very little traffic followed by 1 day of a lot of traffic. And you really have no idea how that's going to manifest over time; you can't really know which way your pendulum is swinging unless you ARE actually recording 100% of the traffic to begin with. The affect of all this is that it will absolutely destroy your trended data, which is the core principle of making any kind of meaningful analysis.
So basically, if you really want to reliably output tracking to a % of traffic, you will need a mechanism in place that does in fact record 100% of traffic. If I were going to roll my own homebrewed "sampler", I would do this:
In either a flatfile or a database table I would have two columns, one representing "yes", one representing "no". And each time a request is made, I look for the cookie. If the cookie does NOT exist, I count this as a new visitor. Since it is a new visitor, I will increment one of those columns by 1.
Which one? It depends on what percent of traffic I am wanting to (not) track. In this example, we're doing a very simple 50/50 split, so really, all I need to do is increment whichever one is lower, and in the case that they are currently both equal, I can pick one at random. If you want to do a more uneven split, e.g. 30% tracked, 70% not tracked, then the formula becomes a bit more complex. But that's a different topic for discussion ( also, there are a lot of papers and documents and wikis out there published by people a lot smarter than me that can explain it a lot better than me! ).
Then, if it is fated that that I incremented the "yes" column, I set the "track" cookie to "yes". Otherwise I set the "track" cookie to "no".
Then in in my controller (or bootstrap, router, whatever all requests go through), I would look for the cookie called "track" and see if it has a value of "yes" or "no". If "yes" then I output the tracking script. If "no" then I do not.
So in summary, process would be:
Request is made
Look for cookie.
If cookie is not set, update database/flatfile incrementing either yes or no.
Set cookie with yes or no.
If cookie is set to yes, output tracking
If cookie is set to no, don't output tracking
Note: Depending on language/technology of your server, cookie won't actually be set until next request, so you may need to throw in logic to look for a returned value from db/flatfile update, then fallback to looking for cookie value in last 2 steps.
Another (more general) note: In general, you should beware sampling. It is true that some tracking tools (most notably Google Analytics) samples data. But the thing is, it initially records all of the data, and then uses complex algorithms to sample from there, including excluding/exempting certain key metrics from being sampled (like purchases, goals, etc.).
Just think about that for a minute. Even if you take the time to setup a proper "sampler" as described above, you are basically throwing out the window data proving people are doing key things on your site - the important things that help you decide where to go as far as giving visitors a better experience on your site, etc..so now the only way around it is to start recording everything internally and factoring those things in to whether or not to send the data to AA.
But all that aside.. Look, I will agree that hits are something to be concerned about on some level. I've worked with very, very large clients with effectively unlimited budgets, and even they worry about hit costs racking up.
But the bottom line is you are paying for an enterprise level tool. If you are concerned about the cost from Adobe Analytics as far as your site traffic.. maybe you should consider moving away from Adobe Analytics, and towards a different tool like GA, or some other tool that doesn't charge by the hit. Adobe Analytics is an enterprise level tool that offers a lot more than most other tools, and it is priced accordingly. No offense, but IMO that's like leasing a Mercedes and then cheaping out on the quality of gasoline you use.
I think the question has been answered here before,but i could not find the desired topic.I am a newbie in web scraping.I have to develop a script that will take all the google search result for a specific name.Then it will grab the related data against that name and if there is found more than one,the data will be grouped according to their names.
All I know is that,google has some kind of restriction on scraping.They provide a custom search api.I still did not use that api,but hoping to get all the resulted links corresponding to a query from that api. But, could not understand what will be the ideal process to do the scraping of the information from that links.Any tutorial link or suggestion is very much appreciated.
You should have provided a bit more what you have been doing, it does not sound like you even tried to solve it yourself.
Anyway, if you are still on it:
You can scrape Google through two ways, one is allowed one is not allowed.
a) Use their API, you can get around 2k results a day.
You can up it to around 3k a day for 2000 USD/year. You can up it more by getting in contact with them directly.
You will not be able to get accurate ranking positions from this method, if you only need a lower number of requests and are mainly interested in getting some websites according to a keyword that's the choice.
Starting point would be here: https://code.google.com/apis/console/
b) You can scrape the real search results
That's the only way to get the true ranking positions, for SEO purposes or to track website positions. Also it allows to get a large amount of results, if done right.
You can Google for code, the most advanced free (PHP) code I know is at http://scraping.compunect.com
However, there are other projects and code snippets.
You can start off at 300-500 requests per day and this can be multiplied by multiple IPs. Look at the linked article if you want to go that route, it explains it in more details and is quite accurate.
That said, if you choose route b) you break Googles terms, so either do not accept them or make sure you are not detected. If Google detects you, your script will be banned by IP/captcha. Not getting detected should be a priority.