is it possible to control the budget for each training session in azure ML.
Lets say I am training a model and before starting the run, I want to set a budget of X dollars. and if it crosses, then my model training should pause unless I increase the budget to X+Y dollars.
Once the budget is increased, then only model training should resume. Is there any raw rest HTTP API or SDK options to control this?
Today, there is no way to control cost at the granularity of a job.
Here is what can be done today:
You can define quotas and usage both at the subscription and the workspace level (i.e. control which compute can be used) -- this is done in https://portal.azure.com
You can set up budgets on the resource group level with Cost Management -- but these are just to control and alert and won't limit any jobs (meaning, that a resource group can exceed it's budget).
You can set up a spending limit on the subscription level, but that shuts down most/all services that cost money, so not a great option either.
https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/spending-limit
Related
I use Google Optimize with server-side A/B experiments. Server-side code targets audiences, consistently serves the appropriate variant to each user, etc. I only use the Optimize interface to create experiments, set objectives, create variants, and view reports. I configure Optimize as described in the Server-side experiments with Universal Analytics (analytics.js) documentation.
I have performed A/A test of the Homepage with 50:50 split. The target objective is event action - click some specific button.
I run UI load tests that open the Homepage and then in 30% of cases click the desired button otherwise opens another page. In total 4000 iterations.
I expect to see calculated conversion rate 30% (this is a part of the UI load test logic) but Optimize shows conversion rate 16% (almost a half from the expected). The number of experiment sessions is much higher then 4000 (as assumed from the test), thus, the conversion rate is lower.
The question is how Google Optimize calculates experiment sessions? Why the number of sessions is higher then actual number of the experiment page views? How conversion rate is calculated?
I'm currently building a data-driven attribution model for my company. It relies on Markov Chains, and I have used the package ChannelAttribution in R which is great!
It gave me results at global level: what is the weight of every channel on total conversions and revenue. But now I have to move towards deployment phase in the systems.
To do that, I need that every new transaction happening gets its revenue distributed on channels in the path. My problem is that Markov Model for attribution is not a "predictive" model so there is no output at the transaction level (it basically simulates paths to calculate removal effects so no granular info).
Does anyone have an idea on how to translate the global output of the model into a set of rules that would allow for distributing revenue of new transactions in real time (or approx. real time like everyday)? I guess there can be additional hypothesis or another layer of modelling that could do the trick but I can't put my finger on it.
Any help appreciated!
Thanks,
Baptiste
I have created and implemented a predictive model that provides us with the probability that an account will convert. We also have the expected value of each account.
Our team members are graded on two parts:
(Accounts Converted)/(Total Accounts Assigned to Member)
And
(Value of Accounts converted)/(Total Value of all assigned accounts)
The average of the two is used to Grade each team member. So converting accounts is not always the best idea, you want to convert big accounts that are worth more $$.
The Question:
If a team member makes 200 calls a day, which accounts should he/she be working on to maximize their grade?
Since I have the probability that each account will convert, I would like to run a simulation to design a strategy to optimize the team members efforts and optimize their Grade. I am not sure where to start or how to design the simulation.
Would a Monte-Carlo Simulation work for this particular problem?
I would usually provide my attempt, but I am not sure where to start here.
Since the simulation is entirely based on probabilities(is not empirical), you dont need montecarlo, you can simply compute the expected value and assume an error in your predictions.
I'm attempting to train a model to identify certain tags in images. I tried using the 1 hour free version and after an hour the training ended. The results weren't as accurate as I would like, so I took the plunge and selected the option which did not define a specific time limit for training the model.
At this time, it is showing "training vision classification model" and "Training can take 15 minutes to several hours or more, depending on the compute hours assigned. In the meantime, you can close this window. You will be emailed once training completes."
It's been running for over 24 hours now. Google charges $20 per training hour so I would like to know if there is a way to see how much longer it would take to fully train the model.
I'm using it using 2594 images with only single labels.
I understand Google AutoML Vision is still in beta phase but at this time, there is no way to know how long this training will last. I'm under a strict budget so I will stop the training if it exceeds the time -- but I would still like to know if it was near completion.
The time required to train your model will completely depends and vary based on the amount of data included within your dataset and the accuracy desired for your model; therefore, I think it is not possible to determine the number of hours needed to complete the training phase. The GCP pricing documentation mentions the following suggestion:
Many customers find that one hour is sufficient to build an experimental model and use additional training hours to increase accuracy to a production level.
Note: Keep in mind that the accuracy of your model generally depends on how long you allow it to train and the quality of your training dataset.
On the other hand, in case you have an specific budget for your model, it is recommended to use the training budget parameter in order to specify the number of hours of training to use; In this way, AutoML Vision can guarantee that the actual training time will be less than or equal to the training budget. I suggest you to take a look on the AutoML Vision API Tutorial where you can find an example of the usage of the training model property.
Additionally, it is important to note that, when training a model, a compute hour represents internal compute usage, which means that this time might not exactly match an actual hour on the clock, as mentioned in the Training costs. Based on this, I recommend you to consider this aspect when calculating the your budget in order to avoid exceeding it.
What will happen to total project cost, if activity which is in progress (2 days complete out of 5), is deleted in primavera?
The Budgeted Total Cost for the project will no longer include the cost for that activity (assuming the deleted activity has an associated budgeted Resource cost or Budgeted Expense cost). Also, there were Actual Costs associated with the 2 days of progress, then the Total Actual Cost of the project would not include those costs as well.
Michael - Plan Academy
In the simple case where there is no baseline, then all the costs associated with the activity are removed from the calculations in Primavera. You can see this by adding the At Completion Total Cost to the WBS view, making a note of the values and then deleting the Activity.
If there is a Baseline attached to the project then the costs for the deleted activities remain inside the Baseline but they cannot be seen. This is because Primavera links from the Current Project into the Baseline Project through the Actiivity Id.
This also means the Activity and it's cost stays in the Baseline. If you want to see what I mean just create a Baseline and make it the Project Baseline. Then add the BL Project Total Cost to the WBS view. Take a note of the values. Then make a note of the Activity Id of an Activity with cost on it. Then delete or dissolve that Activity. The BL Project Total Cost and the At Completion Total Cost goes down in value.
Now create a brand new Activity and give it the same Activity Id as the one that has been deleted, but do not add any costs to it. You will see there is no change to the At Completion Total Cost, but the BL Project Total Cost has increased.
So what? Away from the technical capability of P6, it isn't a good idea to just delete Activities from a project, particularly when they have actual costs on them. It is far better to give the activity an Actual Finish date of when it was cancelled and make sure the actual costs are correct.
When there is a Baseline as well (and why shouldn't there be a baseline :-)) the activity in the baseline may need to be changed if the cancelling of the activity is a change to the scope.