I'm setting up Graphite, and hit a problem with how data is represented on the screen when there's not enough pixels.
I found this post whose first answer is very close to what I'm looking for:
No what is probably happening is that you're looking at a graph with more datapoints than pixels, which forces Graphite to aggregate the datapoints. The default aggregation method is averaging, but you can change it to summing by applying the cumulative() function to your metrics.
Is there any way to get this cumulative() behavior by default?
I've modified my storage-aggregation.conf to use 'aggregationMethod = sum', but I believe this is for historical data and not for data that's displayed in the UI.
When I apply cumulative() everything is perfect, I'm just wondering if there's a way to get this behavior by default.
I'm guessing that even though you've modified your storage-aggregation.conf to use 'aggregationMethod = sum', your metrics you've already created have not changed their aggregationMethod. The rules in storage-aggregation.conf only affect new metrics.
To change your existing metrics to be summed instead of averaged, you'll need to use whisper-resize.py. Or you can delete your existing metrics and they'll be recreated with sum.
Here's an example of what you might need to run:
whisper-resize.py --xFilesFactor=0.0 --aggregationMethod=sum /opt/graphite/storage/whisper/stats_counts/path/to/your/metric.wsp 10s:28d 1m:84d 10m:1y 1h:3y
Make sure to run that as the same user who owns the file, or at least make sure the files have the same ownership when you're done, otherwise they won't be writeable for new data.
Another possibility if you're using statsd is that you're just using metrics under stats instead of stats_counts. From the statsd README:
In the legacy setting rates were recorded under stats.counter_name
directly, whereas the absolute count could be found under
stats_count.counter_name. With disabling the legacy namespacing those
values can be found (with default prefixing) under
stats.counters.counter_name.rate and stats.counters.counter_name.count
now.
Basically, metrics are aggregated differently under the different namespaces when using statsd, and you want stuff under stats_count or stats.counters for things that should be summed.
Related
I am looking to get the values from a reactive function, based on a specific set of inputs, but without the dashboard open or visible. Is this possible?
I have two possible solutions, both of them I am not sure are possible, and which of the two would be better. Also, I do think there could be a better solution.
Start the dashboard at a specific time and write away a csv with the required information. I don't know how to put the input values to the right settings though (they would be different from the normal initial settings). Hopefully start this without a visible endpoint (headless chromedriver for example).
Within Shiny, do a check if it's a specific time. If that's the case, change all inputs to the right settings, and export/do something with the required values. I don't prefer this, as there will be users using the dashboard at that time, and I don't want to disturb their work.
To give a bit more context, I would like to obtain values/dataframes from several dashboards at a specific time. These values are calculated via a chain of multiple reactives and inputs. I cannot trust that these dashboards are already running, so I need to start them.
I would like to grab satellite positions from the page(s) below, but I'm not sure if scraping is appropriate because the page appears to be updating itself every second using some internal code (it keeps updating after I disconnect from the internet). Background information can be found in my question at Space Stackexchange: A nicer way to download the positions of the Orbcomm-2 satellites.
I need a "snapshot" of four items simultaneously:
UTC time
latitude
longitude
altitude
Right now I use screen shots and manual typing. Since these values are being updated by the page - is conventional web-scraping going to work here? I found a "screen-scraping" tag, should I try to learn about that instead?
I'm looking for the simplest solution to get those four values, I wonder if I can just use urllib or urllib2 and avoid installing something new?
example page: http://www.satview.org/?sat_id=41186U I need to do 41179U through 41189U (the eleven Orbcomm-2 satellites that SpaceX just put in orbit)
Those values are calculated with a little math using javascript. The calculations are detailed here: https://www.satview.org/track.js
So, I guess one option is to write that script (plus any dependencies) in the language of your choice and use it to return your desired values.
There is one major function track() which takes arg $modo which can be one of two values - tic or plot.
There may be other source files (dependencies) referenced:
The easier way would probably be to use something which allows javascript to run e.g. automating a browser, and extracting the calculated values as they are generated.
I am starting a customer lifetime project at work and want to share how the data looks with the business, as I want to be able to identify the important variables with them. I plan to do this using the excellent rpivottable package and launch a shiny app to see where there are basic differences in groups to select my features.
This would mean I have my customer base of 4million customers and slice and dice them in a number of ways.
However, following GDPR we need to ensure no group is shown that has less than 7 customers in it. Therefore I need somekind of background calculation to ensure that less than 7 customers are never shown.
If I think logically about this, the only way I could see it working would be to make a change to the pivottable, have some form of submit button, so that the size of groups could be calculated, and then a filter (which needs to be hidden from the user so it cannot be switched off) is applied.
I know I should provide code, but I do not know where to start here. Has anyone had similar issues and has a potential solution to all or part of the problem?
Has anyone built a hidden filter into their rpivottable?
Has anyone been able to restrict their output to only show 90% of their data?
Thanks,
J
To be absolutely sure, you would need to load in a data frame that looks like "dim, dim, dim, count" where count is always greater than 7. Basically just a bit of preprocessing on your input data. Unfortunately, this means that you will be restricted to a small number of coarse dimensions, else you will end up filtering out everything.
Two questions: first, is it true that the only way to update '.priority' via REST is by using PUT, which forces you to rewrite all the other values? And I'm afraid to ask, but does that also apply to the Javascript SDK?
Second, is there maybe some other way other than using Priorities to order your collection 'server-side'?
Sorting things on the client doesn't work for me because I'm using a masonry-type plugin for layout which goes bonkers whenever the order changes client-side, but it seems to work fine server side. I'm using PHP to degrade the '.priority' value over time, lowering the item's position in the collection, but I'm forced to rewrite every other field at cron run. It works, but it would be better to just be able to update '.priority' or some other value that controls the position.
Question 1: .priority via REST API
You can set the priority without modify the record by calling PUT on the .priority directly. These examples are found in the REST API doc:
So, to reiterate, add .priority into the URL.
Question 2: Hacking masonry sorting in Angular using the data store?
If you haven't explored your options here, there are several libs dedicated to integrating masonry and angular which you may want to check out (e.g. angular-masonry). You may also have luck address sorting data in masonry directly, in its own SO question, rather than trying to solve it with your data store, which seems like an XY Problem.
The records in Firebase are sorted lexicographically, so you have three options:
name your items so they sort in the desired order
use push ids, which are ordered chronologically
use priorities to enforce a sort order other than the record id's natural sorting
Keep in mind that numeric keys, when mixed with lexicographic strings, cause behave strangely--err, by design--in Chrome. And, tangentially related, such a debate has waged that the ECMAScript standard will actually be changed to force them to correct it
I've started using Access 2010 recently and started testing some of the new features, namely the Calculated Field datatype.
I had hoped that this was something that based on a formula (expression builder) would remove an amount of data and shrink an ACCDB file because Access only has the formula not actual data.
However, my new version of the file seems to be larger than the original which IMHO makes the feature a bit useless.
I've searched the interweb regarding the feature and can only really find people who show how to create one rather than any pros and cons about the feature.
As it stands I'm going to go back to the old method of calculations in a query but before I do I thought I'd ask on StackOverflow just in case anybody has used it.
Access stores the results of calculated fields for each record, so yes, that will increase the size of the database. However your claim that this "makes the feature a bit useless" misses the point:
The primary advantage of using calculated fields is that the calculation (expression) is defined once, at the table level. Once the calculated field has been defined it can simply be used much like any other field in queries, reports, etc..
Sure, you can "go back to the old method of calculations in a query" if that suits your purposes, but it also means that
You will have to repeat the (same) calculation logic in all of your queries.
If the calculation logic ever changes then you'll have to go back and edit all of those queries.
Every time you run one of those queries it will have to re-do the calculation for every record, instead of simply retrieving the calculated field from the table.