Applying clip filter to several sources - pipeline

I have several data source in Paraview, and clip all of them (with the Clip filter) to only the are of interest I need to see at this moment. It istedious to set the clip domain always three (or more) times. Is it somehow possible to share those parameters, or apply the same filter instance to multiple data sources?

You can select all the data sources, then apply the Group Datasets filter. This will create a single data set (without actually duplicate the heavy data). Now you can just apply a single Clip filter.
You can alternatively use the Links Manager, accessible from Tools | Manage Links to set up property links between several Clip filters. This, however, had issues in some old versions. I'd suggest using 4.1.0 if you're going this route.

Related

How to scrape data on a web site that is changing contents in active manner?

I would like to grab satellite positions from the page(s) below, but I'm not sure if scraping is appropriate because the page appears to be updating itself every second using some internal code (it keeps updating after I disconnect from the internet). Background information can be found in my question at Space Stackexchange: A nicer way to download the positions of the Orbcomm-2 satellites.
I need a "snapshot" of four items simultaneously:
UTC time
latitude
longitude
altitude
Right now I use screen shots and manual typing. Since these values are being updated by the page - is conventional web-scraping going to work here? I found a "screen-scraping" tag, should I try to learn about that instead?
I'm looking for the simplest solution to get those four values, I wonder if I can just use urllib or urllib2 and avoid installing something new?
example page: http://www.satview.org/?sat_id=41186U I need to do 41179U through 41189U (the eleven Orbcomm-2 satellites that SpaceX just put in orbit)
Those values are calculated with a little math using javascript. The calculations are detailed here: https://www.satview.org/track.js
So, I guess one option is to write that script (plus any dependencies) in the language of your choice and use it to return your desired values.
There is one major function track() which takes arg $modo which can be one of two values - tic or plot.
There may be other source files (dependencies) referenced:
The easier way would probably be to use something which allows javascript to run e.g. automating a browser, and extracting the calculated values as they are generated.

Making Graphite UI data cumualtive by default

I'm setting up Graphite, and hit a problem with how data is represented on the screen when there's not enough pixels.
I found this post whose first answer is very close to what I'm looking for:
No what is probably happening is that you're looking at a graph with more datapoints than pixels, which forces Graphite to aggregate the datapoints. The default aggregation method is averaging, but you can change it to summing by applying the cumulative() function to your metrics.
Is there any way to get this cumulative() behavior by default?
I've modified my storage-aggregation.conf to use 'aggregationMethod = sum', but I believe this is for historical data and not for data that's displayed in the UI.
When I apply cumulative() everything is perfect, I'm just wondering if there's a way to get this behavior by default.
I'm guessing that even though you've modified your storage-aggregation.conf to use 'aggregationMethod = sum', your metrics you've already created have not changed their aggregationMethod. The rules in storage-aggregation.conf only affect new metrics.
To change your existing metrics to be summed instead of averaged, you'll need to use whisper-resize.py. Or you can delete your existing metrics and they'll be recreated with sum.
Here's an example of what you might need to run:
whisper-resize.py --xFilesFactor=0.0 --aggregationMethod=sum /opt/graphite/storage/whisper/stats_counts/path/to/your/metric.wsp 10s:28d 1m:84d 10m:1y 1h:3y
Make sure to run that as the same user who owns the file, or at least make sure the files have the same ownership when you're done, otherwise they won't be writeable for new data.
Another possibility if you're using statsd is that you're just using metrics under stats instead of stats_counts. From the statsd README:
In the legacy setting rates were recorded under stats.counter_name
directly, whereas the absolute count could be found under
stats_count.counter_name. With disabling the legacy namespacing those
values can be found (with default prefixing) under
stats.counters.counter_name.rate and stats.counters.counter_name.count
now.
Basically, metrics are aggregated differently under the different namespaces when using statsd, and you want stuff under stats_count or stats.counters for things that should be summed.

Working with google maps api

I am trying to build a map based query interface for my website and I am having difficulty finding a starting point besides http://developer.google.com. I assume this is a rather simple task but I feel as though I am on a wild goose chase. Anyway the problem is the existing site places people into a category based on their address (primarily the zip code), this is not working out because of odd shapes and user density so I would like to solve the problem by creating custom zones.
I am not looking for a proprietary solution because I would really like to accomplish this on my own, I just need some better places to start or better suggestions for searches.
I understand that I will need to create a map with my predetermined polygons.
I understand how to create a map with polygons via js.
I do not understand how data will request which zone it is within and how it will return it as a hash I can store. eg. user=>####, zone=>####, section=>#####
http://blog.appdelegateinc.com./point-in-polygon-checking-with-google-maps.html
has some JS you can add to give the ability to test whether a point is within a polygon (sample: http://blog.appdelegateinc.com./static/samples/point_in_polygon.html ) using this approach: http://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm
I think as you place the markers, you'll hold them in an array (of objects)...then loop through, doing some sort of reduction of which polygons to test, testing those that remain, if inPoly, set marker.zone and marker.section to whatever suits your needs

Coloring a map according to mapped database values

Question:
On
http://www.bbc.co.uk/news/10150007
one can see a map of european countries colored according to state debt/deficit.
Now I would have already found it useful several times if I was able to do such a thing myself, for example to visualize regional sales data.
Does anybody know:
Is there any (OpenSource) tool with which I can color a world/continental/reginal map according to colors mapped to values in a database ?
Or any tool that can construct a custom map ?
Or if there is no such thing, how would one do such a thing oneselfs ?
Get the outlines of countries from somewhere, make everything outside the country outlines transparent, set the coordinates and z-indeces to stack several images over one another, and then replace the base color with the selected color in each image, then merge the result to a single picture ?
I normally do this in R. Here are a bunch of examples of how to do this in R.
I also played a bit with QQis, and IIRC it can take input from a postgres PostGIS file.
The canonical commercial tool is ArcView, but it ends up being pricey.
The standard file format for maps is ESRI Shapefiles. These are actually collections of files with the attributes stored in a DBase IV format. Googling for 'shapefile viewer' will get you lots of tools.
There is also mapserver, which allows you to generate maps directly to the web.

Visualisation of Tree Hierarchy in HTML

I am looking of inspiration for doing interaction design on a hierachy/tree structure. (products with a number of subproducts, rules that apply for selecting subproducts).
I want to have a tree where there is a visible connection between sub-nodes and their parent node. And i also want to visualize which rules that apply for selecting them.
Typical rules:
mandatory: select only one of one sub-product
optional: select 0 or more of several subproducts
mutual exclusive: select only one of several subproducts
I hope you get the idea.
I am looking for any inspiration in this area. Any suggestions, examples, tips are welcome
Here are several:
http://thejit.org/
http://www.jsviz.org/blog/
http://vis.stanford.edu/protovis/
If you are willing to use something other than html/javascript, Flare is an excellent library for Adobe Flash.
I've used Infoviz library for such scenario (here's the demo). You could use distinct node colors for different selection rules together with some textual description, although it wouldn't be very intuitive at first.
Default tree orientation is horizontal, which may look odd, but makes sense when you add textual node names of variable length.

Resources