When using Fsharp.Charting for plotting, is it possible to:
set grid interval for DateTime value (format hh:mm:ss, for example on X-axis)?
round X-axis grid value?
I'm not exactly sure what is the configuration that you want to get, but the key point is that F# Charting allows you to set things like grid, ticks and axis properties in various ways - but if your keys are DateTime values, the configuration still has to use float. The numbers to give it are number of days.
Experimentally, it seems that the following two functions work for converting dates and time spans to floats. I'm completely puzzled as to why you need to use 30/12/1899, but it works.
let dtf (d:DateTime) =
(d - DateTime(1899,12,30)).TotalDays
let stf (ts:TimeSpan) =
ts.TotalDays
To specify a grid property you can use:
Chart.Line [
for d in 10 .. 20 ->
DateTime(2022,1,d), sin(float d / 10.0)]
|> Chart.WithXAxis
( Min=dtf(DateTime(2022,1,1)),
Max=dtf(DateTime(2022,1,31)),
MajorGrid=Grid(Interval=stf(TimeSpan(1,0,0,0)), LineColor=Color.Red))
|> Chart.Show
It is also worth noting that FSharp.Charting is pretty much obsolete now and it would be a good idea to use Plotly.NET or something else instead.
Related
I am trying to read a .tif-file in julia as a Floating Point Array. With the FileIO & ImageMagick-Package I am able to do this, but the Array that I get is of the Type Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2}.
I can convert this FixedPoint-Array to Float32-Array by multiplying it with 255 (because UInt8), but I am looking for a function to do this for any type of FixedPointNumber (i.e. reinterpret() or convert()).
using FileIO
# Load the tif
obj = load("test.tif");
typeof(obj)
# Convert to Float32-Array
objNew = real.(obj) .* 255
typeof(objNew)
The output is
julia> using FileIO
julia> obj = load("test.tif");
julia> typeof(obj)
Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2}
julia> objNew = real.(obj) .* 255;
julia> typeof(objNew)
Array{Float32,2}
I have been looking in the docs quite a while and have not found the function with which to convert a given FixedPoint-Array to a FloatingPont-Array without multiplying it with the maximum value of the Integer type.
Thanks for any help.
edit:
I made a small gist to see if the solution by Michael works, and it does. Thanks!
Note:I don't know why, but the real.(obj) .* 255-code does not work (see the gist).
Why not just Float32.()?
using ColorTypes
a = Gray.(convert.(Normed{UInt8,8}, rand(5,6)));
typeof(a)
#Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2}
Float32.(a)
The short answer is indeed the one given by Michael, just use Float32.(a) (for grayscale). Another alternative is channelview(a), which generally performs channel separation thus also stripping the color information from the array. In the latter case you won't get a Float32 array, because your image is stored with 8 bits per pixel, instead you'll get an N0f8 (= FixedPointNumbers.Normed{UInt8,8}). You can read about those numbers here.
Your instinct to multiply by 255 is natural, given how other image-processing frameworks work, but Julia has made some effort to be consistent about "meaning" in ways that are worth taking a moment to think about. For example, in another programming language just changing the numerical precision of an array:
img = uint8(255*rand(10, 10, 3)); % an 8-bit per color channel image
figure; image(img)
imgd = double(img); % convert to double-precision, but don't change the values
figure; image(imgd)
produces the following surprising result:
That second "all white" image represents saturation. In this other language, "5" means two completely different things depending on whether it's stored in memory as a UInt8 vs a Float64. I think it's fair to say that under any normal circumstances, a user of a numerical library would call this a bug, and a very serious one at that, yet somehow many of us have grown to accept this in the context of image processing.
These new types arise because in Julia we've gone to the effort to implement new numerical types (FixedPointNumbers) that act like fractional values (e.g., between 0 and 1) but are stored internally with the same bit pattern as the "corresponding" UInt8 (the one you get by multiplying by 255). This allows us to work with 8-bit data and yet allow values to always be interpreted on a consistent scale (0.0=black, 1.0=white).
I am trying to work with nolearn and use the ConcatLayer to combine multiple inputs. It works great as long as every input has the same type and shape. I have three different types of inputs that will eventually produce a single scalar output value.
The first input is an image of dimensions (288,1001)
The second input is a vector of length 87
The third is a single scalar value
I am using Conv2DLayer(s) on the first input.
The second input utilizes Conv1DLayer or DenseLayer (not sure which would be better since I can't get it far enough to see what happens)
I'm not even sure how the third input should be set up since it is only a single value I want to feed into the network.
The code blows up at the ConcatLayer with:
'Mismatch: input shapes must be the same except in the concatenation axis'
It would be forever grateful if someone could write out a super simple network structure that can take these types of inputs and output a single scalar value. I have been googling all day and simply cannot figure this one out.
The fit function looks like this if it is helpful to know, as you can see I am inputting a dictionary with an item for each type of input:
X = {'base_input': X_base, 'header_input': X_headers, 'time_input':X_time}
net.fit(X, y)
It is hard to properly answer the question, because - it depends.
Without having information on what you are trying to do and what data you are working on, we are playing the guessing game here and thus I have to fall back to giving general tips.
First it is totally reasonable, that ConcatLayer complains. It just does not make a lot of sense to append a scalar to the Pixel values of an Image. So you should think about what you actually want. This is most likely combining the information of the three sources.
You are right by suggesting to process the Image with 2D convolutions and the sequence data with 1D convolutions. If you want to generate a scalar value, you propably want to use dense layers later on, to condense the information.
So it would be naturally, to leave the lowlevel processing of the three branches independent and then concatenate them later on.
Something along the lines of:
Image -> conv -> ... -> conv -> dense -> ... -> dense -> imValues
Timeseries -> conv -> ... -> conv -> dense ... -> dense -> seriesValues
concatLayer([imValues, seriesValues, Scalar] -> dense -> ... -> dense with num_units=1
Another less often reasonable Option would be, to add the Information at the lowlevel processing of the Image. This might make sense, if the local processing is much easier, given the knowledge of the scalar/timeseries.
This architecture might look like:
concatLayer(seriesValues, scalar) -> dense -> ... -> reshape((-1, N, 1, 1))
-> Upscale2DLayer(Image.shape[2:3]) -> globalInformation
concatLayer([globalInformation, Image]) -> 2D conv filtersize=1 -> conv -> ... -> conv
Note that you will almost certainly want to go with the first Option.
One unrelated Thing I noticed, is the huge size of your Input Image. You should reduce it(resizing/patches). Unless you have a gigantic load of data and tons of Memory and computing power, you will otherwise either overfit or waste Hardware.
I am using Images.jl in Julia. I am trying to convert an image into a graph-like data structure (v,w,c) where
v is a node
w is a neighbor and
c is a cost function
I want to give an expensive cost to those neighbors which have not the same color. However, when I load an image each pixel has the following Type RGBA{U8}(1.0,1.0,1.0,1.0), is there any way to convert this into a number like Int64 or Float?
If all you want to do is penalize adjacent pairs that have different color values (no matter how small the difference), I think img[i,j] != img[i+1,j] should be sufficient, and infinitely more performant than calling colordiff.
Images.jl also contains methods, raw and separate, that allow you to "convert" that image into a higher-dimensional array of UInt8. However, for your apparent application this will likely be more of a pain, because you'll have to choose between using a syntax like A[:, i, j] != A[:, i+1, j] (which will allocate memory and have much worse performance) or write out loops and check each color channel manually. Then there's always the slight annoyance of having to special case your code for grayscale and color, wondering what a 3d array really means (is it 3d grayscale or 2d with a color channel?), and wondering whether the color channel is stored as the first or last dimension.
None of these annoyances arise if you just work with the data directly in RGBA format. For a little more background, they are examples of Julia's "immutable" objects, which have at least two advantages. First, they allow you to clearly specify the "meaning" of a certain collection of numbers (in this case, that these 4 numbers represent a color, in a particular colorspace, rather than, say, pressure readings from a sensor)---that means you can write code that isn't forced to make assumptions that it can't enforce. Second, once you learn how to use them, they make your code much prettier all while providing fantastic performance.
The color types are documented here.
Might I recommend converting each pixel to greyscale if all you want is a magnitude difference.
See this answer for a how-to:
Converting RGB to grayscale/intensity
This will give you a single value for intensity that you can then use to compare.
Following #daycaster's suggestion, colordiff from Colors.jl can be used.
colordiff takes two colors as arguments. To use it, you should extract the color part of the pixel with color i.e. colordiff(color(v),color(w)) where v would be RGBA{U8(0.384,0.0,0.0,1.0) value.
So I can't find, for the life of me, a clear breakdown of the components of ExtendScript's UnitValue object. Every source I found had something to do with Adobe, and didn't explain it. I'd really like to have a full reference on it, but if no one can find one, I need at least a few questions answered concerning it.
First, what are its constructors? I've seen UnitValue(10,'px') which makes sense, but I've also seen UnitValue(20,20)
Second, how can you convert from one unit to another?
Third, how can you find its value and its unit?
I think I've seen uv.value but nothing getting the unitsLastly, as my tags indicate, this is for Adobe, of course, since I've never seen or heard of any other program that uses ExtendScript.
UnitValue is documented in the Adobe JavaScript Tools Guide.
In particular, you create an object with v = new UnitValue(number, unit), where unit is a string value like in (inch), mm (millimeter), pt (point), px (pixel), etc.
To convert a UnitValue to an actual number, use the as method, e.g. v.as("cm") to convert v into centimeters.
Well, no one else seemed to know, and I finally figured some of it out, so I guess I'll answer it myself:
<This site> was a little helpful as a documentation, but I think Adobe functions slightly different from it.
UnitValue's main constructor is:
UnitValue(numericalvalue,unit)
I've also seen an alternative that accepts one string:
UnitValue("42 in")
For conversion, UnitValue comes with a handy as method which accepts the unit to convert to (as a string), and then returns its measurement in that unit, i.e.:
UnitValue(5,'ft').as('in') === 60
(Note, according to the reference I found, I believe the as method should return the UnitValue instance after being converted to the unit indicated; Adobe's implementation, however, seems to instead merely return the value - therefore I'm use the type-equality operator above to show Adobe's method)
For getting the numerical value and measurement unit, the following two properties exist:
UnitValue.value // number: the numerical value
UnitValue.type // string: the unit of measurement
This is all I could find by my research. If someone has a better answer, post it, and I may accept it.
What's really interesting to me is the baseValue property which allows us to change the frame of reference for the conversion. For instance:
var startUnit = UnitValue(500, "px");
startUnit.baseValue = UnitValue(1 / 72, "in"); // from what I can tell this is the default baseUnit value
$.writeln(v.as("in")); // returns 6.94444444444444 which is what 500px # 72 dpi is as expressed in inches
startUnit.baseValue = UnitValue(1 / 300, "in"); // set to 300dpi
$.writeln(v.as("in")); // returns 1.66666666666667 which is what 500px # 300 dpi is as expressed in inches
I think the default baseValue is always UnitValue(1/72, "in") even if the app.preferences.rulerUnits is set to any other type of measurement but I haven't really looked into it much
I have a some vector data that has been manually created, it is just a list of x,y values. The coordinate of the points is not perfectly accurate - it can be off by a few pixels and it won't make any perceivable difference.
So now I am looking for some way to watermark this data, so that if someone steal the vector data, I can prove that it's indeed been stolen. I'm looking for some method reliable enough that even if someone take my data and shift all the points by a some small amount, I can still prove that it's been stolen.
Is there any way to do that? I know it exists for bitmap data but how about vector data?
PS: the vector graphic itself is rather random - it cannot be copyrighted.
Is the set of points all you can work with? If, for example, you were dealing with SVG, you could export the file with a certain type of XML formatting, a <!-- generated by thingummy --> comment at the top, IDs generated according to such-and-such a pattern, extra attributes specifically yours, a particular style of applying translations, etc. Just like you can work out from a JPEG what is likely to have been used to create it, you can tell a lot about what produced an SVG file by observation.
On the vectors themselves, you could do something like consider them as an ordered sequence and apply offsets given by the values of two pseudo-random sequences, each starting from a known seed, for X and Y translation, in a certain range (such as [-1, 1]). Even if some points are modified, you should be able to build up an argument from how things match the sequence. How to distinguish precisely what has been shifted could do with a bit more consideration, too; if you were simply doing int(x) + random(-1, 1), then if someone just rounded all values your evidence would be lost. A better way of dealing with this would be to, while still rendering at the same screen size, multiply everything by some constant like 953 (an arbitrary near-1000 prime) and then adjust your values by something in that range (viz, [0, 952]). This base-953 system would be superior to a base-10 system because it's much (much much) harder to see what's happening. If the person changes the scaling, it would require a bit more analysis of values, but it should still be quite possible. I've got a gut feeling that that's where picking a prime number could be a bit helpful, but I haven't thought about it terribly much. If in danger or in doubt in such matters, pick a prime number for the sake of it... you may find out later there are benefits to it!
Combine a number of different techniques for best results, of course.