I was trying to train a convolution network. But it is not improving, i.e. loss is not decreasing. And the train function is also terminating much more quickly than usual. Below is the minimal code to show the problem.
using Flux
data=rand(200, 100, 1, 50)
label=rand([0.0,1.0], 1, 50)
model=Chain(
Conv((3,3), 1=>5, pad=(1,1)),
MaxPool((2,2)),
Conv((3,3), 5=>5, pad=(1,1)),
MaxPool((2,2)),
Conv((3,3), 5=>5, pad=(1,1)),
MaxPool((2,2)),
x->reshape(x, :, size(x, 4)),
x->σ.(x),
Dense(1500,100),
Dense(100,1)
)
model(data)
loss=Flux.mse
opt=Descent(0.1)
param=params(model)
loss(model(data), label) #=>0.3492440767136241
Flux.train!(loss, param, zip(data, label), opt)
loss(model(data), label) #=>0.3492440767136241
The first argument to Flux.train! needs to be function which accepts the data, runs the model, and returns the loss. Its loop looks something like this:
for dtup in zip(data, label)
gradient(() -> loss(dtup...), params)
...
end
But the function loss you provide doesn't call the model at all, it just compares the data point to the label directly.
There is more to fix here though. What's being iterated over is tuples of numbers, starting with zip(data, label) |> first, which I don't think is what you want. Maybe you wanted Flux.DataLoader to iterate batches of images?
Related
Here's a minimal working example.
using Flux
myvector = []
for i in 1:3
push!(myvector, 0.1 * i) #Produces a vector [0.1, 0.2, 0.3]
end
Q = Chain(Dense(3, 64, relu), Dense(64, 32, relu), Dense(32, 16, relu),
Dense(16, 1, identity)) #Creates a neural network. Exact parameters aren't important.
θ, re = Flux.destructure(Q)
f(x) = -re(θ)(x)[1]
gs = gradient(f, myvector)[1] #Takes the gradient of f at the point u.
The code creates a vector myvector and a neural network Q. It then feeds the neural network myvector and tries to take the gradient. There is a mutating arrays error:
Mutating arrays is not supported -- called setindex!(Vector{Any}, ...)
This error occurs when you ask Zygote to differentiate operations that change
the elements of arrays in place (e.g. setting values with x .= ...)
Why is there such an error? Nothing seems to be changing the elements of the array. How can I fix this error?
If instead of using !push to define the vector I had simply defined myvector = [0.1, 0.2, 0.3], then there is no error. However, I don't see why - the !push doesn't modify the elements, it just increases the length of myvector. For my use case as well, the elements of the vector are complicated and need to be calculated individually, so I can't just define myvector as in this MWE.
There is also no error if I define myvector = AbstractFloat[]. However, this seems to cause issues elsewhere in my use case (which I will ask about separately if making myvector an AbstractFloat is the best solution).
Your issue is related to your variable, myvector. You made a Vector{Any}, which is not supported by Zygote. You could make a Vector{Float64} like this: myvector=[0.1 * i for i in 1:3]. In most cases, Float32 is needed for machine learning, myvector=[0.1 * i for i in 1:3].|>Float32
If you really want to use push!(), I would suggest using myvector = Float64[] instead of myvector = AbstractFloat[].
using Plots
p_arr = Array{Plots.Plot{Plots.GRBackend}}(undef,10,10);
x=5;
y=10;
p_arr[1,1] = scatter!([x],[y],markersize=5,legend=false,markercolor = :green, showaxis =
false,size=(500,500));
p_arr[1,2] = scatter!([x],[y],markersize=5,legend=false,markercolor = :green, showaxis =
false,size=(500,500));
this is a very simple code of storing a point plot into an array. I just want to know how to change the x and y coordination for this point by the object it is stored in ?. in other words, I want to set x and y values by the object itself.
is there a better way to do this.
I am new to julia and I do not know where to search
While I'm not quite sure what you'll need for your end use case, storing an array of Plots.jl Plots.Plots has a lot of overhead and will not make it at all easy to modify the underlying points.
One approach that could be dramatically simpler would be to work directly with an array of Points instead. For example, let us first define:
struct Point{T}
x::T
y::T
end
then you have a type which can represent just an x-y point by itself. You can make an array of them:
p_arr = Array{Point{Int64}}(undef, 10, 10) # uninitialized, Int64
or to make this a little more interesting, an array of random points:
julia> p_arr = Point.(rand(10,10), rand(10,10))
10×10 Matrix{Point{Float64}}:
Point{Float64}(0.561232, 0.39038) … Point{Float64}(0.0564275, 0.851144)
⋮ ⋱
Point{Float64}(0.804435, 0.0717767) Point{Float64}(0.110489, 0.670536)
If you want to be able to plot these, then let's define some functions to let Plots.jl know how to plot our Point type:
using Plots
import Plots.scatter
scatter(p::Point, args...; kwargs...) = scatter([p.x], [p.y], args...; kwargs...)
scatter(p::AbstractArray{<:Point}, args...; kwargs...) = scatter(map(p->p.x, p), map(p->p.y, p), args...; kwargs...)
# You can do the same for `scatter!` if you want that version too
then you can write
scatter(p_arr, label="", framestyle=:box)
to obtain a plot that looks like:
(note that each column gets its own series in Plots.jl, hence the multiple colors; call vec on your matrix of Points first if you don't want that)
Now, say you want to move one of those points. Let's say the first one:
julia> p_arr[1,1] = Point(5.0, 10.0)
Point{Float64}(5.0, 10.0)
Then plotting again
scatter(p_arr, label="", framestyle=:box)
will yield
newbie need help again.I'm playing around a dataset with UMAP, a dimension reduction tool. Things like this will have 2 parameters that need to tune and look. Previously I have used tSNE, and it requires one parameter tuning. For tSNE, the parameter is called perplexity. To trial a few values for perplexity and visualise the result, I think the map function in purrr works great to automate this.
#for this purpose the sample data can be anything
#only that my dataset has lots labels
df <- data.frame(replicate(110,sample(-10:10,1000,rep=TRUE)))
df.label <- df[,1:20]
df.data <- df[,21:110]
library(tsne)
library(purrr)
#set the test values for perplexity a vector
#map along a vector
perplex=c(10,20,50,100)
map(perplex,tsne(df.data,perplexity = perplex))
The result of tense() will generate a x/y coordinate for each row(sample) then I can plot them.
Although, a little help here to teach me how to automatically map out all 4 test results will be awesome, otherwise I have to use plot 4 times, each with x=tsne[,1] and y=tsne[,2].
Now, for the umap that I want to test. I want to test 2 parameters, n_neighbors and min_dist the same way. And the complexity is for each value I pick for n_neighbors, I want to test all min_dist test values. For example if :
n_neighbors= 10,50,20
min_dist= 0.1, 0.5, 1, 10
I want to run the umap function on my data for n_neighbors=10, and iterate min_dist= 0.1, 0.5, 1, 10. And repeat this for the rest of n_neighbors value.
Then I'm stuck with the map function in purrr. I think I can only pass 1 vector in the function.
#map along a vector
n_neighbors.test= c(10,50,20)
min_dist.test= c(0.1, 0.5, 1, 10)
map(?,umap(df.data,n_neighbors = n_neighbors.test, min_dist=min_dist.test ))
and then also the plotting issue. UMAP also gives a list, one matrix is the layout that contains x/y coordinates of the rows.
Try :
expand.grid(n_neighbors.test,n_neighbors) %>% transpose() %>% map(~{umap(df.data,n_neighbors = .x[[1]], min_dist=.x[[2]] )})
Alternatively, you can use imbricated maps:
unlist(map(n_neighbors.test,function(x){
map(min_dist.test,function(y){umap(df.data,x,y)})
}))
Background
I read here that newton method fails on function x^(1/3) when it's inital step is 1. I am tring to test it in julia jupyter notebook.
I want to print a plot of function x^(1/3)
then I want to run code
f = x->x^(1/3)
D(f) = x->ForwardDiff.derivative(f, float(x))
x = find_zero((f, D(f)),1, Roots.Newton(),verbose=true)
Problem:
How to print chart of function x^(1/3) in range eg.(-1,1)
I tried
f = x->x^(1/3)
plot(f,-1,1)
I got
I changed code to
f = x->(x+0im)^(1/3)
plot(f,-1,1)
I got
I want my plot to look like a plot of x^(1/3) in google
However I can not print more than a half of it
That's because x^(1/3) does not always return a real (as in numbers) result or the real cube root of x. For negative numbers, the exponentiation function with some powers like (1/3 or 1.254 and I suppose all non-integers) will return a Complex. For type-stability requirements in Julia, this operation applied to a negative Real gives a DomainError. This behavior is also noted in Frequently Asked Questions section of Julia manual.
julia> (-1)^(1/3)
ERROR: DomainError with -1.0:
Exponentiation yielding a complex result requires a complex argument.
Replace x^y with (x+0im)^y, Complex(x)^y, or similar.
julia> Complex(-1)^(1/3)
0.5 + 0.8660254037844386im
Note that The behavior of returning a complex number for exponentiation of negative values is not really different than, say, MATLAB's behavior
>>> (-1)^(1/3)
ans =
0.5000 + 0.8660i
What you want, however, is to plot the real cube root.
You can go with
plot(x -> x < 0 ? -(-x)^(1//3) : x^(1//3), -1, 1)
to enforce real cube root or use the built-in cbrt function for that instead.
plot(cbrt, -1, 1)
It also has an alias ∛.
plot(∛, -1, 1)
F(x) is an odd function, you just use [0 1] as input variable.
The plot on [-1 0] is deducted as follow
The code is below
import numpy as np
import matplotlib.pyplot as plt
# Function f
f = lambda x: x**(1/3)
fig, ax = plt.subplots()
x1 = np.linspace(0, 1, num = 100)
x2 = np.linspace(-1, 0, num = 100)
ax.plot(x1, f(x1))
ax.plot(x2, -f(x1[::-1]))
ax.axhline(y=0, color='k')
ax.axvline(x=0, color='k')
plt.show()
Plot
That Google plot makes no sense to me. For x > 0 it's ok, but for negative values of x the correct result is complex, and the Google plot appears to be showing the negative of the absolute value, which is strange.
Below you can see the output from Matlab, which is less fussy about types than Julia. As you can see it does not agree with your plot.
From the plot you can see that positive x values give a real-valued answer, while negative x give a complex-valued answer. The reason Julia errors for negative inputs, is that they are very concerned with type stability. Having the output type of a function depend on the input value would cause a type instability, which harms performance. This is less of a concern for Matlab or Python, etc.
If you want a plot similar the above in Julia, you can define your function like this:
f = x -> sign(x) * abs(complex(x)^(1/3))
Edit: Actually, a better and faster version is
f = x -> sign(x) * abs(x)^(1/3)
Yeah, it looks awkward, but that's because you want a really strange plot, which imho makes no sense for the function x^(1/3).
Gauss function has an infinite number of jump discontinuities at x = 1/n, for positive integers.
I want to draw diagram of Gauss function.
Using Maxima cas I can draw it with simple command :
f(x):= 1/x - floor(1/x); plot2d(f(x),[x,0,1]);
but the result is not good ( near x=0 it should be like here)
Also Maxima claims:
plot2d: expression evaluates to non-numeric value somewhere in plotting
range.
I can define picewise function ( jump discontinuities at x = 1/n, for positive integers )
so I tried :
define( g(x), for i:2 thru 20 step 1 do if (x=i) then x else (1/x) - floor(1/x));
but it don't works.
I can also use chebyshew polynomials to aproximate function ( like in : A Graduate Introduction to Numerical Methods From the Viewpoint of Backward Error Analysis by Corless, Robert, Fillion, Nicolas)
How to do it properly ?
For plot2d you can set the adapt_depth and nticks parameters. The default values are 5 and 29, respectively. set_plot_option() (i.e. with no argument) returns the current list of option values. If you increase adapt_depth and/or nticks, then plot2d will use more points for plotting. Perhaps that makes the figure look good enough.
Another way is to use the draw2d function (in the draw package) and explicitly tell it to plot each segment. We know that there are discontinuities at 1/k, for k = 1, 2, 3, .... We have to decide how many segments to plot. Let's say 20.
(%i6) load (draw) $
(%i7) f(x):= 1/x - floor(1/x) $
(%i8) makelist (explicit (f, x, 1/(k + 1), 1/k), k, 1, 20);
(%o8) [explicit(f,x,1/2,1),explicit(f,x,1/3,1/2),
explicit(f,x,1/4,1/3),explicit(f,x,1/5,1/4),
explicit(f,x,1/6,1/5),explicit(f,x,1/7,1/6),
explicit(f,x,1/8,1/7),explicit(f,x,1/9,1/8),
explicit(f,x,1/10,1/9),explicit(f,x,1/11,1/10),
explicit(f,x,1/12,1/11),explicit(f,x,1/13,1/12),
explicit(f,x,1/14,1/13),explicit(f,x,1/15,1/14),
explicit(f,x,1/16,1/15),explicit(f,x,1/17,1/16),
explicit(f,x,1/18,1/17),explicit(f,x,1/19,1/18),
explicit(f,x,1/20,1/19),explicit(f,x,1/21,1/20)]
(%i9) apply (draw2d, %);
I have made a list of segments with ending points. The result is :
and full code is here
Edit: smaller size with shorter lists in case of almost straight lines,
if (n>20) then iMax:10 else iMax : 250,
in the GivePart function