I'm trying to adapt some cleanrl code to a robot learning environment. Cleanrl expects the observation space to have a "shape" property, but many simulated robot environments use dictionary observations. Is there a simple way to tell pytorch's SyncVectorEnv how to unwrap a dictionary observation into one it can use?
For simple cases, I guess the best way would be to concatenate the observations before SyncVectorEnv, using a custom wrapper. There is a convenient way to do it with Gym:
class DictConcatWrapper(gym.ObservationWrapper):
def observation(self, obs):
# (if dict, concatenate its elements here...)
return obs
and later:
env = ...
env = DictConcatWrapper(env)
env = SyncVectorEnv(env)
Related
I have the following function: problema_firma_emprestimo(r,w,r_emprestimo,posicao,posicao_banco), where all input are scalars.
This function return three different matrix, using
return demanda_k_emprestimo,demanda_l_emprestimo,lucro_emprestimo
I need to run this function for a series of values of posicao_banco that are stored in a vector.
I'm doing this using a for loop, because I need three separate matrix with each of them storing one of the three outputs of the function, and the first dimension of each matrix corresponds to the index of posicao_banco. My code for this part is:
demanda_k_emprestimo = zeros(num_bancos,na,ny);
demanda_l_emprestimo = similar(demanda_k_emprestimo);
lucro_emprestimo = similar(demanda_k_emprestimo);
for i in eachindex(posicao_bancos)
demanda_k_emprestimo[i,:,:] , demanda_l_emprestimo[i,:,:] , lucro_emprestimo[i,:,:] = problema_firma_emprestimo(r,w,r_emprestimo[i],posicao,posicao_bancos[i]);
end
Is there a fast and clean way of doing this using vectorized functions? Something like problema_firma_emprestimo.(r,w,r_emprestimo[i],posicao,posicao_bancos) ? When I do this, I got a tuple with the result, but I can't find a good way of unpacking the answer.
Thanks!
Unfortunately, it's not easy to use broadcasting here, since then you will end up with output that is an array of tuples, instead of a tuple of arrays. I think a loop is a very good approach, and has no performance penalty compared to broadcasting.
I would suggest, however, that you organize your output array dimensions differently, so that i indexes into the last dimension instead of the first:
for i in eachindex(posicao_bancos)
demanda_k_emprestimo[:, :, i] , ...
end
This is because Julia arrays are column major, and this way the output values are filled into the output arrays in the most efficient way. You could also consider making the output arrays into vectors of matrices, instead of 3D arrays.
On a side note: since you are (or should be) creating an MWE for the sake of the people answering, it would be better if you used shorter and less confusing variable names. In particular for people who don't understand Portuguese (I'm guessing), your variable names are super long, confusing and make the code visually dense. Telling the difference between demanda_k_emprestimo and demanda_l_emprestimo at a glance is hard. The meaning of the variables are not important either, so it's better to just call them A and B or X and Y, and the functions foo or something.
It could be a very easy question, given that I am very unfamiliar with R. I know normally one can use deparse(substitute(.)) to extract the name of a variable. However, if I have a long list of variables (let's say it's built without names), how can I extract the name of each variable efficiently? I was thinking about using loops, but the deparse(substitute(.)) method would obviously generate the 'general' variable name we used to denote every item.
Sample code:
countries<-
list(austria,belgium,czech,denmark,france,germany,italy,luxemberg,netherlands,poland,swiss)
Suppose I want to get countryNames equals to list("austria","belgium",...,"swiss"), how shall I code? I tried generating the list using countries <- list(countryA = countryA, countryB = countryB, ...), but it was extremely tedious, and in some cases I might only have an unnamed input list from elsewhere.
countries would just have values of each individual objects (austria,belgium etc.). To access the names you need to create a named list while creating countries which can be done like :
countries <- list(austria = austria,belgium = belgium....)
However, if this is very tedious you can use tibble::lst which creates the names automatically without explicitly mentioning them.
countries <- tibble::lst(austria,belgium....)
In both the case you can access the names using names(countries).
If the country objects are the only ones loaded in the global environment, we can do this easily with ls and mget to return a named list of values
countries <- mget(ls())
This function is OK in R:
f <- function(x) {
x + y
}
Because if the variable y is not defined inside the function f(), R will look for it outside the environment of the function, in its parent environment.
Apart from the fact that this behavior can be a bug generator, what is the point of functions having input parameters? Anyway, all the variables inside a function can be searched outside of it.
Is there any way not to look for variables outside a function if they do not exist within the function?
Some reasons for using parameters that came to my mind:
Without parameters, users have to define variables before using the function, and these variable names need to match the variable names used within the function -- this is impractical.
How is anyone supposed to know/remember the names of the variables within a function? How do I know which variables within a function are purely local, and which variables have to exist outside of the function?
Input parameters can be passed directly as values or as a variable (and the variable name does not matter).
Input parameters communicate the intended usage of the function; it is clear what data is needed to operate it (or at the very least: how many values need to be inserted by the user of the function)
Input parameters can be documented properly using Rd files (or roxygen syntax)
I am sure there are many other reasons to use input parameters.
M. Papenberg provides a very good explanation.
Here's a quick addendum how to make a function not look for objects in parental environments:
Just provide them in the parameter list! This might sound stupid, but that's what you should always do unless you have good reason to do otherwise. In your example only x is passed to the function. So, if the idea here is that x should be returned if y doesn't exist, you can go for default parameters. In this case this could be done as
f <- function(x, y = 0) {
x + y
}
I store important metadata in R objects as attributes. I want to migrate my workflow to Julia and I am looking for a way to represent at least temporarily the attributes as something accessible by Julia. Then I can start thinking about extending the RData package to fill this data structure with actual objects' attributes.
I understand, that annotating with things like label or unit in DataFrame - I think the most important use for object' attributes - is probably going to be implemented in the DataFrames package some time (https://github.com/JuliaData/DataFrames.jl/issues/35). But I am asking about about more general solution, that doesn't depend on this specific use case.
For anyone interested, here is a related discussion in the RData package
In Julia it is ideomatic to define your own types - you'd simply make fields in the type to store the attributes. In R, the nice thing about storing things as attributes is that they don't affect how the type dispatches - e.g. adding metadata to a Vector doesn't make it stop behaving like a Vector. In julia, that approach is a little more complicated - you'd have to define the AbstractVector interface for your type https://docs.julialang.org/en/latest/manual/interfaces/#man-interface-array-1 to have it behave like a Vector.
In essence, this means that the workflow solutions are a little different - e.g. often the attribute metadata in R is used to associate metadata to an object when it's returned from a function. An easy way to do something similar in Julia is to have the function return a tuple and assign the result to a tuple:
function ex()
res = rand(5)
met = "uniformly distributed random numbers"
res, met
end
result, metadata = ex()
I don't think there are plans to implement attributes like in R.
I know that rm(list=ls()) can delete all objects in the current environment.
However, environment has three categories: Data, Values, Functions. I wonder how I can only delete all the objects in one particular category? Something like
rm(list=ls(type="Values"))
You could use ls.str to specify a mode, or lsf.str for functions. The functions have print methods that make it look otherwise, but underneath are just vectors of object names, so
rm(list = lsf.str())
will remove all user-defined functions, and
rm(list = ls.str(mode = 'numeric'))
will remove all numeric vectors (including matrices). mode doesn't correspond exactly to class, though, so there's no way to distinguish between lists and data.frames with this method.
One option is that you can change the view to grid view and check all the boxes next to the ones you want to delete and click the broom button.
So far as I'm aware, Data, Values and Functions are terms used by the RStudio interface. Data = variables with dimensions e.g. data frames, matrices, Values = other variables (e.g. vectors). They are not terms that can be accessed via R code.