In A2C algorithm, multiple environments are run in different processes in order to parallelize computation.
In openai baselines implementation, SubprocVecEnv is the class that creates different processes for the different environments. (You can see almost the same implementation in stable_baselines).
The point I don't understand is: why is the argument to the processes a function that returns the environment called env_fn and not directly the environment itself called env?
In other words, why is L61 like this, and not like this:
self.ps = [ctx.Process(target=worker, args=(work_remote, remote, CloudpickleWrapper(env)))
Thanks!
Because to create multiple environments you need the function to create them. If you pass the environment itself to it, it's a pass by reference to an instance of the environment and all workers would operate on the same instance.
The function on the other hand allows to create multiple instances so each process can operate on an instance.
Related
To use OpenCL kernel the following is needed:
Put the kernel code in a string
call clCreateProgramWithSource
call clBuildProgram
call clCreateKernel
call clSetKernelArg (x number of arguments)
call clEnqueueNDRangeKernel
This need to be done for each kernel. Is there a way to do this repeating less code for each kernel?
There is no way to speed up the process. You need to go step by step as you listed.
But it is important to know why it is needed these steps, to understand how flexible the chain is.
clCreateProgramWithSource: Allows to add different strings from different sources to generate the program. Some string might be static, but some might be downloaded from a server, or loaded from disk. It allows the CL code to be dynamic and updated over time.
clBuildProgram: Builds the program for a given device. Maybe you have 8 devices, so you need to call this multiple times. Each device will produce a different binary code.
clCreateKernel: Creates a kernel. But a kernel is an entry point in a binary. So it is possible you create multiple kernels from a program (for different functions). Also the same kernel might be created multiple times, since it holds the arguments. This is useful for having ready-to-be-launched instances with proper parameters.
clSetKernelArg: Changes the parameters in the instance of the kernel. (it is stored there, so it can used multiple times in the future).
clEnqueueNDRangeKernel: Launches it, configuring the size of the launch and the chain of dependencies with other operations.
So, even if you could have a way to just call "getKernelFromString()", the functionality will be very limited, and not very flexible.
You can have look at wrapper libraries
https://streamhpc.com/knowledge/for-developers/opencl-wrappers/
I suggest you look into SYCL. The building steps are performed offline, saving execution time by skipping the clCreateProgramWithSource. The argument setting is done automatically by the runtime, extracting the information from the user lambda
There is also CLU: https://github.com/Computing-Language-Utility/CLU - see https://www.khronos.org/assets/uploads/developers/library/2012-siggraph-opencl-bof/OpenCL-CLU-and-Intel-SIGGRAPH_Aug12.pdf for more info. It is a very simple tool, but should make life a bit easier.
I have written a ColdFusion UDF which calls itself recursively. What I would like to know is whether local/var scoped variables set in a parent call to the function are available in the child function or whether they are only available in the function call where they were set.
I am away from my workstation for a while so am unable to test for myself so wondered if anybody already knew the answer to this.
Thanks!
Function-local variables are local to the current call to that function. Each function call has its own memory space, and function local variables reside in that memory space. Recursive calls are no different in this regard.
As far as I know, JavaScript is no different here: I'd like to see an example that bears out your assertion that it's different.
I am using parLapply function on 20 cores. I guess it is the same for the other function parSapply etc...
First, is it a bad practice to pass a cluster as an argument into a function so that the function can then dispatch the use of the cluster between different subfunctions?
Second, I pass this cluster argument into a function so i suppose it is the same cluster everytime i use parLapply, would it be better to use a new cluster for every parLapply call?
Thanks
Rgds
I am not an expert in parallel computing but will venture an answer anyway.
1) It is not bad practice to pass the cluster around as a function argument. It is just a collection of connections to worker processes, similar to connections to a file.
2) Restarting the cluster between calls is not needed. There will be problems if something has gone seriously wrong with a worker process, but in that case I would recommend cancelling the whole computation and restarting the master process too.
It's not bad practice to pass cluster objects as function arguments, so I don't see anything wrong with using them to dispatch between different sub-functions.
The problem with creating cluster objects for each operation is that it may take significant time, especially when starting many workers via ssh on a cluster, for example. However, it may be useful in some cases, and I think that that may be the purpose of Fork clusters, which are created by the makeForkCluster function in the parallel package. Fork cluster workers are very much like the workers forked by mclapply, which does create workers every time it's called. However, I think this is rarely done.
This is a basic R question: R has the concept of environment. So what purpose does it have, when do I need to start more then one and how do I switch between them? What is the advantage of multiple environments (other then looking up content of .Rdata file)?
The idea of environments is important and you use them all the time, mostly without realizing it. If you are just using R and not doing anything fancy then the indirect use of environments is all that you need and you will not need to explicitly create and manipulate environments. Only when you get into more advanced usage will you need to understand more. The main place that you use (indirectly) environments is that every function has its own environment, so every time you run a function you are using new envirnments. Why this is important is because this means that if the function uses a variable named "x" and you have a variable named "x" then the computer can keep them straight and use the correct one when it needs to and your copy of "x" does not get over written by the functions version.
Some other cases where you might use environments: Each package has its own environment so 2 packages can both be loaded with the same name of an internal function and they won't interfere with each other. You can keep your workspace a little cleaner by attaching a new enironment and loading function definitions into that environment rather than the global or working environment. When you write your own functions and you want to share variables between functions then you will need to understand about environments. Environmets can be used to emulate pass-by-reference instead of pass-by-value if you are ever in a situation where that matters (if you don't recognize those phrases then it probably does not matter).
You can think of environments as unordered lists. Both datatypes offer something like the hash table data structure to the user, i.e., a mapping from names to values. The lack of ordering in environments offers better performance when compared with lists on similar tasks.
The access functions [[ and $ work for both.
A nice fact about environments which is not true for lists is that environments pass by reference when supplied as function arguments, offering a way to improve performance when working large objects.
Personally, I never work directly with environments. Instead, I divide my scripts up in functions. This leads to an increased reusability, and to more structure. In addition, each function runs in its own environment, ensuring minimum interference in variables etc.
This is more of a beginner's question. Say I have the following code:
library("multicore")
library("iterators")
library("foreach")
library("doMC")
registerDoMC(16)
foreach(i in 1:M) %dopar% {
##do stuff
}
This code then will run on 16 cores, if they are available. Now if I understand correctly, using Amazon EC2, on one instance, I get depending on the instance only few cores. So if I want to run simulations on 16 cores, I need to use several instances, which means as I far as I understand launching new R processes. But then I need to write additional code outside of R to gather the results.
So my question is, is there an R package, which lets to launch EC2 instances from within R, automagicaly distributes the load between these instances, and gathers the results in the initial R launched?
To be precise, the maximum instance type on EC2 is currently 8 cores, so anyone, even users of R, would need multiple instances in order to have run concurrently on more than 8 cores.
If you want to use more instances, then you have two options for deploying R: "regular" R invocations or MapReduce invocations. In the former case, you will have to set up code to launch instances, distribute tasks (e.g. the independent iterations in foreach), return results, etc. This is doable, but you're not likely to enjoy it. In this case, you can use something like rmr or RHipe to manage a MapReduce grid, or you can use snow and many other HPC tools to create a simple grid. Use of snow may make it easier to keep your code intact, but you will have to learn how to tie this stuff together.
In the latter case, you can build upon infrastructure that Amazon has provided, such as Elastic MapReduce (EMR) and packages that make that simpler, such as JD's segue. I'd recommend segue as a good starting point, as others have done, as it has a gentler learning curve. The developer is also on SO, so you can easily embarrass query him when it breaks.