Right now, I am using Horovod to run distributed training of my pytorch models. I would like to start using hydra config for the --multirun feature and enqueue all jobs with SLURM. I know there is the Submitid plugin. But I am not sure, how would the whole pipeline work with Horovod. Right now, my command for training looks as follows:
CUDA_VISIBLE_DEVICES=2,3 horovodrun -np 2 python training_script.py \
--batch_size 30 \
...
Say I want to use hydra --multirun to run several multi-gpu experiments, I want to enqueue the runs with slurm since my resources are limited and would be run sequentially most of the time and I want to use Horovod to synchronize gradients of my networks. Would this setup run out of the box? Would I need to specify CUDA_VISIBLE_DEVICES if slurm took care of the resources? How would I need to adjust my run command or other settings to make this setup plausible? I am especially interested in how the multirun feature handles GPU resources. Any recommendations are welcome.
The Submitit plugin does support GPU allocation, but I am not familiar with Horovod and have no idea if this can work in conjunction with it.
One new feature of Hydra 1.0 is the ability to set or copy environment variables from the launching process.
This might come in handy in case Horovod is trying to set some environment variables. See the docs for info about it.
Related
I don't know what is a merit of a terminal multiplexer like screen or tmux compared to combination of a standard terminal application and a job control feature of a shell.
Typically good features of a terminal multiplexer are cited as follows:
persistance
multiple windows
session sharing
First two features are, however, achieved with a terminal application like iTerm2 and a job control feature of a shell like bash.
Session sharing is a novel feature but it seems to be required in a quite rare situation.
What is a merit of terminal multiplexer? Why do you use it?
I'm interested especially in its merit in daily task.
I can tell you from my perspective as a devops/developer.
Almost every day I have to deploy a bunch of apps (a particular version) on multiple servers. Handling that without something like Terminator or Tmux would be a real pain.
On a single window I can put something like 4 panes (four windows in one) and monitor stuff on 4 different servers...which by it self is a huge deal...without tabs or other terminal instances and what not....
On the first pane I can shutdown nginx, on the second server I can shut down all the processes with supervisord (process manager), and on the third pane I can do the deploy process...if I quickly need to jump to some other server I just use the fourth pane...
Some colleagues that only use a bunch of terminal instances can get really confused when they have to do a bunch of things quickly, constantly ssh-ing in and out ...and if they are not careful they can go to the wrong server because they switched to the wrong terminal instance and entered a command that wasn't meant for that particular server :)...
A terminal multiplexer like Tmux really does help me to be quick and accurate.
There is an package manager for Tmux, which lets you install plugins and really supercharge you terminal even more!
On a different note, a lot of people are using Tmux in combination with Vim...which lets you create some awesome things together...
All in all, those were my two cents on the benefit of using a terminal multiplexer...
is there any way to play inform7 from the command line? I'd like to write some automated test script that plays the play with certain commands and don't want to do it manually. Is there any way to do that?
This is easiest to do with the CLI Linux package of Inform 7. It contains a perl CLI script you can run, but you may also like to consider this alternative script I wrote: https://github.com/i7/kerkerkruip/blob/master/tools/build-i7-project
You can invoke this with
build-i7-project -p "Project Folder"
(Leave off the .inform.)
You can also run the binaries which are installed with the IDE packages by themselves instead of installing the CLI Linux package. The command line options are probably mostly the same in other operating systems, but you may need to change them slightly. If you can't get it to work, compare with what the Inform 7 IDE says when you build with it.
If what you really want to do is periodically run some test scripts that verify that your work is still performing as expected, then Inform 7 has the capability do do that from within the IDE. Take a look at chapter 24.2 of Writing with Inform for details. In combination with good use of the Skein, this should handle the more common unit-testing requirements.
Of course, if you're doing something more outré, running bash scripts from the command line may wind up being the way to go. Still, don't do any more work than you have to. :)
I want to know that if a shell script is created and tested on ubuntu kernel, then will it always without fail also run on RHEL kernel provided the correct shell is invoked for running the script.
Ways in which the execution may differ when used on different distributions and different kernels:
Differences in the version and configuration of the Linux kernel - this may affect presence and format of the contents of files such as those in /proc and /sys, or the presence of particular device drivers.
Differences in the version of the shell used - /bin/sh may be Bash on one system and Dash on another, or Bash 3.x on one system and Bash 4.x on the other.
Differences in the installed programs your script invokes (and, if you got your package dependencies wrong, whether those programs are even present - what's "essential" on one distribution may be "optional" on another).
In short, different distributions have the same issues as different versions of one distribution, but more so.
It depends on what shell/interpreter it was written for and versions of the particular shell it was written for. For example, a bash script written using bash-4.4 may not work in bash-2.0 and so on. It's not quite related to to the distribution/kernel version you use but the shell you use.
So, without details, it's not possible to assert whether a script that works on Ubuntu will work on RHEL. If you use the same shell and same version
on both machines then yes, it's going to work as expected (barring some very odd cases).
We have set of automated test cases around 2000 and we need to run them daily on every new build that come up. It's currently taking 4 hours to finish the test on one machine. In order reduce this we planning to run tests in batches (500 per batch) on same machine by initiating multiple browsers of same type. Say 4 firefox browser sessions per test suite. So it can finish in 1 hour time.Is it possible to achieve this using selenium webdriver and testng? Please suggest.
It is possible using Selenium Grid and TestNG. Grid can help distribute your tests on various machines or browser instances as you require. To get started, refer : Grid2
I think you might need to change your driver instantiation code a bit to include RemoteWebDriver instead of concrete drivers, but would be ok if your driver instantiation code in your framework is isolated. TestNG and Grid can help provided your tests are well written to support parallel execution.
For TestNG, you can refer, parallel running in TestNG.
If you are using Python - the best way to go is to use py.test to drive the tests. To distribute the tests the pytest-xdist plugin can be used.
Anyway, for both Java and Python you can use Jenkins to run/distribute your tests (using Selenium plugin)
I am pretty newbie by this question.
I want to know about possibilities of how to manipulate Jmeter through the console (bash or cmd).
My goal for a start consists in understanding of how to run my testplan.jmx for several URLS. For this I add "server" and "port" parameters into my testplan.
How could I can change these parameters through the console and then run Jmeter ?
Morover, I want to ask you guys to suggest any free online tutorials where I can learn more about "Jmeter in non gui mode" and possibilities for integration Jmeter between different frameworks to use for automated testing.
Thank you very much indeed.
See:
http://jmeter.512774.n5.nabble.com/How-to-Run-Jmeter-in-command-line-td2640725.html
You can launch your test plan from the command line, specifying parameters, like:
jmeter -n -t plan.jmx -Jmy_url=http://www.firsturl.com
Inside your testplan you'd reference that command line param as ${__P(my_url)}
In terms of capturing results when running in non-gui mode, you may want to see:
http://blogs.amd.com/developer/2009/03/31/using-apache-jmeter-in-non-gui-mode/
Personally, my experience is with using the GUI and writing and running test plans that way but this seems workable.