I am running Airflow on an Ubuntu VM.
Here is my config file values for Airflow:
plugins_folder = /home/helia/airflow/utils
dags_folder = /home/helia/airflow/dags
Here are my environment variables set accordingly:
>>echo $PYTHONPATH
/home/helia/airflow:/home/helia/airflow/utils
>>>echo $AIRFLOW_HOME
/home/helia/airflow
If I set my plugins folder path in the cfg file to some non-existing file, the webserver successfully starts and produces the errors fast.
Howver, if I change the cfg file plugins path to the correct path, the OS crashes every time with 100% of CPU, swap space and Memory usage.
I tried to increase number of processors and memory size but nothing got better.
I am using:
Airflow 1.10.12
Python 3.8
UBUNTU 20.1
I am afraid it's your custom plugin that crash the OS. There is not much that anyone could do without knowing and having access to your plugins. You have to revert to those who wrote those plugins to debug them. –
Plugins are loaded by webserver (one of the features of plugins is that they can extend the UI of webserver). Depending on your deployment the "plugin" folder should be shared with the webserver (usually via shared filesystem) –
Related
I'm an undergraduate research assistant working on a Linux server without root privilege. I'm trying to install the Rstudio server but the Rstudio website only provides the installation method for sudoers. Is it possible to install it without root access? I'm asking because I'm really not sure if I could get access from the manager. Any help will be appreciated!
No, you can't install it without root access. But there are a couple of things you could do to piece together a solution. Here are two options:
Extract the server and run it directly
You have to be root to install packages, so you can't install the .deb/.rpm file yourself. However, you could extract the contents of the file to a directory inside your home directory and run RStudio Server from there, by executing the rserver program in a regular shell.
Note that this will probably require an afternoon of editing the rserver.conf file to tell it where to find the rest of the files in the installation (since it presumes they are installed in /usr/lib by default). You can get some inspiration for how to do this here: https://github.com/rstudio/rstudio/blob/master/src/cpp/conf/rserver-dev.conf
Run the desktop version and forward the graphics
The other route is to run RStudio Desktop on the server; we make several builds of RStudio Desktop that are installer-less and can just be unpacked into your home directory. Then run an X11 server on your own computer and an X11 client on the RStudio server, so that the RStudio Desktop instance appears on your computer instead of the server.
Yes, you can run rserver without root priveliges.
For RStudio 1.4 I patched the following line into src/cpp/core/LogOptions.cpp
const FilePath kDefaultLogPath = core::system::xdg::userDataDir().completePath("log");
Then you need to set the system environment variables to some location read-writeable for the user, like
RSTUDIO_CONFIG_DIR=$HOME/.config/rstudio
RSTUDIO_CONFIG_HOME=$HOME/.config/rstudio
RSTUDIO_DATA_HOME=$HOME/.local/share/rstudio
And start rserver with the option
--server-data-dir={directory writeable for user}
--server-pid-file={file-path creatable for user}
--database-config-file={config-file}
With these adjustments it runs for me when I start it as a simple user (no root privileges) with
rserver --auth-none=1 --www-frame-origin=same --www-port={port} --www-verify-user-agent=0 --server-data-dir={my-tmp-path} --server-pid-file={my-tmp-path}/rstudio.pid --database-config-file={my-tmp-path}/db.conf}
ATTENTION:
But be aware, that anyone who can reach your system and the specified port from the network has access to the running RStudio in his browser and therefore can run any command in the name of the user on your system now.
The output in GoLand's terminal:
The output of go env in the OS terminal:
GoLand manages some of the environment variables and settings and overrides what's configured in the system so that it can create a reproducible environment between running code in the builtin terminal or when using the editor itself.
From what I can see in the images, I see that there are some differences around the GOFLAGS, GOPROXY, and GOMOD.
If I understand correctly what you are trying to do, then you need to enable the Go Modules support via Preferences | Go | Go Modules (vgo) and enable the Go Modules integration. Once you do this, you'll see that the GOFLAGS value will change.
There you can also set the Proxy field value to configure the GOPROXY environment variable.
The GOMOD difference comes from the directory where you invoked the go env command, as in this case they seem to be different directories. Invoke the command in the same directory in both IDE terminal and OS terminal and you'll see the same value. It indicates which, if any, go.mod file is used in the current command.
Finally, I recommend upgrading to GoLand 2019.3 as it will automatically enable Go Modules support when it detects that the project is created in a directory with a go.mod file present.
I'm trying to run xv6 operating system on VirtualBox or VMWare in a Linux host. The official instructions said how to run the OS on qemu only. However, the official page (https://pdos.csail.mit.edu/6.828/2014/xv6.html) mentioned that xv6 can be booted directly on hardware also, but it's not clear how.
I want to boot xv6 on VirtualBox or VMware first. I extracted the following command from the Makefile, which runs xv6 from the command line after it's compiled using make command.
/usr/bin/qemu-system-i386 -serial mon:stdio -drive file=fs.img,index=1,media=disk,format=raw -drive file=xv6.img,index=0,media=disk,format=raw -smp 2 -m 512
Please help me how to proceed. If the procedure is already documented some reference will be helpful.
The instructions are here which is linked (via 6.828 tools page) from your link though they are a bit terse:
Using a Virtual Machine
Otherwise, the easiest way to get a compatible toolchain is to install
a modern Linux distribution on your computer. With platform
virtualization, Linux can cohabitate with your normal computing
environment. Installing a Linux virtual machine is a two step process.
First, you download the virtualization platform.
VirtualBox (free for Mac, Linux, Windows) — Download page
VMware Player (free for Linux and Windows, registration required)
VMware Fusion (Downloadable from IS&T for free).
VirtualBox is a little slower and less flexible, but free!
Once the virtualization platform is installed, download a boot disk
image for the Linux distribution of your choice.
Ubuntu Desktop is what we use.
This will download a file named something like
ubuntu-10.04.1-desktop-i386.iso. Start up your virtualization platform
and create a new (32-bit) virtual machine. Use the downloaded Ubuntu
image as a boot disk; the procedure differs among VMs but is pretty
simple. Type objdump -i, as above, to verify that your toolchain is
now set up. You will do your work inside the VM.
I can see how one could read that and not see the answer.
After the virtual machine is installed, download the Ubuntu Desktop .iso. Install that into the VM and fire it up. Presumably the Desktop will provide a clear mechanism for loading your OS. (Wait, I'm giving it a try. Will update with the result.)
Turns out that is simply a Ubuntu client desktop, and isn't anything special for running a sub-operating system.
Looking around some more, I found the commentary to be the best potential clue. It contains this (head scratcher) phrase:
To run xv6, install the QEMU PC simulators. To run in QEMU, run "make qemu".
If only it specified the context to get to that point! (Sorry I am not more help.)
I see that you want to boot it on VirtualBox or VMware, but another option would be to using docker to run xv6. A great guide for getting started with xv6 through docker is here.
The full guide is elaborate and can help you with getting started.
It is an alternative option, but one that can get you going fast hopefully.
It will only take 4 steps to get going with the xv6:
Step 1
Download and set up docker here
Step 2
- Run this command in PowerShell or bash to pull the ubuntu image with xv6 docker pull grantbot/xv6
Step 3
- To run the docker image and get going with xv6 run this command docker run -it grantbot/xv6
Step 4
- Now inside the shell in the ubuntu image run cd /home/a/xv6-public/ to enter the root folder of the xv6.
Done
- Now you can compile and run the xv6 with make qemu-nox
Step 1.Compile xv6
Download the code, unzip it and enter the directory, compile the operating system image and root file system, the command is as follows:
make xv6.img&&make fs.img
Step 2. Write image to disk
Create two disks in a existed vmware virtual machine（my vmware version is 15.2.2, linux version is Centos7.8）, the operation steps are: virtual machine settings -> add -> disk -> SCSI -> create a new virtual disk -> size 0.005 (allocate immediately, single file) -> name the disk "os", which means this disk is the operating system.
Create another disk named "fs" in the same way to put the root file system.
At this time, there should be "sdb" and "sdc" in the /dev/ directory (sda is the current operating system itself). If you do not see the "sdb" and "sdc", restart the guest operating system.
Write the operating system and root file system to the disk with the following command:
dd if=./xv6.img of=/dev/sdb bs=4k count=1000
dd if=./fs.img of=/dev/sdc bs=4k count=1000
shutdown the current virtual machine to ensure that the file has sync to the disk. At this time, the two images have been written to the disk, vmware saves the disk as a file, the location is in the directory of the current virtual machine, named os.vmdk, fs.vmdk, the next step will load these two files into the new virtual machine.
Step 3. Create xv6 virtual machine
To create an empty virtual machine, the operation steps are: customize (advanced) -> next -> install the operating system later -> choose other operating system type (choose other versions) -> take the virtual machine name as xv6 (name depend on you) ) -> Then use the default configuration all the way to "Next" to completion.
Right-click the created virtual machine and delete the disk created by default. Add the disk file created in the previous step to the current virtual machine. The operation steps are: add -> "disk" -> ide (note that this is an IDE instead of a SCSI disk, because xv6 reads an IDE format disk) -> use an existing virtual disk -> select the os.vmdk generate in the step 2->complete
Add fs.vmdk in the same way. Note that you must add os.vmdk first. Because os.vmdk is the operating system, it needs to be the first hard disk.
Now, you create a virtual machine which has two disk. one is os disk, another is root file system disk, all is ready.
Start the virtual machine, and the xv6 will start successfully.
I am using my own laptop locally with win 10 system and intel parallel studio .
After I compiled my mpi code with mpiifort and run it with mpiexec for the first time. It warns me to input account and password, like below
I am sure I put in the correct password. But it just didn't work. What does "execvp error" mean? I never encountered this problem before on my old win8 system. I just installed this new win10 system on my laptop, everything is new. Could somebody please help me instead of making close vote without any comment? At least, say something
execvp error on file is the error from doing execvp system call. It is variant of exec system call used to start programs. In your case the mpiexec program tries to start the mpi-learning-pack.exe file on the target hosts (according to settings, probably some environment settings). This error says that it can't start your program on target hosts, because either it is not executable file, or cannot be found (not copied to target hosts or have no full path).
mpiexec does not copy file to targets, you should copy it to every target hosts.
You can also check if it executable by manually starting it on target host: just login to target host and type mpi-learning-pack.exe without mpiexec;
program may not start if there are no any of required library on target.
Or your account has no enough privileges like https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/607844 https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/624054
Or you just should use relative (mpiexec [options] .\mpi-learning-pack.exe) or full path (mpiexec [options] e:\w\work\fortran\_test_and_learning\mpi-learning-pack.exe) of target executable like in https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/624054
I installed the MSI file from win.meteor.com
I reboot the computer, and open up CMD and type meteor create app but nothing happens. It doesn't say the command was undefined, so I don't know what's wrong here.
I am using windows 7 64-bit home.
I have discovered an issue with node fibers (on windows) and have a tentative fix - at least it fixed the problem for me, and I would be interested to see if that also fixes it for others.
Ultimately the problem was apparently sudden death due stack overflow and access violation. The root cause however was that the thread locals were not correctly being switched on a fiber switch.
you should check the environment path
add Meteor root directory to PATH
like this
PATH: ...;d:\Meteor;d:\Meteor\bin
METEOR_PATH: d:\Meteor
NODE_PATH: d:\Meteor\lib\node_modules
I have this issue too..
METEOR_PATH = C:\Program Files (x86)\Meteor\
NODE_PATH = C:\Program Files (x86)\Meteor\lib\node_modules
PATH = C:\Program Files (x86)\VMware\VMware vSphere CLI\Perl\site\bin;C:\Program Files (x86)\VMware\VMware vSphere CLI\Perl\bin;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Enterprise Vault\EVClient\;C:\Program Files (x86)\Heroku\bin;C:\Program Files (x86)\git\cmd;C:\Program Files\Java\jre6\bin\;C:\Program Files\Sublime Text 2;C:\Program Files (x86)\Meteor\;C:\Program Files (x86)\Meteor\bin
When you I run the meteor command, it does open meteor.js in node (if I add console.log('xxx') to meteor.js I see output), but the script finishes with no output.