Please tell me the difference between a process group ID and a jobid. Is jobid a builtin of a shell program or is it related to the kernel? What are the uses of each of them? When a process is run in background, is only jobid set or is the pgid set as well?
What are the uses of the setpgid() function?
When a process is run in background, is the kernel also involved or does the shell take care of which is background or foreground?
Good questions. The job id is mostly just a shell construct. There is support in the kernel in the form of the signals that are involved in job control, and the way in which the kernel knows exactly which processes to send the job control signals to.
Strictly speaking, the answer to your first question is that the job id is purely a shell creation. It exists because a pipeline (or, rarely, another shell grouped construct) may consist of multiple processes that should be controlled as a unit.
To answer your last question, the shell starts all processes by first doing a fork(2) and then doing an execve(2). The only difference with & is that the shell does not do a wait(2) (or a related variant) and so the program can continue "in the background". There is actually little distinction in Unix between foreground and background.
The process group is an association defined by shells so that the kernel knows about a single "foreground" process that handles a set of various "background" processes. This is mainly important so that the background processes will generate a signal should they decide to suddenly read from a terminal. (Such terminal probably being connected to standard input.) This will cause the "job" to generate a signal and the shell will prompt the user to do something.
Try (sleep 5; read x)& and after 6 seconds type a return or something so that the shell wakes up. That's when you see something like...
[1]+ Stopped ( sleep 5; read x )
...and you then type fg to pull it into the foreground.
Originally, Unix had pipelines, and it had &, but there was no way to move a command or pipeline between foreground and background and no way to help a background process that suddenly decided to read standard input.
Job control and the kernel support for it were added by Bill Joy and others in early versions of BSD and csh(1). These were picked up line-for-line by commercial Unix and in cloned for the work-alike Linux kernel.
Regarding the questions about process groups and ps(1)...
In order to support job control in shells, the kernel process state includes a process group ID and a session ID. A process group and a job are the same thing, but a job number is just a handle the shell makes up. A process is a session leader if the session ID is the same as the pid, and a process is a process group leader if the pgid is the same as the pid. I believe something a bit more subtle is happening with the + that ps(1) prints. Each terminal knows what its foreground process group is, so I believe a process gets a + if pid == pgid && (pgid is the foreground pg for its controlling terminal).
In summary, the kernel keeps several items of state: pid, pgid, sid, and a process may have a controlling terminal and a terminal may have a foreground pgid. These credentials are mostly intended to support job control but are also used to revoke access to a terminal when a user logs out.
Related
What's the UNIX command to see the processes table, remember that table contains:
process status
pointers
process size
user ids
process ids
event descriptors
priority
etc
The "process table" as such lives in the kernel's memory. Some systems (such as AIX, Solaris and Linux--which is not "unix") have a /proc filesystem which makes those tables visible to ordinary programs. Without that, programs such as ps (on very old systems such as SunOS 4) required elevated privileges to read the /dev/kmem (kernel memory) special device, as well as having detailed knowledge about the kernel memory layout.
Your question is open ended, and an answer to a specific question you may have had can be looked up in any man page as #Alfasin suggests in his answer. A lot depends on what you are trying to do.
As #ThomasDickey points out in his response, in UNIX and most of its' derivatives, the command for viewing processes being run in the background or foreground is in fact the ps command.
ps stands for 'process status', answering your first bullet item. But the command uses over 30 options and depending on what information you seek, and permissions granted to you by the systems administrator, you can get various types of information from the command.
For example, for the second bullet item on your list above, depending on what you are looking for, you can get information on 3 different types of pointers - the session pointer (with option 'sess'), the terminal session pointer (tsess), and the process pointer (uprocp).
The rest of your items that you have listed are mostly available as standard output of the command.
Some UNIX variants implement a view of the system process table inside of the file system to support the running of programs such as ps. This is normally mounted on /proc (see #ThomasDickey response above)
Typical reasons for understanding the working of the command include system-administration responsibilities such as tracking the origin of the initiated processes, killing runaway or orphaned processes, examining the file size of the process and setting limits where necessary, etc. UNIX developers can also use it in conjunction with ipc features, etc. An understanding of the process table and status will help with associated UNIX features such as the kvm interface to examine crash dump, etc. or to get or set the kernal state.
Hope this helps
When I start a process in background in a terminal and some how if terminal gets closed then we can not interact that process any more. I am not sure but I think process also get killed. Can any one please tell me how can I detach that process from my terminal. So even if I close terminal then I can interact with same process in new terminal ?
I am new to unix so your extra information will help me.
The command you're looking for is disown.
disown <processid>
This is as close as you can get to a nohup. It detaches the process from the current login and allows it to continue running. Thanks David Korn!
http://www2.research.att.com/~gsf/man/man1/disown.html
and I just found reptyr which lets you reparent a disowned process.
https://github.com/nelhage/reptyr
It's already in the packages for ubuntu.
BUT if you haven't started the process yet and you're planning on doing this in the future then the way to go is screen and tmux. I prefer screen.
You might also consider the screen command. It has the "restore my session" functionality. Admittedly I have never used it, and forgot about it.
Starting the process as a daemon, or with nohup might not do everything you want, in terms of re-capturing stdout/stdin.
There's a bunch of examples on the web. On google try, "unix screen command" and "unix screen tutorial":
http://www.thegeekstuff.com/2010/07/screen-command-examples/
GNU Screen: an introduction and beginner's tutorial
First google result for "UNIX demonizing a process":
See the daemon(3) manpage for a short overview. The main thing of daemonizing
is going into the background without quiting or holding anything up. A list of
things a process can do to achieve this:
fork()
setsid()
close/redirect stdin/stdout/stderr to /dev/null, and/or ignore SIGHUP/SIGPIPE.
chdir() to /.
If started as a root process, you also want to do the things you need to be root
for first, and then drop privileges. That is, change effective user to the "daemon"
user or "nobody" with setuid()/setgid(). If you can't drop all privileges and need
root access sometimes, use seteuid() to temporary drop it when not needed.
If you're forking a daemon then also setup child handlers and, if calling exec,
set the close on exec flags on all file descriptors your children won't need.
And here's a HOWTO on creating Unix daemons: http://www.netzmafia.de/skripten/unix/linux-daemon-howto.html
'Interact with' can mean a couple of things.
The reason why a program, started at the command-line, exits when the terminal ends, is because the shell, when it exits, sends that process a HUP signal (see documentation for kill(1) for some introduction; HUP, by the way, is short for 'hang up', and originally indicated that the user had hung up the modem/telephone). The default response to a HUP signal is that a process is terminated – that is, the invoked program exits.
The details are slightly more fiddly, but this is the general intuition.
The nohup command tells the shell to start the program, and to do so in a way that this HUP signal is ignored. That is, the program keeps going after the invoking terminal exits.
You can still interact with this program by sending it signals (see kill(1) again), but this is a very limited sort of interaction, and depends on your program being written to do sensible things when it receives those signals (signals USR1 and USR2 are useful things to trap, if you're into that sort of thing). Alternatively, you can interact via named pipes, or semaphores, or other bits of inter-process communication (IPC). That gets fiddly pretty quickly.
I suspect what you're after, though, is being able to reattach a terminal to the process. That's a rather more complicated process, and applications like screen do suitably complicated things behind the scenes to make that happen.
The nohup thing is a sort of quick-and-dirty daemonisation. The daemon(3) function does the daemonisation 'properly', doing various bits of tidy-up as described in YePhIcK's answer, to comprehensively break the link with the process/terminal that invoked it. You can interact with that daemonised process with the same IPC tools as above, but not straightforwardly with a terminal.
I'm using mac/linux and I know that ctrl-z stops the currently running command in terminal, but I frequently see the process is still running when i check the system monitor. What is the right way to stop a command in terminal?
Typically I run into this issue when running python or ruby apps, i'm not sure if that has something to do with it, just thought I would add that.
Using control-z suspends the process (see the output from stty -a which lists the key stroke under susp). That leaves it running, but in suspended animation (so it is not using any CPU resources). It can be resumed later.
If you want to stop a program permanently, then any of interrupt (often control-c) or quit (often control-\) will stop the process, the latter producing a core dump (unless you've disabled them). You might also use a HUP or TERM signal (or, if really necessary, the KILL signal, but try the other signals first) sent to the process from another terminal; or you could use control-z to suspend the process and then send the death threat from the current terminal, and then bring the (about to die) process back into the foreground (fg).
Note that all key combinations are subject to change via the stty command or equivalents; the defaults may vary from system to system.
if you do ctrl-z and then type exit it will close background applications.
Ctrl+Q is another good way to kill the application.
Take a look at Job Control on UNIX systems
If you don't have control of your shell, simply hitting ctrl + C should stop the process. If that doesn't work, you can try ctrl + Z and using the jobs and kill -9 %<job #> to kill it. The '-9' is a type of signal. You can man kill to see a list of signals.
I would like to know if there is a way that an MPI process send a kill signal to another MPI process?
Or differently, is there a way to exit from an MPI environment graciously, when one of the process is still active? (i.e. mpi_abort() prints an error message).
Thanks
No, this is not possible within an MPI application using the MPI library.
Individual processes would not be aware of the location of the other processes, nor of the process IDs of the other processes - and there is nothing in the MPI spec to make the kill you are wanting.
If you were to do this manually, then you'd need to MPI_Alltoall to exchange process IDs and hostnames across the system, and then you would need to spawn ssh/rsh to visit the required node when you wanted to kill something. All in all, it's not portable, not clean.
MPI_Abort is the right way to do what you are trying to achieve. From the Open MPI manual:
"This routine makes a "best attempt" to abort all tasks in the group of comm." (ie. MPI_Abort(MPI_COMM_WORLD, -1) is what you need.
Any output during MPI_Abort would be machine specific - so you may, or may not, receive the error message you mention.
I asked a related question
'Difference between process group id and job id in Unix',
but I have a doubt that was not answered. In that answer, it was written that job id is built in data related to shell (kernel has nothing to do with it), so foreground / background is understanding of shell, but when we do "ps x" then it display in stats as "R+" or "R" as foreground and background process respectively but ps is executed by kernel so how does kernel know that job is back/foreground.
Job IDs are definitely shell features.
However, the operating system has the notion of current terminal process group ID, which is also used by the shell to facilitate job control. Processes which have the same process group ID as the current terminal process receive keyboard signal such as SIGINT and are able to do I/O on the terminal. These processes are foreground processes. If the process group ID is different, then the process is a background process. The ps command displays + when the process is a foreground process in this notion. It still does not need to know anything about job control as implemented by the shell.
I wouldn't say 'kernel has nothing to do with it' since the shell uses routines in the kernel to achieve its purposes.
The shell decides whether a process is foreground or background, but the kernel must be aware of it too, and can report the status of processes when the shell asks.