Processes: fork, exec and wait

Why this matters

Every command you type in a shell becomes a new process, created in two steps that are unique to Unix: fork copies the running shell, and exec replaces the copy with the program you asked for. The same two calls start your editor's compiler, your web server's worker processes and every container in the sandbox you are using right now. Understanding them explains how shells work, why exit statuses exist, where "zombie" processes come from, and why output sometimes appears twice. The next lesson builds pipes on top of them, and the final project in this track is a tiny shell made from exactly these pieces.

What a process is

A process is a running instance of a program: its own memory (code, globals, heap, stack, as in the very first lesson), its own file descriptor table, its current directory, environment, user ID, and a unique process ID (PID). getpid() returns your PID and getppid() your parent's. Every process except the first (PID 1, init or systemd) was created by another process, so processes form a tree, which you can see with ps -ef --forest in the sandbox.

fork: one process becomes two

fork() creates a new process that is an almost exact copy of the caller: same code, a copy of all memory, copies of all file descriptors (pointing to the same open files), at the same point in the program. Both processes return from fork; the return value tells them apart:

                         pid = fork();
                              |
             +----------------+----------------+
             |                                 |
   parent: fork returns                child: fork returns 0
   the child's PID (> 0)               (getppid() is the parent)
             |                                 |
     continues, usually waits          does its job, then exits

fork returns -1 (with errno) if it fails, typically because a process limit was reached; the sandbox allows 256 processes.

The child gets a copy of memory, not shared memory: changing a variable in one process does not affect the other. Copying gigabytes would be slow, so the kernel uses copy-on-write: both processes initially share the same physical pages marked read-only, and a page is copied only when one of them writes to it.

exec: become another program

The exec family replaces the current process's program with a new one: the code, globals, heap and stack are thrown away and the new program's main starts. The PID and open file descriptors stay. exec does not return on success, because the code that called it no longer exists; if it returns, it failed.

The variants differ in how you pass arguments (list vs vector) and whether the program is looked up in PATH (p):

args is an array of strings ending with NULL, exactly like the argv your own main receives, and by convention args[0] is the program's name.

wait: collect the child

The parent learns how a child ended with waitpid(pid, &status, 0) (wait for that child) or wait(&status) (wait for any child). Both block until a child has ended and return its PID. The status int encodes how it ended; decode it with macros from <sys/wait.h>:

This is where the exit status from the first module goes: your return 3; in main reaches the parent through WEXITSTATUS. When the shell prints $? as 139 after a crash, that is 128 plus WTERMSIG = 11, SIGSEGV.

All three together

#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void)
{
    int x = 10;
    fflush(stdout);                       /* see "buffers and fork" below */
    pid_t pid = fork();
    if (pid < 0) {
        perror("fork");
        return 1;
    }
    if (pid == 0) {                       /* child */
        x += 5;
        printf("child: x = %d\n", x);
        fflush(stdout);
        char *args[] = {"echo", "hello from echo", NULL};
        execvp("echo", args);
        perror("execvp");                 /* only reached if exec failed */
        _exit(127);
    }
    int status;                           /* parent */
    if (waitpid(pid, &status, 0) < 0) {
        perror("waitpid");
        return 1;
    }
    printf("parent: x = %d\n", x);
    if (WIFEXITED(status))
        printf("child exited with status %d\n", WEXITSTATUS(status));
    return 0;
}

child: x = 15
hello from echo
parent: x = 10
child exited with status 0

The child's change to x is invisible to the parent: separate memory. The child then turned into echo, which printed its argument and exited with 0. The parent waited, so its lines come last. Without the waitpid, the order of parent and child output would depend on the scheduler.

This is precisely what a shell does for echo hello from echo: fork, exec in the child, wait in the parent, store the status in $?. The convention that "command not found" gives 127 comes from children like this one that _exit(127) after a failed exec.

Why _exit in the child?

After a failed exec, the child is still a copy of the parent. Calling exit would run the parent's atexit handlers and flush copies of the parent's stdio buffers, possibly printing the parent's pending output a second time. _exit ends the process immediately without that cleanup. Use it whenever a forked child ends without having exec'd.

Buffers and fork

Because fork copies all memory, it also copies stdio's output buffer. This program prints its line twice when stdout is a pipe:

printf("before fork\n");
pid_t pid = fork();
if (pid == 0)
    return 0;
waitpid(pid, NULL, 0);
return 0;

$ ./prog | cat
before fork
before fork

On a terminal, stdout is line buffered and the \n flushed the line before the fork, so it prints once. Through a pipe the line was still sitting in the buffer, both processes inherited it, and both flushed it at exit. The fix is one line: fflush(stdout); before every fork.

Zombies and orphans

When a child exits, the kernel keeps a small record of it (PID and exit status) until the parent calls wait. Such an exited-but-uncollected process is a zombie (<defunct> in ps). A long-running parent that never waits accumulates zombies until the system runs out of PIDs. If the parent exits first, its children become orphans and are adopted by PID 1, which waits for them. The rule: every fork needs a matching wait.

Common mistakes

• Not checking which process you are in. Code after fork runs twice. Always branch on the return value immediately.
• Letting the child fall through into the parent's code (for example back into a loop that forks again). End every child path with exec or _exit.
• Forgetting NULL at the end of the argument list for execl/execv: the new program reads garbage arguments or exec fails with EFAULT.
• Forgetting to wait, leaving zombies and nondeterministic output order.
• Reading status directly instead of using the macros. A child that exited with 1 gives a raw status of 256.
• Forgetting fflush(stdout) before fork, duplicating output.

Recap

• fork duplicates the process; it returns 0 in the child and the child's PID in the parent. Memory is copied (copy-on-write), descriptors are shared.
• exec* replaces the program in the current process and returns only on failure.
• wait/waitpid collect a child; decode with WIFEXITED, WEXITSTATUS, WIFSIGNALED, WTERMSIG.
• Shells are fork + exec + wait; 127 means "could not exec", 128 + N means "killed by signal N".
• fflush(stdout) before forking; _exit in children that do not exec; always wait.

Going further: read fork(2), execve(2) and wait(2) at man7.org (the sandbox lacks section 2 pages). In the sandbox terminal, run sleep 30 & and then ps -o pid,ppid,stat,cmd to see your shell as the parent of sleep.