File descriptors: open, read, write, close

Why this matters

Everything you have done with files so far went through <stdio.h>: fopen, fprintf, fgets. Underneath, stdio is a library that calls a handful of system calls, requests to the operating system kernel. Learning those calls shows you what a file really is to a process, explains puzzles like output appearing in the wrong order, and opens the door to everything stdio cannot do: pipes between processes, terminals, sockets, devices. On Unix, all of those are used through the same four calls this lesson teaches, which is what people mean by "everything is a file".

System calls

A process cannot touch the disk, the terminal or another process directly. The CPU runs your code in user mode, where such operations are forbidden. To get anything done, the program executes a special instruction (syscall on x86-64) that switches into the kernel, which checks the request, performs it, and returns a result. The C library wraps each system call in an ordinary-looking function: open, read, write, close, fork and so on. These are POSIX functions, declared in <unistd.h> and <fcntl.h>, not part of ISO C. To see them with -std=c17 you request POSIX explicitly by defining a feature test macro before any include:

#define _POSIX_C_SOURCE 200809L

Without it, gcc in strict C17 mode hides POSIX declarations, and you get errors such as implicit declaration of function 'fdopen'.

File descriptors

When a process opens a file, the kernel creates an open file description (which file, the current offset, the access mode) and records it in the process's file descriptor table. The program gets back the index into that table: a small non-negative int called a file descriptor (fd).

  process                              kernel
  fd table                             open file descriptions
  +---+-----------------+
  | 0 | o---------------+-------->  terminal (read)
  | 1 | o---------------+-------->  terminal (write)
  | 2 | o---------------+-------->  terminal (write)
  | 3 | o---------------+-------->  notes.txt, offset 0, write-only
  +---+-----------------+

Every process starts with three descriptors already open: 0 standard input, 1 standard output, 2 standard error (STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO in <unistd.h>). The shell sets them up before running your program, and redirections like > out.txt just make descriptor 1 point somewhere else. open always returns the lowest unused number, so the first file you open is normally 3. On Linux you can see the table of any process in /proc/PID/fd.

The four calls

int     open(const char *path, int flags, ... /* mode_t mode */);
ssize_t read(int fd, void *buf, size_t count);
ssize_t write(int fd, const void *buf, size_t count);
int     close(int fd);

open takes flags combined with |, exactly the flag-set technique from the bit manipulation module. Exactly one access mode, plus any options:

The third argument, the mode, gives a newly created file's permissions in octal: 0644 is rw-r--r--. The process's umask (usually 022) then removes bits from it. open returns the new descriptor, or -1 with errno set.

read asks for up to count bytes and returns how many it actually got: possibly fewer (a short read), 0 at end of file, or -1 on error. Fewer is normal: a terminal returns one line at a time, a pipe returns whatever is available, and the end of a file gives a partial last chunk.

write writes up to count bytes and returns how many it wrote. For regular files it almost always writes everything, but for pipes, sockets and when interrupted by a signal it may write fewer, so robust code loops.

close releases the descriptor. The number becomes free for reuse; the kernel tracks how many descriptors refer to each open file and releases it when the last one closes. When a process exits, the kernel closes all its descriptors.

Neither read nor write knows anything about strings or lines: they move raw bytes. There is no terminating '\0' in what read gives you, so use the returned count.

A worked example

#define _POSIX_C_SOURCE 200809L
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

/* write all len bytes, retrying after short writes */
int write_all(int fd, const char *buf, size_t len)
{
    while (len > 0) {
        ssize_t n = write(fd, buf, len);
        if (n < 0) {
            if (errno == EINTR)
                continue;          /* interrupted by a signal: try again */
            return -1;
        }
        buf += n;
        len -= (size_t)n;
    }
    return 0;
}

int main(void)
{
    int fd = open("notes.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd < 0) {
        perror("open notes.txt");
        return 1;
    }
    printf("opened notes.txt as fd %d\n", fd);
    const char *msg = "first line\nsecond line\n";
    if (write_all(fd, msg, strlen(msg)) != 0) {
        perror("write");
        close(fd);
        return 1;
    }
    close(fd);

    fd = open("notes.txt", O_RDONLY);
    if (fd < 0) {
        perror("open");
        return 1;
    }
    char buf[8];
    ssize_t n;
    int chunks = 0;
    while ((n = read(fd, buf, sizeof buf)) > 0) {
        chunks++;
        write_all(STDOUT_FILENO, buf, (size_t)n);
    }
    if (n < 0)
        perror("read");
    close(fd);
    printf("read in %d chunks\n", chunks);

    if (open("/etc/new-file", O_WRONLY | O_CREAT, 0644) < 0)
        printf("open /etc/new-file: %s (errno %d)\n", strerror(errno), errno);
    if (open("missing.txt", O_RDONLY) < 0)
        perror("open missing.txt");
    return 0;
}

The 23-byte file is read with an 8-byte buffer in 3 chunks (8 + 8 + 7). In the sandbox the root file system is read-only, so creating a file in /etc fails. Run with output going through a pipe (as in the exercises), it prints:

open missing.txt: No such file or directory
first line
second line
opened notes.txt as fd 3
read in 3 chunks
open /etc/new-file: Read-only file system (errno 30)

That order looks wrong, and understanding it is the most practical lesson here:

• perror writes to stderr, which stdio never buffers, so it appears immediately (here it is even listed first because stdout and stderr went to separate pipes).
• write_all(STDOUT_FILENO, ...) goes straight to the kernel.
• printf goes into stdio's buffer. On a terminal stdout is line buffered, so each \n would flush it, but on a pipe or file it is fully buffered and flushed only when full or at exit.

So the printf lines all came out together at the end. On a terminal you would see the "natural" order. The rule: do not mix stdio and raw write on the same descriptor, or call fflush(stdout) before switching.

stdio versus system calls

FILE * streams are a buffer plus a file descriptor (fileno(f) gives you the descriptor). Each system call costs a switch into the kernel and back, maybe a few hundred nanoseconds; calling write once per character would be painfully slow, which is why stdio collects output and issues one write per few kilobytes. Use stdio for text and formatted I/O; use descriptors when you need precise control (no buffering, exact byte counts, special files) or when working with pipes and processes, as the next lessons do.

Errors

System calls report failure by returning -1 and setting errno, as you learned in the I/O module. Common values here: ENOENT (no such file), EACCES (permission denied), EEXIST (with O_EXCL), EROFS (read-only file system), EBADF (not an open descriptor), EINTR (interrupted by a signal; just retry).

Common mistakes

• Forgetting the mode with O_CREAT. The file gets whatever garbage is in the register: random permissions. gcc cannot warn because open takes variable arguments.
• Writing a decimal mode like 644 instead of octal 0644: that is 01204, a strange mode.
• Treating read's buffer as a string: printf("%s", buf) reads past the data looking for a '\0'. Use the count.
• Ignoring short reads and writes. Code that assumes one read gets the whole file works in testing on small files and fails on pipes.
• Leaking descriptors. A process can only have so many open (often 1024, see ulimit -n); a long-running program that forgets close eventually fails with EMFILE.
• Mixing printf and write on stdout without fflush.

Recap

• System calls are the kernel's interface; open, read, write and close are POSIX wrappers needing _POSIX_C_SOURCE under -std=c17.
• A file descriptor is a small int indexing the process's open file table; 0, 1 and 2 are open at start, and open returns the lowest free number.
• read and write move raw bytes and may do less than asked; loop. read returns 0 at end of file.
• O_CREAT needs a mode; O_APPEND appends; O_TRUNC empties.
• stdio buffers on top of descriptors; that explains output ordering surprises.

Going further: the sandbox has no section 2 man pages, so read open(2), read(2) and write(2) at man7.org. In the sandbox terminal, run ls -l /proc/self/fd and then ls -l /proc/self/fd < /etc/hostname to watch descriptor 0 change.