Foundations: from source to running program · about 40 min

From source to executable

Why this matters

When a build fails, the error message tells you which stage failed, and each stage fails for different reasons. A missing header, a syntax error and an "undefined reference" look alike to a beginner, but they come from three different programs and need three different fixes. Knowing the pipeline also explains things that otherwise look like magic: what #include really does, why you need -lm for math functions, and how large programs are built from many files.

The four stages

When you run gcc main.c -o prog, gcc is really a driver that runs four tools in turn:

 main.c            your source text
   |  1. preprocessor (cpp)       handles # lines: #include, #define, #if
   v
 main.i            pure C text, headers pasted in, macros expanded
   |  2. compiler (cc1)           checks types and grammar, translates C to assembly
   v
 main.s            assembly text for your processor
   |  3. assembler (as)           assembly to machine code bytes
   v
 main.o            object file: machine code with "holes" for outside names
   |  4. linker (ld)              joins object files + libraries, fills the holes
   v
 prog              executable

Normally the intermediate files are temporary and deleted. You can stop after any stage to see them, and doing it once is the best way to understand the pipeline.

Here is a small program, hello.c:

#include <stdio.h>

#define GREETING "Hello"

int main(void)
{
    printf("%s, world\n", GREETING);
    return 0;
}

Stage 1: the preprocessor (`gcc -E`)

The preprocessor is a text processor that knows almost nothing about C. It handles every line starting with #:

#include <stdio.h> is replaced by the entire contents of the file /usr/include/stdio.h (and whatever it includes). That header contains declarations such as int printf(const char *format, ...); which tell the compiler that printf exists and what arguments it takes. It does not contain the code of printf.
#define GREETING "Hello" defines a macro; every later GREETING in the text is replaced by "Hello".
Comments are removed.

$ gcc -E hello.c -o hello.i
$ wc -l hello.i
819 hello.i
$ tail -6 hello.i
int main(void)
{
    printf("%s, world\n", "Hello");
    return 0;
}

Your nine lines became over 800, almost all from stdio.h, and GREETING is gone. Errors from this stage are about # lines, for example fatal error: stdoi.h: No such file or directory.

Stage 2: the compiler proper (`gcc -S`)

The compiler reads the preprocessed text, checks it against the rules of C (grammar, types, declarations), and translates it into assembly language. This is where almost all of your error messages come from: missing semicolons, unknown names, wrong argument types.

$ gcc -S -O0 -fno-asynchronous-unwind-tables hello.c -o hello.s
$ grep -A11 '^main:' hello.s
main:
        pushq   %rbp
        movq    %rsp, %rbp
        leaq    .LC0(%rip), %rax
        movq    %rax, %rsi
        leaq    .LC1(%rip), %rax
        movq    %rax, %rdi
        movl    $0, %eax
        call    printf@PLT
        movl    $0, %eax
        popq    %rbp
        ret

(The -fno-asynchronous-unwind-tables flag just hides bookkeeping lines starting with .cfi that would otherwise clutter the listing.)

You do not need to read assembly, but notice two things: your function became a short list of instructions, and printf is just a call to a name. The compiler does not know where printf's code is. It only knows, from the header, how to call it.

Stage 3: the assembler (`gcc -c`)

The assembler converts the assembly text into machine code bytes and writes an object file (.o). An object file contains your machine code plus a symbol table: the names it defines (main) and the names it uses but does not define (printf).

$ gcc -c hello.c -o hello.o
$ nm hello.o
0000000000000000 T main
                 U printf

T means "defined here, in the code (text) section". U means "undefined: used here, defined somewhere else". An object file cannot run on its own: the call to printf still points nowhere.

Stage 4: the linker (`ld`)

The linker takes one or more object files plus libraries and produces the executable. Its job is to resolve every U symbol to a definition somewhere. printf is defined in the C standard library (libc), which gcc links automatically. For most of the C library, linking is dynamic: the executable records "I need libc.so.6", and the operating system loads that shared library when the program starts.

$ gcc hello.o -o hello
$ ./hello
Hello, world
$ ldd hello
        linux-vdso.so.1 (0x00007fc7610cf000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc760ecb000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc7610d1000)

If a symbol cannot be resolved, the linker fails with undefined reference. The math functions (sqrt, sin, pow) live in a separate library, libm, which is not linked by default on Linux. That is what -lm is for: "link library m". The -l flags go after the source files, because the linker resolves names left to right.

Declarations versus definitions

The pipeline explains one of the most important distinctions in C:

A declaration tells the compiler a name exists and what its type is: void greet(void); This is enough to compile a call to it.
A definition provides the actual thing: the function body, which becomes machine code. This is what the linker needs.

Headers like stdio.h are mostly declarations. The definitions are already compiled, in libraries. When you split your own programs into several files later in the track, you will write headers full of declarations and .c files full of definitions, and the linker will join them.

The build command used in this track

$ gcc -std=c17 -Wall -Wextra -g -O0 -o prog main.c -lm

flag	meaning
`-std=c17`	Use the C17 standard (the 2017/2018 revision of ISO C). Fixes the language rules.
`-Wall -Wextra`	Turn on most useful warnings. Despite the name, `-Wall` is not all of them.
`-g`	Include debugging information so `gdb` can map machine code back to your lines.
`-O0`	No optimization: the machine code follows your source closely, good for learning and debugging. Real builds often use `-O2`.
`-o prog`	Name the output `prog` (the default is `a.out`).
`main.c`	The source file.
`-lm`	Link the math library.

Common mistakes

Misspelled header (preprocessor):

main.c:1:10: fatal error: stdoi.h: No such file or directory

Syntax error (compiler). The compiler reports where it noticed the problem, which is often the line after the real mistake:

main.c:6:5: error: expected ';' before 'return'

Calling a function with no declaration. With GCC 14 (the version in the sandbox) this is an error:

main.c:5:5: error: implicit declaration of function 'prnitf'; did you mean 'printf'? [-Wimplicit-function-declaration]

Older compilers only warned and then guessed the function's type, which is why old tutorials seem to "get away with" missing includes.

Missing definition (linker): a declaration was present, so the compiler was happy, but no definition exists:

/usr/bin/ld: /tmp/cc1aB2.o: in function `main':
main.c:7: undefined reference to `greet'
collect2: error: ld returned 1 exit status

Forgetting -lm gives undefined reference to 'sqrt' in larger programs. (With constant arguments like sqrt(2.0) gcc may compute the result at compile time and not need the library at all, which makes this mistake appear and disappear mysteriously.)

Recap

gcc drives four stages: preprocess (-E), compile to assembly (-S), assemble to an object file (-c), link into an executable.
#include pastes a header's text in; headers mostly hold declarations.
Object files list defined (T) and undefined (U) symbols; the linker resolves them against other object files and libraries such as libc and libm (-lm).
Error messages tell you the stage: "No such file" (preprocessor), "expected ..." or "implicit declaration" (compiler), "undefined reference" (linker).

Going further: in the sandbox terminal, write a hello program and run gcc -E, gcc -S, gcc -c and nm on it yourself. Then try gcc -v hello.c to watch the driver print every command it runs.

Exercises

Exercise 1 · quiz

The order of the stages

In what order does gcc process a C source file into an executable?

compile, preprocess, link, assemble
preprocess, compile, assemble, link
link, preprocess, compile, assemble
preprocess, link, compile, assemble

Answer

preprocess, compile, assemble, link

Preprocess (text substitution: #include, #define), compile (C to assembly), assemble (assembly to machine code in an object file), link (combine object files and libraries into one executable). gcc runs all four for you; -E, -S and -c stop after the first, second and third.

Exercise 2 · quiz

Which stage reported the error?

You build a program and see:

/usr/bin/ld: /tmp/ccQ2x1.o: in function `main':
main.c:5:(.text+0x9): undefined reference to `compute_total'
collect2: error: ld returned 1 exit status

Which stage failed?

The preprocessor
The compiler proper
The assembler
The linker

Answer

The linker

ld is the linker, and "undefined reference" is its signature error. The compiler was satisfied because it had seen a declaration of compute_total (or at least a call it accepted), but when the linker tried to connect the call to actual machine code, no object file or library defined that function. Typical causes: a typo in the function name, forgetting to compile the .c file that defines it, or forgetting a library flag such as -lm.

Exercise 3 · quiz

What does -o do?

In gcc -std=c17 -Wall -Wextra -g -O0 -o prog main.c -lm, what does -o prog mean? Answer in a few words.

Answer

-o names the output file. Without it gcc writes the executable as a.out. The other flags: -std=c17 selects the 2017 C standard, -Wall -Wextra turn on most useful warnings, -g adds debugging information for gdb, -O0 disables optimization so the machine code follows your source closely, and -lm links the math library.

Exercise 4 · write code

Fix the undefined reference

The starter declares a function greet and calls it from main, but nobody ever wrote the function itself. It compiles but fails to link with undefined reference to 'greet'.

Add a definition of greet (below main is fine, since the declaration at the top already told the compiler about it) so that the program prints exactly:

Hello, linker!
Done.

File: main.c

Starting point

#include <stdio.h>

void greet(void);   /* declaration: "a function called greet exists somewhere" */

int main(void)
{
    greet();
    printf("Done.\n");
    return 0;
}

Example

expected output

Hello, linker!
Done.

Hint 1

A definition looks like the declaration, but with a body in braces instead of a semicolon.

Hint 2

Write void greet(void) followed by { ... } containing one printf call.

Reference solution

#include <stdio.h>

void greet(void);   /* declaration: "a function called greet exists somewhere" */

int main(void)
{
    greet();
    printf("Done.\n");
    return 0;
}

void greet(void)
{
    printf("Hello, linker!\n");
}

Exercise 5 · write code

Call into the math library

The sqrt function lives in the C math library, declared in the header <math.h>. The build command already links it with -lm.

Complete the program so it prints the square root of 2 with 6 decimal places, exactly:

1.414214

Use printf("%f\n", value); to print a decimal number with six digits after the point.

File: main.c

Starting point

#include <stdio.h>
/* include the math header here */

int main(void)
{
    /* print sqrt(2.0) using %f */
    return 0;
}

Example

expected output

1.414214

Hint 1

Add #include <math.h> under the stdio include.

Hint 2

printf("%f\n", sqrt(2.0)); prints it.

Reference solution

#include <stdio.h>
#include <math.h>

int main(void)
{
    printf("%f\n", sqrt(2.0));
    return 0;
}

From source to executable

Why this matters

The four stages

Stage 1: the preprocessor (gcc -E)

Stage 2: the compiler proper (gcc -S)

Stage 3: the assembler (gcc -c)

Stage 4: the linker (ld)

Declarations versus definitions

The build command used in this track

Common mistakes

Recap

Exercises

The order of the stages

Which stage reported the error?

What does -o do?

Fix the undefined reference

Call into the math library

Stage 1: the preprocessor (`gcc -E`)

Stage 2: the compiler proper (`gcc -S`)

Stage 3: the assembler (`gcc -c`)

Stage 4: the linker (`ld`)