Input, output and files · about 55 min

Robust parsing with fgets and strtol

Why this matters

Input comes from people and from other programs, and both get it wrong. A user types 12abc, a CSV file has a blank line, someone pastes a number too large for any integer type. Programs that read numbers with a bare scanf("%d", &n) behave badly on all of these: they silently accept garbage, loop forever, or invoke undefined behavior. The robust technique used in real C code is simple: read a whole line, then parse the line, with functions that tell you exactly how much of the text was valid.

What goes wrong with scanf

#include <stdio.h>

int main(void)
{
    int age = -1;
    int r = scanf("%d", &age);
    printf("scanf returned %d, age = %d\n", r, age);
    int c = getchar();
    printf("next character: '%c'\n", c);
    return 0;
}

$ echo "12abc" | ./scanbad
scanf returned 1, age = 12
next character: 'a'
$ echo "abc" | ./scanbad
scanf returned 0, age = -1
next character: 'a'
$ echo "99999999999" | ./scanbad
scanf returned 1, age = 1215752191
next character: '
'

Three separate problems:

Partial input is accepted. 12abc is reported as a success with the value 12. Nothing tells you that junk followed.
Failed input is not consumed. With abc, scanf stops at a and leaves it there. A loop like while (scanf("%d", &n) != 1) printf("try again\n"); spins forever, because every attempt sees the same a.
Overflow is undefined behavior. The C standard says that if the number does not fit in the type, the behavior is undefined. glibc happened to store a wrapped value above; another library could do something else. You cannot detect it.

scanf is fine for quick tools and for input that is generated by another program you trust. For anything a person types, use the approach below.

Step 1: read the line with fgets

char line[128];
while (fgets(line, sizeof line, stdin) != NULL) {
    /* line holds one line, including '\n' if it fit */
}

fgets never writes beyond the buffer, and it consumes input a line at a time, so a bad line is gone once you have read it: no infinite loops. Two details matter:

The newline is kept. The idiom to remove it is strcspn(s, set) (from <string.h>) returns the length of the initial part of s containing none of the characters in set, which here is the index of the newline, or the length of the string if there is none. Writing '\0' there cuts the string. It works even for a last line without a newline, where it simply overwrites the existing terminator.
```
line[strcspn(line, "\n")] = '\0';
```
A line longer than the buffer comes back in pieces. If the buffer is full and its last character is not '\n', the line was truncated. Decide what to do: reject it, or keep reading until the newline arrives.

  input:  "  -17 apples\n"
  line:   [' '][' ']['-']['1']['7'][' ']['a']['p']['p']['l']['e']['s']['\n']['\0'] ...
                                                                    ^ strcspn finds index 12

Step 2: parse with strtol

long strtol(const char *s, char **endptr, int base);

strtol ("string to long", in <stdlib.h>) skips leading whitespace, accepts an optional sign and digits in the given base, and returns the value. Its superpower is endptr: it stores, in the char * you point it at, the address of the first character it did not use. Comparing that pointer with the start of the string, and looking at what it points to, tells you everything.

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

static void show(const char *s)
{
    char *end;
    errno = 0;
    long v = strtol(s, &end, 10);
    printf("%-24s value=%-20ld consumed=%-2d rest=\"%s\"%s\n",
           s, v, (int)(end - s), end, errno == ERANGE ? " ERANGE" : "");
}

int main(void)
{
    show("42");
    show("  -17 apples");
    show("12abc");
    show("abc");
    show("");
    show("99999999999999999999");
    show("0x1F");
    return 0;
}

42                       value=42                   consumed=2  rest=""
  -17 apples             value=-17                  consumed=5  rest=" apples"
12abc                    value=12                   consumed=2  rest="abc"
abc                      value=0                    consumed=0  rest="abc"
                         value=0                    consumed=0  rest=""
99999999999999999999     value=9223372036854775807  consumed=20 rest="" ERANGE
0x1F                     value=0                    consumed=1  rest="x1F"

Reading the table:

No digits: end == s (consumed 0). The return value 0 is meaningless; "abc" and "0" both give 0, and only end tells them apart.
Junk after the number: *end is not '\0'. Whether that is an error is your decision; whitespace is usually allowed, anything else usually is not.
Overflow: the result is clamped to LONG_MAX or LONG_MIN and errno is set to ERANGE. This is defined behavior, unlike scanf. Because strtol does not reset errno on success, you must set errno = 0 yourself before the call.
Base: with base 10, 0x1F parses as 0 followed by junk. With base 16 it parses as 31, and with base 0, strtol detects the base from the prefix (0x hex, 0 octal), which is handy for configuration files and surprising for users who type 010.

The pointer subtraction end - s counts characters between two positions in the same array; pointer arithmetic is exactly what you learned in the pointers module.

Putting it together

A strict parser in a dozen lines:

int parse_long(const char *s, long *out)
{
    char *end;
    errno = 0;
    long v = strtol(s, &end, 10);
    if (end == s)
        return 0;                       /* no digits */
    if (errno == ERANGE)
        return 0;                       /* out of range */
    while (isspace((unsigned char)*end))
        end++;                          /* allow trailing whitespace */
    if (*end != '\0')
        return 0;                       /* junk */
    *out = v;
    return 1;
}

The function returns success or failure and delivers the value through a pointer parameter, a pattern you saw in the pointers module. It also leaves *out alone on failure, so callers can keep a default. The cast in isspace((unsigned char)*end) matters: the <ctype.h> functions require a value that fits in unsigned char (or EOF), and a char holding a byte above 127 is negative on x86, which is undefined behavior for isspace.

A loop that asks until it gets a valid answer then looks like:

char line[64];
long age;
for (;;) {
    printf("Age: ");
    fflush(stdout);
    if (fgets(line, sizeof line, stdin) == NULL)
        return 1;                       /* end of input: give up */
    if (parse_long(line, &age) && age >= 0 && age < 150)
        break;
    printf("Please enter a whole number between 0 and 149.\n");
}

Each bad line is consumed and discarded, end of input is handled, and the range is checked separately from the syntax.

Other parsers in the same family

strtoul, strtoll, strtoull for unsigned and long long results. Note that strtoul accepts a leading minus sign and negates the result, which surprises people.
strtod for floating point, with the same endptr and ERANGE behavior.
sscanf(line, ...) applies a scanf format to a string. It is convenient for structured lines like "%d,%d", and safer than scanf because a bad line is already consumed, but it still has the overflow problem. The %n conversion stores how many characters were consumed, which lets you check for trailing junk.
atoi and atol exist and are best avoided: they return 0 on failure, indistinguishable from a real 0, and overflow is undefined.

Common mistakes

Testing the return value of strtol to detect errors. 0 is a valid number. Use endptr.
Not resetting errno to 0 before strtol, then seeing a stale ERANGE from an earlier call.
Forgetting the newline from fgets, so "every" line has junk after the number. Allow trailing whitespace or strip the newline first.
Passing a plain char to isspace or isdigit. Cast to unsigned char.
Checking range before syntax. atol("abc") < 150 is true. Parse first, then check bounds.
Mixing scanf and fgets on the same stream: scanf("%d", &n) leaves the newline behind, and the following fgets returns an empty line immediately.

Recap

scanf("%d") accepts partial input, leaves bad input unread, and has undefined behavior on overflow.
Read lines with fgets, strip the newline with line[strcspn(line, "\n")] = '\0', then parse.
strtol(s, &end, base) reports how far it got through end, and overflow through errno (set it to 0 first). No digits: end == s. Junk: *end != '\0'.
Wrap the checks in a small function that returns success and stores the value through a pointer.

Going further: man 3 strtol has a complete example program in its EXAMPLES section. Extend parse_long into parse_double with strtod, and test it on "1e309", "nan" and " .5".

Exercises

Exercise 1 · quiz

What scanf leaves behind

The user types 12abc and presses Enter. The program runs:

int n = -1;
int r = scanf("%d", &n);

What are r and n afterwards, and what is still waiting in the input?

r is 0, n is -1, the whole line is discarded
r is 1, n is 12, and abc plus the newline are still unread
r is 1, n is 12, and the rest of the line is discarded
r is EOF, because the input was not a number

Answer

r is 1, n is 12, and abc plus the newline are still unread

%d reads as many characters as can form a number and stops at the first one that cannot. It converted "12" successfully, so it returns 1, and "abc\n" stays in the stream. The next scanf("%d") will see "abc", fail immediately and return 0 without consuming anything, which is how a loop around scanf turns into an infinite loop on bad input. Reading the whole line first and then parsing it avoids this.

Exercise 2 · write code

A strict integer parser

Write a function

int parse_long(const char *s, long *out);

that decides whether the string s is exactly one integer, and if so stores it in *out and returns 1. Otherwise it returns 0 and does not touch *out.

The rules:

leading whitespace is allowed, then an optional + or -, then at least one decimal digit;
after the digits only whitespace may follow (so a trailing "\n" from fgets is fine);
values that do not fit in a long are rejected.

Examples: "42", " -7", "15\n" and "+3 " succeed; "", "abc", "12abc", "1 2", "3.5" and "99999999999999999999" fail.

Use strtol with an end pointer and errno. Keep a main in your file (anything is fine); the checker calls your function directly with many inputs.

File: main.c

Starting point

#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

int parse_long(const char *s, long *out)
{
    *out = atol(s);   /* atol cannot report errors: replace this */
    return 1;
}

int main(void)
{
    long v;
    if (parse_long("123", &v))
        printf("parsed %ld\n", v);
    return 0;
}

Hint 1

Call strtol(s, &end, 10) after setting errno = 0. If end == s, no digits were found.

Hint 2

If errno == ERANGE afterwards, the value overflowed.

Hint 3

Skip whitespace from end with while (isspace((unsigned char)*end)) end++; and then require *end == '\0'.

Reference solution

#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

int parse_long(const char *s, long *out)
{
    char *end;
    errno = 0;
    long v = strtol(s, &end, 10);
    if (end == s)
        return 0;                       /* no digits at all */
    if (errno == ERANGE)
        return 0;                       /* too big or too small */
    while (isspace((unsigned char)*end))
        end++;
    if (*end != '\0')
        return 0;                       /* junk after the number */
    *out = v;
    return 1;
}

int main(void)
{
    long v;
    if (parse_long("123", &v))
        printf("parsed %ld\n", v);
    return 0;
}

Exercise 3 · quiz

fgets and long lines

With char buf[8];, the input line is hello world followed by a newline. What does the first call fgets(buf, sizeof buf, stdin) put in buf?

"hello world\n" (fgets grows the buffer)
"hello w" followed by the terminating zero; the rest stays in the stream
"hello wo" with no terminating zero
Nothing; fgets returns NULL because the line does not fit

Answer

"hello w" followed by the terminating zero; the rest stays in the stream

fgets stores at most size - 1 = 7 characters and always adds '\0'. The remaining "orld\n" is returned by the next call. You can detect a truncated line because the buffer does not end in a newline (unless the file ended without one). A robust reader either uses a big enough buffer and rejects longer lines, or keeps reading until it sees the newline.

Exercise 4 · write code

Sum the valid lines

Read standard input line by line with fgets (lines are shorter than 100 characters) and add up every line that is a valid integer by the rules of the previous exercise (whitespace around the number is fine). Handle the other lines like this:

a line that is empty or only whitespace is skipped silently;
any other line produces, on stdout, line N: not a number: TEXT, where N is the line number counting from 1 and TEXT is the line without its newline.

At the end print sum = X. Exit with 0 if every non-blank line was valid, and with 1 if any line was rejected. For this input:

10
  20

abc
5x
-3

the output is exactly:

line 4: not a number: abc
line 5: not a number: 5x
sum = 27

and the exit status is 1. You can copy your parse_long from the previous exercise.

File: main.c

Starting point

#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
    char line[128];
    long sum = 0;
    while (fgets(line, sizeof line, stdin) != NULL) {
        sum += atol(line);
    }
    printf("sum = %ld\n", sum);
    return 0;
}

Example

input

10
  20

abc
5x
-3

expected output

line 4: not a number: abc
line 5: not a number: 5x
sum = 27