strtok() Function in C with Examples

What is strtok() Function in C?

The strtok() function is a standard library function in C, declared in the <string.h> header file. It is used to split a string into a series of tokens based on specified delimiter characters. This function is commonly used for parsing and processing strings.

Syntax

char *strtok(char *str, const char *delim);

• str: The input string to tokenize. For the first call, provide the original string. For subsequent calls, pass NULL to continue tokenizing the same string.
• delim: A string containing one or more delimiter characters that separate tokens.​

How strtok() Works

• On the first call, strtok() scans the string from the beginning until it finds a delimiter. It replaces the delimiter with a null character (\0) and returns a pointer to the beginning of the token.
• On subsequent calls, you must pass NULL instead of the original string. strtok() remembers the position from the last call and continues scanning from there.
• This behavior makes strtok() stateful and not thread-safe, because it uses internal static memory to track the current position in the string.

Note: Since strtok() modifies the original string by inserting null characters (\0), make a copy of the original string if you need to preserve it.

Practical Examples of strtok() Function in C

Example 1: Tokenizing a Sentence

#include <stdio.h>
#include <string.h>
 
int main()
{
    char text[] = "Learn C with Sanfoundry";
    char *word = strtok(text, " ");
 
    while (word != NULL) {
        printf("%s\n", word);
        word = strtok(NULL, " ");
    }
 
    return 0;
}

Output:

Learn
C
with
Sanfoundry

This program uses strtok() to split a sentence into words. It starts with the string “Learn C with Sanfoundry” and breaks it wherever there’s a space. Each word is printed on a new line. The loop continues until all words are processed. This is a simple way to tokenize strings in C.

Example 2: Using Multiple Delimiters

#include <stdio.h>
#include <string.h>
 
int main()
{
    char input[] = "math+science-history,english programming";
    char *subject;
 
    subject = strtok(input, "+-, ");  // multiple delimiters
 
    while (subject != NULL) {
        printf("%s\n", subject);
        subject = strtok(NULL, "+-, ");
    }
 
    return 0;
}

Output:

math
science
history
english
programming

This program breaks a long string into smaller parts using different symbols like +, -, ,, and space. The input string is “math+science-history,english programming”. The strtok() function finds each subject and prints it on a new line. It’s an easy way to separate words that are split by various characters.

Limitations of strtok() Function in C

While strtok() is useful, it has some important limitations:

• Modifies the Original String: strtok() replaces each delimiter character with a null character (\0). If you need to preserve the original string, make a copy before using it.
• Not Safe for Multi-threading: strtok() uses internal memory to remember where it left off. This makes it unsafe in multi-threaded programs. If you need a thread-safe version, use strtok_r().
• Works on One String at a Time: You can’t use strtok() on multiple strings at the same time because it keeps only one internal state.

Sanfoundry Global Education & Learning Series – 1000 C Tutorials.