strtok() Function in C with Examples

This Tutorial explains strtok() Function in C programming with examples.

Strings in C comprise of individual parts separated by one or more characters called delimiters. These individual parts are called tokens. In order to process these individual parts, they have to be extracted from the string. That’s what strtok() function does. Empty string doesn’t have any tokens.

The strtok() function in C is used to split a string into multiple tokens. This function takes two arguments: the first is the string to be split, and the second is a delimiter character. The function returns a pointer to the first token in the string, or NULL if there are no tokens.

The strtok() function in C is not thread-safe, so it should not be used in code that is multithreaded. In addition, this function modifies the input string, so it should not be used on strings that are not meant to be modified.

If you need to process a string in multiple threads or do not want to modify the input string, you can use the strtok_r() function, which is a thread-safe version of strtok().

strtok() Function is prototyped below:

    char *strtok(char *str, const char *delimiters);

Let’s understand how strtok() works.

• A sequence of calls to strtok() breaks the string pointed to by ‘str’ into a sequence of tokens, each of which is delimited by a byte from the string pointed to by delimiters.
• The first call in the sequence has ‘str’ as its first argument, and is followed by calls with a null pointer as their first argument.
• The separator string pointed to by delimiters may be different from call to call.
• The first call in the sequence searches the string pointed to by ‘str’ for the first byte that is not contained in the current separator string pointed to by delimiters.
• If no such byte is found, then there are no tokens in the string pointed to by ‘str’ and strtok() shall return a null pointer.
• If such a byte is found, it is the start of the first token.
• The strtok() function then searches from there for a byte (or multiple, consecutive bytes) that is contained in the current separator string.
• If no such byte is found, the current token extends to the end of the string pointed to by ‘str’, and subsequent searches for a token shall return a null pointer.
• If such a byte is found, it is overwritten by a null byte, which terminates the current token.
• The strtok() function saves a pointer to the following byte, from which the next search for a token shall start.

Each subsequent call, with a null pointer as the value of the first argument, starts searching from the saved pointer and behaves as described above. Then function prototype becomes as:

    char *strtok(NULL, const char *delimiters);

Because the strtok() functions parses the string passed to it, it modifies the string. Therefore, in order string must not be changed, copy the string and pass copy as an argument. Remember, don’t copy the address and pass the same. Instead, copy the entire string and pass it as an argument. Let’s take an example:

/* strtok.c -- program finds and extracts tokens in a given string */
#include <stdio.h>
#include <string.h>
 
int main(void)
{
    char str[100];
    char delimiters[] = " \t\n\v\f\r";  /* possible space delimiters */
    char *token;
 
    puts("\n**Program extracts Tokens and displays them One per Line.**\n");
    puts("User, enter a string...");
    gets(str);
 
    /* let's parse the string */
    for (token = strtok(str, delimiters); token != NULL;
         token = strtok(NULL, delimiters)) /* 'for loop' conditional part */
        /* prints token one per line */
        puts(token);
 
    return 0;
}

Output as follows:

 
**Program extracts Tokens and displays them One per Line.**
 
User, enter a string...
strtok()	isolates tokens from    the string
 
strtok()
isolates
tokens
from
the
string
[[email protected] ch09_StringsCharactersAndBytes]# ./strtok 
 
**Program extracts Tokens and displays them One per Line.**
 
User, enter a string...
 
 
[[email protected] ch09_StringsCharactersAndBytes]# ./strtok 
 
**Program extracts Tokens and displays them One per Line.**
 
User, enter a string...

Notice that strtok(), in its first input, parsed the given string into tokens and displayed them one/line. When input was an empty string or was comprising entirely with characters from delimiters string, it returned NULL. Since NULL doesn’t have graphical symbol associated with it, it’s not printed.

Sanfoundry Global Education & Learning Series – 1000 C Tutorials.