std::strtok - cppreference.com (original) (raw)

| | | | | ------------------------------------------------ | | | | char* strtok( char* str, const char* delim ); | | |

Tokenizes a null-terminated byte string.

A sequence of calls to std::strtok breaks the string pointed to by str into a sequence of tokens, each of which is delimited by a character from the string pointed to by delim. Each call in the sequence has a search target :

• If str is non-null, the call is the first call in the sequence. The search target is null-terminated byte string pointed to by str.
• If str is null, the call is one of the subsequent calls in the sequence. The search target is determined by the previous call in the sequence.

Each call in the sequence searches the search target for the first character that is not contained in the separator string pointed to by delim, the separator string can be different from call to call.

• If no such character is found, then there are no tokens in the search target. The search target for the next call in the sequence is unchanged.[1]
• If such a character is found, it is the start of the current token. std::strtok then searches from there for the first character that is contained in the separator string.
  • If no such character is found, the current token extends to the end of search target. The search target for the next call in the sequence is an empty string.[2]
  • If such a character is found, it is overwritten by a null character, which terminates the current token. The search target for the next call in the sequence starts from the following character.

If str or delim is not a pointer to a null-terminated byte string, the behavior is undefined.

1. ↑ A token may still be formed in a subsequent call with a different separator string.
2. ↑ No more tokens can be formed in subsequent calls.

Contents

• 1 Parameters
• 2 Return value
• 3 Notes
• 4 Possible implementation
• 5 Example
• 6 See also

[edit] Parameters

[edit] Return value

Returns a pointer to the first character of the next token, or a null pointer if there is no token.

[edit] Notes

This function is destructive: it writes the '\0' characters in the elements of the string str. In particular, a string literal cannot be used as the first argument of std::strtok.

Each call to this function modifies a static variable: is not thread safe.

Unlike most other tokenizers, the delimiters in std::strtok can be different for each subsequent token, and can even depend on the contents of the previous tokens.

[edit] Possible implementation

char* strtok(char* str, const char* delim) { static char* buffer;   if (str != nullptr) buffer = str;   buffer += std::strspn(buffer, delim);   if (buffer == '\0') return nullptr;   char const tokenBegin = buffer;   buffer += std::strcspn(buffer, delim);   if (*buffer != '\0') *buffer++ = '\0';   return tokenBegin; }

Actual C++ library implementations of this function delegate to the C library, where it may be implemented directly (as in MUSL libc), or in terms of its reentrant version (as in GNU libc).

[edit] Example

#include #include #include   int main() { char input[] = "one + two * (three - four)!"; const char* delimiters = "! +- ()"; char token = std::strtok(input, delimiters); while (token) { std::cout << std::quoted(token) << ' '; token = std::strtok(nullptr, delimiters); }   std::cout << "\nContents of the input string now:\n""; for (std::size_t n = 0; n < sizeof input; ++n) { if (const char c = input[n]; c != '\0') std::cout << c; else std::cout << "\0"; } std::cout << ""\n"; }

Output:

"one" "two" "three" "four" Contents of the input string now: "one\0+ two\0* (three\0- four\0!\0"

[edit] See also

| | finds the first location of any character from a set of separators (function) [edit] | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | returns the length of the maximum initial segment that consistsof only the characters not found in another byte string (function) [edit] | | | returns the length of the maximum initial segment that consistsof only the characters found in another byte string (function) [edit] | | | a view over the subranges obtained from splitting another view using a delimiter(class template) (range adaptor object)[edit] | | |