How to Replace a String in Python (original) (raw)

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Replacing a String in Python

Replacing strings in Python is a fundamental skill. You can use the .replace() method for straightforward replacements, while re.sub() allows for more advanced pattern matching and replacement. Both of these tools help you clean and sanitize text data.

In this tutorial, you’ll work with a chat transcript to remove or replace sensitive information and unwanted words with emojis. To achieve this, you’ll use both direct replacement methods and more advanced regular expressions.

By the end of this tutorial, you’ll understand that:

You’ll be playing the role of a developer for a company that provides technical support through a one-to-one text chat. You’re tasked with creating a script that’ll sanitize the chat, removing any personal data and replacing any swear words with emojis.

You’re only given one very short chat transcript:

Even though this transcript is short, it’s typical of the type of chats that agents have all the time. It has user identifiers, ISO time stamps, and messages.

In this case, the client johndoe filed a complaint, and company policy is to sanitize and simplify the transcript, then pass it on for independent evaluation. Sanitizing the message is your job!

The first thing you’ll want to do is to take care of any swear words.

How to Remove or Replace a Python String or Substring

The most basic way to replace a string in Python is to use the .replace() string method:

As you can see, you can chain .replace() onto any string and provide the method with two arguments. The first is the string that you want to replace, and the second is the replacement.

Now it’s time to apply this knowledge to the transcript:

Loading the transcript as a triple-quoted string and then using the .replace() method on one of the swear words works fine. But there’s another swear word that’s not getting replaced because in Python, the string needs to match exactly:

As you can see, even if the casing of one letter doesn’t match, it’ll prevent any replacements. This means that if you’re using the .replace() method, you’ll need to call it various times with the variations. In this case, you can just chain on another call to .replace():

Success! But you’re probably thinking that this isn’t the best way to do this for something like a general-purpose transcription sanitizer. You’ll want to move toward some way of having a list of replacements, instead of having to type out .replace() each time.

Set Up Multiple Replacement Rules

There are a few more replacements that you need to make to the transcript to get it into a format acceptable for independent review:

Now that you’re starting to have more strings to replace, chaining on .replace() is going to get repetitive. One idea could be to keep a list of tuples, with two items in each tuple. The two items would correspond to the arguments that you need to pass into the .replace() method—the string to replace and the replacement string:

In this version of your transcript-cleaning script, you created a list of replacement tuples, which gives you a quick way to add replacements. You could even create this list of tuples from an external CSV file if you had loads of replacements.

You then iterate over the list of replacement tuples. In each iteration, you call .replace() on the string, populating the arguments with the old and new variables that have been unpacked from each replacement tuple.

With this, you’ve made a big improvement in the overall readability of the transcript. It’s also easier to add replacements if you need to. Running this script reveals a much cleaner transcript:

That’s a pretty clean transcript. Maybe that’s all you need. But if your inner automator isn’t happy, maybe it’s because there are still some things that may be bugging you:

If these are your concerns, then you may want to turn your attention to regular expressions.

Leverage re.sub() to Make Complex Rules

Whenever you’re looking to do any replacing that’s slightly more complex or needs some wildcards, you’ll usually want to turn your attention toward regular expressions, also known as regex.

Regex is a sort of mini-language made up of characters that define a pattern. These patterns, or regexes, are typically used to search for strings in find and find and replace operations. Many programming languages support regex, and it’s widely used. Regex will even give you superpowers.

In Python, leveraging regex means using the re module’s sub() function and building your own regex patterns:

While you can mix and match the sub() function with the .replace() method, this example only uses sub(), so you can see how it’s used. You’ll note that you can replace all variations of the swear word by using just one replacement tuple now. Similarly, you’re only using one regex for the full time stamp:

Now your transcript has been completely sanitized, with all noise removed! How did that happen? That’s the magic of regex.

The first regex pattern, "blast\w*", makes use of the \w special character, which will match alphanumeric characters and underscores. Adding the * quantifier directly after it will match zero or more characters of \w.

Another vital part of the first pattern is that the re.IGNORECASE flag makes it a case-insensitive pattern. So now, any substring containing blast, regardless of capitalization, will be matched and replaced.

The second regex pattern uses character sets and quantifiers to replace the time stamp. You often use character sets and quantifiers together. A regex pattern of [abc], for example, will match one character of a, b, or c. Putting a * directly after it would match zero or more characters of a, b, or c.

There are more quantifiers, though. If you used [abc]{10}, it would match exactly ten characters of a, b or c in any order and any combination. Also note that repeating characters is redundant, so [aa] is equivalent to [a].

For the time stamp, you use an extended character set of [-T:+\d] to match all the possible characters that you might find in the time stamp. Paired with the quantifier {25}, this will match any possible time stamp, at least until the year 10,000.

The time stamp regex pattern allows you to select any possible date in the time stamp format. Seeing as the the times aren’t important for the independent reviewer of these transcripts, you replace them with an empty string. It’s possible to write a more advanced regex that preserves the time information while removing the date.

The third regex pattern is used to select any user string that starts with the keyword "support". Note that you escape (\) the square bracket ([) because otherwise the keyword would be interpreted as a character set.

Finally, the last regex pattern selects the client username string and replaces it with "Client".

With regex, you can drastically cut down the number of replacements that you have to write out. That said, you still may have to come up with many patterns. Seeing as regex isn’t the most readable of languages, having lots of patterns can quickly become hard to maintain.

Thankfully, there’s a neat trick with re.sub() that allows you to have a bit more control over how replacement works, and it offers a much more maintainable architecture.

Use a Callback With re.sub() for Even More Control

One trick that Python and sub() have up their sleeves is that you can pass in a callback function instead of the replacement string. This gives you total control over how to match and replace.

To get started building this version of the transcript-sanitizing script, you’ll use a basic regex pattern to see how using a callback with sub() works:

The regex pattern that you’re using will match the time stamps, and instead of providing a replacement string, you’re passing in a reference to the sanitize_message() function. Now, when sub() finds a match, it’ll call sanitize_message() with a match object as an argument.

Since sanitize_message() just prints the object that it’s received as an argument, when running this, you’ll see the match objects being printed to the console:

$ python transcript_regex_callback.py <re.Match object; span=(15, 40), match='2025-01-24T10:02:23+00:00'> <re.Match object; span=(79, 104), match='2025-01-24T10:03:15+00:00'> <re.Match object; span=(159, 184), match='2025-01-24T10:03:30+00:00'> <re.Match object; span=(235, 260), match='2025-01-24T10:04:03+00:00'>

A match object is one of the building blocks of the re module. The more basic re.match() function returns a match object. sub() doesn’t return any match objects but uses them behind the scenes.

Because you get this match object in the callback, you can use any of the information contained within it to build the replacement string. Once it’s built, you return the new string, and sub() will replace the match with the returned string.

Apply the Callback to the Script

In your transcript-sanitizing script, you’ll make use of the .groups() method of the match object to return the contents of the two capture groups, and then you can sanitize each part in its own function or discard it:

Instead of having lots of different regexes, you can have one top level regex that can match the whole line, dividing it up into capture groups with brackets (()). The capture groups have no effect on the actual matching process, but they do affect the match object that results from the match:

The content of the capturing groups will be available as separate items in the match object by calling the .groups() method, which returns a tuple of the matched strings.

The two groups are the user string and the message. The .groups() method returns them as a tuple of strings. In the sanitize_message() function, you first use unpacking to assign the two strings to variables:

Note how this architecture allows a very broad and inclusive regex at the top level, and then lets you supplement it with more precise regexes within the replacement callback.

The sanitize_message() function makes use of two functions to clean up usernames and bad words. It additionally uses f-strings to justify the messages. Note how censor_bad_words() uses a dynamically created regex while censor_users() relies on more basic string processing.

This is now looking like a good first prototype for a transcript-sanitizing script! The output is squeaky clean:

Nice! Using sub() with a callback gives you far more flexibility to mix and match different methods and build regexes dynamically. This structure also gives you the most room to grow when your bosses or clients inevitably change their requirements on you!

Conclusion

In this tutorial, you’ve learned how to replace strings in Python. Along the way, you’ve gone from using the basic Python .replace() string method to using callbacks with re.sub() for absolute control. You’ve also explored some regex patterns and deconstructed them into a better architecture to manage a replacement script.

With all that knowledge, you’ve successfully cleaned a chat transcript, which is now ready for independent review. Not only that, but your transcript-sanitizing script has plenty of room to grow.

Frequently Asked Questions

Now that you have some experience with replacing strings in Python, you can use the questions and answers below to check your understanding and recap what you’ve learned.

These FAQs are related to the most important concepts you’ve covered in this tutorial. Click the Show/Hide toggle beside each question to reveal the answer.

You replace parts of a string in Python using the .replace() method, chaining it onto the string you want to modify and providing the method with two arguments: the string you want to replace and the replacement string.

To replace all occurrences of a substring in a string, use the .replace() method with the substring you want to replace and the new string as arguments. This method replaces all instances of the substring in the original string.

Yes, you can use the re.sub() function from the re module in Python to replace strings using regular expressions, which allows for more complex and flexible replacements.

To remove part of a string, you can use the .replace() method, specifying the part you want to remove and using an empty string as the replacement.

You replace a letter in a string by using the .replace() method, where the first argument is the letter you want to replace, and the second argument is the new letter you want to use.

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Replacing a String in Python