Custom Python Lists: Inheriting From list vs UserList (original) (raw)

At some point in your Python coding adventure, you may need to create custom list-like classes with modified behavior, new functionalities, or both. To do this in Python, you can inherit from an abstract base class, subclass the built-in list class directly, or inherit from UserList, which lives in the collections module.

In this tutorial, you’ll learn how to:

You’ll also write some examples that’ll help you decide which parent class, list or UserList, to use when creating your custom list classes.

To get the most out of this tutorial, you should be familiar with Python’s built-in list class and its standard features. You’ll also need to know the basics of object-oriented programming and understand how inheritance works in Python.

Creating List-Like Classes in Python

The built-in list class is a fundamental data type in Python. Lists are useful in many situations and have tons of practical use cases. In some of these use cases, the standard functionality of Python list may be insufficient, and you may need to create custom list-like classes to address the problem at hand.

You’ll typically find at least two reasons for creating custom list-like classes:

  1. Extending the regular list by adding new functionality
  2. Modifying the standard list’s functionality

You can also face situations in which you need to both extend and modify the list’s standard functionality.

Depending on your specific needs and skill level, you can use a few strategies to create your own custom list-like classes. You can:

There are a few considerations when you’re selecting the appropriate strategy to use. Keep reading for more details.

Building a List-Like Class From an Abstract Base Class

You can create your own list-like classes by inheriting from an appropriate abstract base class (ABC), like MutableSequence. This ABC provides generic implementations of most list methods except for .__getitem__(), .__setitem__(), .__delitem__, .__len__(), and .insert(). So, when inheriting from this class, you’ll have to implement these methods yourself.

Writing your own implementation for all these special methods is a fair amount of work. It’s error-prone and requires advanced knowledge of Python and its data model. It can also imply performance issues because you’ll be writing the methods in pure Python.

Additionally, suppose you need to customize the functionality of any other standard list method, like .append() or .insert(). In that case, you’ll have to override the default implementation and provide a suitable implementation that fulfills your needs.

The main advantage of this strategy for creating list-like classes is that the parent ABC class will alert you if you miss any required methods in your custom implementation.

In general, you should embrace this strategy only if you need a list-like class that’s fundamentally different from the built-in list class.

In this tutorial, you’ll focus on creating list-like classes by inheriting from the built-in list class and the UserList class from the standard-library collections module. These strategies seem to be the quickest and most practical ones.

Inheriting From Python’s Built-in list Class

For a long time, it was impossible to inherit directly from Python types implemented in C. Python 2.2 fixed this issue. Now you can subclass built-in types, including list. This change has brought several technical advantages to the subclasses because now they:

The first item in this list may be a requirement for C code that expects a Python built-in class. The second item allows you to add new functionality on top of the standard list behavior. Finally, the third item will enable you to restrict the attributes of a subclass to only those attributes predefined in .__slots__.

To kick things off and start creating custom list-like classes, say that you need a list that automatically stores all its items as strings. Assuming that your custom list will store numbers as strings only, you can create the following subclass of list:

Your StringList class subclasses list directly, which means that it’ll inherit all the functionality of a standard Python list. Because you want your list to store items as strings, you need to modify all the methods that add or modify items in the underlying list. Those methods include the following:

The other methods that your StringList class inherited from list work just fine because they don’t add or update items in your custom list.

To use StringList in your code, you can do something like this:

Your class works as expected. It converts all the input values into strings on the fly. That’s cool, isn’t it? When you create a new instance of StringList, the class’s initializer takes care of the conversion.

When you append, insert, extend, or assign new values to the class’s instances, the methods that support each operation will take care of the string conversion process. This way, your list will always store its items as string objects.

Subclassing UserList From collections

Another way to create a custom list-like class is to use the UserList class from the collections module. This class is a wrapper around the built-in list type. It was designed for creating list-like objects back when it wasn’t possible to inherit from the built-in list class directly.

Even though the need for this class has been partially supplanted by the possibility of directly subclassing the built-in list class, UserList is still available in the standard library, both for convenience and for backward compatibility.

The distinguishing feature of UserList is that it gives you access to its .data attribute, which can facilitate the creation of your custom lists because you don’t need to use super() all the time. The .data attribute holds a regular Python list, which is empty by default.

Here’s how you can reimplement your StringList class by inheriting from UserList:

In this example, having access to the .data attribute allows you to code the class in a more straightforward way by using delegation, which means that the list in .data takes care of handling all the requests.

Now you almost don’t have to use advanced tools like super(). You just need to call this function in the class initializer to prevent problems in further inheritance scenarios. In the rest of the methods, you just take advantage of .data, which holds a regular Python list. Working with lists is a skill that you probably already have.

This new version works the same as your first version of StringList. Go ahead and run the following code to try it out:

Exposing .data is the most relevant feature of UserList, as you’ve already learned. This attribute can simplify your classes because you don’t need to use super() all the time. You can just take advantage of .data and use the familiar list interface to work with this attribute.

Coding List-Like Classes: Practical Examples

You already know how to use list and UserList when you need to create custom list-like classes that add or modify the standard functionality of list.

Admittedly, when you think of creating a list-like class, inheriting from list probably seems more natural than inheriting from UserList because Python developers know about list. They might not be aware of the existence of UserList.

You also know that the main difference between these two classes is that when you inherit from UserList, you have access to the .data attribute, which is a regular list that you can manipulate through the standard list interface. In contrast, inheriting from list requires advanced knowledge about Python’s data model, including tools like the built-in super() function and some special methods.

In the following sections, you’ll code a few practical examples using both classes. After writing these examples, you’ll be better prepared to select the right tool to use when you need to define custom list-like classes in your code.

A List That Accepts Numeric Data Only

As a first example of creating a list-like class with custom behavior, say that you need a list that accepts numeric data only. Your list should store only integer, float, and complex numbers. If you try to store a value of any other data type, like a string, then your list should raise a TypeError.

Here’s an implementation of a NumberList class with the desired functionality:

In this example, your NumberList class inherits directly from list. This means that your class shares all the core functionality with the built-in list class. You can iterate over instances of NumberList, access and update its items using their indices, call common list methods, and more.

Now, to ensure that every input item is a number, you need to validate each item in all the methods that support operations for adding new items or updating existing items in the list. The required methods are the same as in the StringList example back in the Inheriting From Python’s Built-In list class section.

To validate the input data, you use a helper method called ._validate_number(). This method uses the built-in isinstance() function to check if the current input value is an instance of int, float, or complex, which are the built-in classes representing numeric values in Python.

If the input value is an instance of a numeric data type, then your helper function returns the value itself. Otherwise, the function raises a TypeError exception with an appropriate error message.

To use NumberList, go back to your interactive session and run the following code:

In these examples, the operations that add or modify data in numbers automatically validate the input to ensure that only numeric values are accepted. If you add a string value to numbers, then you get a TypeError.

An alternative implementation of NumberList using UserList can look something like this:

In this new implementation of NumberList, you inherit from UserList. Again, your class will share all the core functionality with a regular list.

In this example, instead of using super() all the time to access methods and attributes in the parent class, you use the .data attribute directly. To some extent, using .data arguably simplifies your code compared to using super() and other advanced tools like special methods.

Note that you only use super() in the class initializer, .__init__(). This is a best practice when you’re working with inheritance in Python. It allows you to properly initialize attributes in the parent class without breaking things.

A List With Additional Functionality

Now say that you need a list-like class with all the standard functionality of a regular Python list. Your class should also provide some extra functionality borrowed from the Array data type of JavaScript. For example, you’ll need to have methods like the following:

Here’s a class that implements all these new features by subclassing list:

The .join() method in CustomList takes a separator character as an argument and uses it to concatenate the items in the current list object, which is represented by self. To do this, you use str.join() with a generator expression as an argument. This generator expression converts every item into a string object using str().

The .map() method returns a CustomList object. To construct this object, you use a generator expression that applies action() to every item in the current object, self. Note that the action can be any callable that takes an item as an argument and returns a transformed item.

The .filter() method also returns a CustomList object. To build this object, you use a generator expression that yields the items for which predicate() returns True. In this case, predicate() must be a Boolean-valued function that returns True or False depending on certain conditions applied to the input item.

Finally, the .for_each() method calls func() on every item in the underlying list. This call doesn’t return anything but triggers some side effects, as you’ll see below.

To use this class in your code, you can do something like the following:

In these examples, you first call .join() on words. This method returns a unique string that results from concatenating all the items in the underlying list.

The call to .map() returns a CustomList object containing uppercased words. This transformation results from applying str.upper() to all the items in words. This method works pretty similarly to the built-in map() function. The main difference is that instead of returning a list, the built-in map() function returns an iterator that yields transformed items lazily.

The .filter() method takes a lambda function as an argument. In the example, this lambda function uses str.startswith() to select those words that start with the "Py" prefix. Note that this method works similarly to the built-in filter() function, which returns an iterator instead of a list.

Finally, the call to .for_each() on words prints every word to the screen as a side effect of calling print() on each item in the underlying list. Note that the function passed to .for_each() should take an item as an argument, but it shouldn’t return any fruitful value.

You can also implement CustomList by inheriting from UserList rather than from list. In this case, you don’t need to change the internal implementation, just the base class:

Note that in this example, you just changed the parent class. There’s no need to use .data directly. However, you can use it if you want. The advantage is that you’ll provide more context to other developers reading your code:

In this new version of CustomList(), the only change is that you’ve replaced self with self.data to make it clear that you’re working with a UserList subclass. This change makes your code more explicit.

Considering Performance: list vs UserList

Up to this point, you’ve learned how to create your own list-like classes by inheriting from either list or UserList. You also know that the only visible difference between these two classes is that UserList exposes the .data attribute, which can facilitate the coding process.

In this section, you’ll consider an aspect that can be important when it comes to deciding whether to use list or UserList to create your custom list-like classes. That’s performance!

To evaluate if there are performance differences between classes that inherit from list vs UserList, you’ll use the StringList class. Go ahead and create a new Python file containing the following code:

These two classes work the same. However, they’re internally different. StringList_list inherits from list, and its implementation is based on super(). In contrast, StringList_UserList inherits from UserList, and its implementation relies on the internal .data attribute.

To compare the performance of these two classes, you should begin by timing standard list operations, such as instantiation. However, in these examples, both initializers are equivalent, so they should perform the same.

Measuring the execution time of new functionalities is also useful. For example, you can check the execution time of .extend(). Go ahead and run the following code:

In this performance test, you use the timeit module along with the min() function to measure the execution time of a piece of code. The target code consists of calls to .extend() on instances of StringList_list and StringList_UserList using some sample data.

The performance difference between the class based on list and the class based on UserList is mostly nonexistent in this example.

Often, when you create a custom list-like class, you’d expect subclasses of list to perform better than subclasses of UserList. Why? Because list is written in C and optimized for performance, while UserList is a wrapper class written in pure Python.

However, in the above example, it looks like this assumption isn’t completely right. For this reason, to decide which superclass is best for your specific use case, make sure to run a performance test.

Performance aside, inheriting from list is arguably the natural way in Python, mostly because list is directly available to Python developers as a built-in class. Additionally, most Python developers will be familiar with lists and their standard features, which will allow them to write list-like classes more quickly.

In contrast, the UserList class lives in the collections module, meaning that you’ll have to import it if you want to use it in your code. Additionally, not all Python developers are aware of the existence of UserList. However, UserList can still be a useful tool because of the convenience of accessing the .data attribute, which can facilitate the creation of custom list-like classes.

Conclusion

You’ve now learned how to create custom list-like classes with modified and new behaviors. To do this, you’ve subclassed the built-in list class directly. As an alternative, you’ve also inherited from the UserList class, which is available in the collections module.

Inheriting from list and subclassing UserList are both suitable strategies for approaching the problem of creating your own list-like classes in Python.

In this tutorial, you learned how to:

Now you’re better prepared to create your own custom lists, allowing you to leverage the full power of this useful and commonplace data type in Python.