Create temporary files in a temporary directory (original) (raw)

January 16, 2024, 10:55pm 1

Often, I find myself needing to create multiple temporary files that are related. It would be nice if tempfile.TemporaryDirectory could make that easier for me using something like this:

with tempfile.TemporaryDirectory() as temp_dir:
     fd1, temp_file1_path = temp_dir.mkstemp()
     fd2, temp_file2_path = temp_dir.mkstemp()
     # Do something with the files...

Which would be equivalent to:

with tempfile.TemporaryDirectory() as temp_dir:
     fd1, temp_file1_path = tempfile.mkstemp(dir=temp_dir)
     fd2, temp_file2_path = temp_dir.mkstemp(dir=temp_dir)
     # Do something with the files...

TemporaryDirectory should similarly provide a .mkdtemp().

This will work nice with how .mkstemp() doesn’t handle deleting the file since it will be cleaned up with the temporary directory if needed.

This way of adding it:

• Doesn’t support using NamedTemporaryFile.
• Doesn’t support extending this pattern with sub-directories using .mkdtemp() as it returns the absolute pathname and not another TemporaryDirectory.
• Doesn’t have very Pythonic method names.

So I also thought about having something like this:

with tempfile.TemporaryDirectory() as temp_dir:
    named_tempfile1 = temp_dir.named_temporary_file()
    sub_tempdir1 = temp_dir.temporary_directory()
     # Do something with the files...

Which would support all arguments supported by NamedTemporaryFile and TemporaryDirectory except those related to deletion and cleanup as they are handled by the top temporary directory.

This feels a bit less consistent with current tempfile functions but has less limitations.

Maybe this idea isn’t really useful since this doesn’t save many characters, but at least in my experience:

• It’s useful to created multiple temporary files and multiple nested temporary directories that organize things but ultimately all serve the same purpose.
• It’s nice not have to worry about deletion policies for the inner files, as you either get to keep the entire directory or none of it.

The changes needed to implemented this don’t seem very big, but I am willing (and happy!) to work on a PR for this if needed.

kknechtel (Karl Knechtel) January 16, 2024, 11:58pm 2

If we’re designing new functionality to be Pythonic, it should at least use Pythonic names, and not be beholden to ancient naming customs. I can’t begin to guess what the s in mkstemp should indicate, and I am only vaguely guessing that mk is short for “make”.

yoavdw (Yoav) January 17, 2024, 12:02am 3

That’s true. I wanted to be compatible with the current tempfile functions but I agree they are much less Pythonic. I’ll add a note about that.

barry-scott (Barry Scott) January 17, 2024, 11:36am 4

Create a file/directory that will not colide with a name that already exists is the reason to use the API.

Once you have a unique named directory that you control what is the use case for using the temporary files API within it?

You can create files with names that, for example, are descriptive of the contents if that makes debuging easier and by design will not collide.

yoavdw (Yoav) January 17, 2024, 7:25pm 5

I don’t think that’s the entire point of using temporary files.

The main point is that the files are temporary, only used for a specific part of your program, and don’t have to persist afterwards. You may create hundreds of temporary files, you may pass the temp dir object to inner functions which will create multiple files themselves which can cause collisions, but most importantly - a lot of the times you simply don’t have to worry about names, since they are only used for an intermediate purpose.

storchaka (Serhiy Storchaka) January 17, 2024, 8:53pm 6

with TemporaryDirectory() as tmpdir:
    ...
    with NamedTemporaryFile(dir=tmpdir) as named_tempfile1, \
         TemporaryDirectory(dir=tmpdir) as sub_tempdir1:
        ...

yoavdw (Yoav) January 20, 2024, 11:02am 8

I understand the use cases where this adds a lot in readability not be as common as I thought, and don’t justify a new function in the stdlib - can be closed.

davidism (David Lord) Closed January 20, 2024, 2:24pm 9

Closed at author’s request.