Asyncio reader trick understanding (original) (raw)

April 17, 2025, 12:54pm 1

Hi,

I was going through the code in asyncio module at file Lib/asyncio/unix_event.py, it has the following chunk of code,

        # On AIX, the reader trick (to be notified when the read end of the
        # socket is closed) only works for sockets. On other platforms it
        # works for pipes and sockets. (Exception: OS X 10.4?  Issue #19294.)
        if is_socket or (is_fifo and not sys.platform.startswith("aix")):
            # only start reading when connection_made() has been called
            self._loop.call_soon(self._loop._add_reader,
                                 self._fileno, self._read_ready)

I was unable to understand what exactly is the reader trick that is being talked about here. can someone help me with understanding this code. Also I see in _UnixWritePipeTransport we call loop._add_reader to add call back _read_ready(self._loop.call_soon in the above line of code), what is the use of read ready in a WritePipeTransport, shouldn’t we be checking write readiness, I am new to asyncio so I may be misinterpreting things, please do correct me. Thank you in advance.

Hello,

this is how I interpret this disclaimer commentary:

From the comments, it is stating that for Unix based (AIX) operating systems, the reader trick is valid for sockets only. For all other operating systems, the reader trick is valid for both pipes and sockets. The reader trick is (per description in parentheses):

“to be notified when the read end of the socket is closed”

There is of course the exception to this per Exception: OS 10.4?, Issue #19294 as it pointed out in paranthesis.

So, if we were to view this in table format as to where the reader trick is valid, we can state it like this ( where
X = Valid
- = Not Valid):

         Sockets   PIPES
       -----------------
AIX    |   X     |  -
       |----------------
Other  |   X     |  X

Does this make sense?

Dislaimer on my part is that I have not used asyncio but from an English comprehension point of view, I believe that this is what it means.

zuhu2195 (Mohammed Zuhaib) April 18, 2025, 2:59pm 3

Thanks Paul. I want to know about the reader trick from code perspective. It has somethings to do with checking the read readiness of write end of a pipe I guess. I want to know the internals.

It explicitly states:

to be notified when the read end of the socket is closed

I don’t think that this means checking the read readiness. If this notification flag is raised, then it is telling the program that the socket is closed (terminated) - it does state this explicity: socket is closed, after all. Thus, to continue communicating, you would have to reinitiate the socket connection. This is how I am interpreting this.

zuhu2195 (Mohammed Zuhaib) April 29, 2025, 5:59am 5

I did some debugging and research on this, the asyncio event loop does internally involve checking read readiness on write end which I was mentioning. The parent process checks for read readiness on its write end to see whether the reader in the child closed. The write end of pipe is in parent and read end of pipe is in child. The trick can be demonstrated by code below,

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <errno.h>
#include <sys/time.h>

int main() {
    int pipefd[2];
    struct timeval timeout;
    timeout.tv_sec = 0;
    timeout.tv_usec = 0;
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }

    pid_t pid = fork();
    if (pid < 0) {
        perror("fork");
        exit(EXIT_FAILURE);
    }

    if (pid == 0) {
        // Child process
        close(pipefd[1]);  // Close write end
        printf("[Child] Sleeping for 20 seconds...\n");
        sleep(20);
        printf("[Child] Closing read end.\n");
        close(pipefd[0]);  // Close read end
        printf("[Child] Sleeping for 5 more second...\n");
        sleep(5);
        printf("[Child] is exiting now\n");
        exit(0);
    } else {
        // Parent process
        close(pipefd[0]);  // Close read end

        fd_set readfds;
        FD_ZERO(&readfds);
        FD_SET(pipefd[1], &readfds);  // Add write end to readfds
        printf("[Parent] Selecting for READ readiness on WRITE-END of pipe...\n");
        int ret = select(pipefd[1] + 1, &readfds, NULL, NULL, NULL);
        printf("return value is %d\n", ret);

        if (ret == -1) {
            perror("select");
        } else if (FD_ISSET(pipefd[1], &readfds)) {
            printf("[Parent] Write-end of pipe reported as READABLE.\n");
        } else {
            printf("[Parent] select returned but write-end not readable.\n");
        }

        close(pipefd[1]);
        wait(NULL);
    }

    return 0;
}

Whenever the child closes it’s read end the parents select system call unblocks and gives an indication. This timeout need not always be NULL, I have used it for demo, even a timeout of 0 can give an indication if child had exited and it’s reader side is closed provided the parent keeps checking it in event loop,

The output is as follows,

[Parent] Selecting for READ readiness on WRITE-END of pipe...
[Child] Sleeping for 20 seconds...
[Child] Closing read end.
[Child] Sleeping for 5 more second...
return value is 1
[Parent] Write-end of pipe reported as READABLE.
[Child] is exiting now

This is the crux of the reader trick which internally happens in asyncio.