RFC-0011-InferenceMode by ailzhang · Pull Request #17 · pytorch/rfcs (original) (raw)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@albanD I kind of want a little more meat here, along the lines of the design principle @apaszke explicated at pytorch/pytorch#12502 (review)

One possible angle here is that no_grad is strictly local: the effects of this context manager affect what happens inside the block, but everything outside of the block is still fair game for full autograd support. (This is not as "hard" a design philosophy, but it's my best understanding right now.) Inference mode, on the other hand, is OK with letting the implications of this context escape; its goal is performance, and so if there is no way to implement something fast without affecting stuff outside of the inference mode block, we will just let it leak out.