[Python-Dev] The untuned tunable parameter ARENA_SIZE (original) (raw)

INADA Naoki songofacandy at gmail.com
Thu Jun 1 05:21:55 EDT 2017

Thanks for detailed info.

But I don't think it's a big problem. Arenas are returned to system by chance. So other processes shouldn't relying to it.

And I don't propose to stop returning arena to system. I just mean per pool (part of arena) MADV_DONTNEED can reduce RSS.

If we use very large arena, or stop returning arena to system, it can be problem.

Regards,

On Thu, Jun 1, 2017 at 6:05 PM, Siddhesh Poyarekar <siddhesh at gotplt.org> wrote:

On Thursday 01 June 2017 01:53 PM, INADA Naoki wrote:

* On Linux, madvice(..., MADVDONTNEED) can be used. madvise does not reduce the commit charge in the Linux kernel, so in high consumption scenarios (and where memory overcommit is disabled or throttled) you'll see programs dying with OOM despite the MADVDONTNEED. The way we solved it in glibc was to use mprotect to drop PROTREAD and PROTWRITE in blocks that we don't need when we detect that the system is not configured to overcommit (using /proc/sys/vm/overcommitmemory). You'll need to fix the protection again though if you want to reuse the block. Siddhesh

More information about the Python-Dev mailing list