[Python-Dev] Idea: more compact, interned string key only dict for namespace. (original) (raw)

INADA Naoki songofacandy at gmail.com
Fri Jun 24 13:51:16 EDT 2016

Hi, all.

I implemented my new idea. (still wip) https://github.com/methane/cpython/pull/3/files

Memory usage when building Python doc with sphinx are:

  1. master (shared key) 176MB

  2. compact (w/ shared key) 158MB

  3. compact (w/o shared key) 166MB

  4. compact & interned (new) 160MB

Memory usage is close to compact w/ shared key, and more efficient than current.

In case of Python benchmark (master vs compact & interned):

$ python perf.py -fm ~/local/python-master/bin/python3 ~/local/python-intern/bin/python3

2to3

Mem max: 20392.000 -> 16936.000: 1.2041x smaller

chameleon_v2

Mem max: 364604.000 -> 359904.000: 1.0131x smaller

django_v3

Mem max: 26648.000 -> 24948.000: 1.0681x smaller

fastpickle

Mem max: 8296.000 -> 8996.000: 1.0844x larger

fastunpickle

Mem max: 8332.000 -> 7964.000: 1.0462x smaller

json_dump_v2

Mem max: 10400.000 -> 9972.000: 1.0429x smaller

json_load

Mem max: 8088.000 -> 7644.000: 1.0581x smaller

nbody

Mem max: 7460.000 -> 7036.000: 1.0603x smaller

regex_v8

Mem max: 12572.000 -> 12520.000: 1.0042x smaller

tornado_http

Mem max: 27860.000 -> 26792.000: 1.0399x smaller

I'll do more hack in next week to prove my idea. (interned string only vs string only, revive embedded small table or not).

If someone interested in, please try my interned-dict branch and report difference of performance and memory usage.

https://github.com/methane/cpython/tree/interned-dict (cb0a125c79 passes most tests, except tests using sys.getsizeof()).

-- INADA Naoki <songofacandy at gmail.com>

More information about the Python-Dev mailing list