ENH: Add method='table' for EWM.mean by mroeschke · Pull Request #42273 · pandas-dev/pandas (original) (raw)
@stuartarchibald if you have time, could you review if this function looks if I have any bad practices in this function. Some context:
- Generally this is run with
@numba.jit(nopython=True, nogil=False, parallel=False) - This function has variables from a global scope
com,adjust,ignore_na,deltas - I cache this function in a dictionary this first time it runs (in
NUMBA_FUNC_CACHE) and later calls will return the cached fuction.
When using this function, sometimes I get a segfault:
(pandas-dev) matthewroeschke@x86_64-apple-darwin13 pandas-mroeschke % ipython
Python 3.8.6 | packaged by conda-forge | (default, Dec 26 2020, 04:50:20)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: df = DataFrame(np.random.rand(3, 30))
In [2]: ewm = df.ewm(com=1, method="table")
In [3]: ewm.mean(engine="numba")
Out[3]:
0 1 2 3 4 ... 25 26 27 28 29
0 0.770949 0.396205 0.728331 0.417464 0.715142 ... 0.237903 0.383182 0.021471 0.837933 0.590543
1 0.417231 0.369916 0.793127 0.961175 0.531131 ... 0.145222 0.152615 0.714216 0.515145 0.238283
2 0.441215 0.220925 0.868065 0.736689 0.858115 ... 0.165413 0.977209 0.549496 0.472534 0.555430
[3 rows x 30 columns]
In [4]: ewm.mean(engine="numba")
Out[4]:
0 1 2 3 4 ... 25 26 27 28 29
0 0.770949 0.396205 0.728331 0.417464 0.715142 ... 0.237903 0.383182 0.021471 0.837933 0.590543
1 0.417231 0.369916 0.793127 0.417464 0.531131 ... 0.145222 0.152615 0.714216 0.515145 0.238283
2 0.441215 0.220925 0.868065 0.417464 0.858115 ... 0.165413 0.977209 0.549496 0.472534 0.555430
[3 rows x 30 columns]
In [5]: ewm.mean(engine="numba")
Out[5]:
0 1 2 3 4 ... 25 26 27 28 29
0 0.770949 0.396205 0.728331 0.417464 0.715142 ... 0.237903 0.383182 0.021471 0.837933 0.590543
1 0.417231 0.369916 0.793127 0.961175 0.531131 ... 0.145222 0.152615 0.714216 0.515145 0.238283
2 0.441215 0.220925 0.868065 0.736689 0.858115 ... 0.165413 0.977209 0.549496 0.472534 0.555430
[3 rows x 30 columns]
In [6]: ewm.mean(engine="numba")
python(3345,0x111f53dc0) malloc: Incorrect checksum for freed object 0x7fd3255c55a8: probably modified after being freed.
Corrupt value: 0x3ff0000000000000
python(3345,0x111f53dc0) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort ipython