ENH: numba engine in df.apply by lithomas1 · Pull Request #54666 · pandas-dev/pandas (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation9 Commits12 Checks0 Files changed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
- closes #xxxx (Replace xxxx with the GitHub issue number)
- Tests added and passed if fixing a bug or adding a new feature
- All code checks passed.
- Added type annotations to new arguments/methods/functions.
- Added an entry in the latest
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.
gentle ping @mroeschke.
I'm planning on following this up with general support for numba in df.apply.
| @@ -9919,6 +9919,8 @@ def apply( |
|---|
| result_type: Literal["expand", "reduce", "broadcast"] | None = None, |
| args=(), |
| by_row: Literal[False, "compat"] = "compat", |
| engine: str = "python", |
| engine_kwargs: dict = {}, |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we usually default this as engine_kwargs: dict[str, bool] | None = None
| @@ -9919,6 +9919,8 @@ def apply( |
|---|
| result_type: Literal["expand", "reduce", "broadcast"] | None = None, |
| args=(), |
| by_row: Literal[False, "compat"] = "compat", |
| engine: str = "python", |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you type as a Literal here?
| else: |
|---|
| first_elem = values[0] |
| dim0 = values.shape[0] |
| res0 = nb_compat_func(first_elem) |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is inferring the shape from the first element similar to what we do for DataFrame.apply?
It would be good to note what type of UDFs are supported in the engine docstring
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mroeschke pushed a commit to mroeschke/pandas that referenced this pull request
ENH: numba engine in df.apply
fixes
more fixes
try to fix
address code review
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
go for green
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
- update type
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This was referenced
Mar 15, 2025
Labels
Apply, Aggregate, Transform, Map
numba-accelerated operations
2 participants
