SAS7BDAT parser: Improve subheader lookup performance by jonashaag · Pull Request #47656 · pandas-dev/pandas (original) (raw)

Avoid constructing _SubheaderPointer objects and make dictionary lookups in C rather than in Python.

Speedup relative to current main:

     <main>           <sas/shlookup~1>
-     8.32±0.07ms      7.51±0.06ms     0.90  io.sas.SAS.time_test_meta2_pagesas7bdat
-      82.8±0.5ms       73.6±0.5ms     0.89  io.sas.SAS.time_read_sas7bdat_2_chunked
       before           after         ratio

Will extend what's new from #47404 once that's merged.

• closes #xxxx (Replace xxxx with the Github issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.