STYLE enable pylint: method-cache-max-size-none by natmokval · Pull Request #49751 · pandas-dev/pandas (original) (raw)

I looked into this a bit more closely (and trying to repeat what Anthony does in the video I linked), and I think there's a real issue here

For this investigation, I applied the following diff:

diff --git a/pandas/io/formats/excel.py b/pandas/io/formats/excel.py index a26b85390f..3a8a9ee960 100644 --- a/pandas/io/formats/excel.py +++ b/pandas/io/formats/excel.py @@ -170,6 +170,9 @@ class CSSToExcelConverter: self.inherited = self.compute_css(inherited) else: self.inherited = None

def del(self) -> None:

   print('heeeelp I\'m being deeeleteeeed')

compute_css = CSSResolver()

@@ -193,6 +196,7 @@ class CSSToExcelConverter: A style as interpreted by ExcelWriter when found in ExcelCell.style. """

   print('I\'m being computed (not looked up in cache)')
   properties = self.compute_css(declarations, self.inherited)
   return self.build_xlstyle(properties)

Once converter.__call__ has been called garbage collection no longer deletes converter!

In [1]: from pandas.io.formats.excel import CSSToExcelConverter

In [2]: import gc

In [3]: converter = CSSToExcelConverter("font-weight: bold")

In [4]: converter('vertical-align: top') I'm being computed (not looked up in cache) Out[4]: {'alignment': {'vertical': 'top'}, 'font': {'bold': True}}

In [5]: converter = None

In [6]: gc.collect() # converter doesn't get deleted! Out[6]: 373

However, if we then apply this trick from the video:

diff --git a/pandas/io/formats/excel.py b/pandas/io/formats/excel.py index a26b85390f..692dd23e95 100644 --- a/pandas/io/formats/excel.py +++ b/pandas/io/formats/excel.py @@ -170,12 +170,20 @@ class CSSToExcelConverter: self.inherited = self.compute_css(inherited) else: self.inherited = None

   self._call_cached = lru_cache(maxsize=None)(self._call_uncached)

compute_css = CSSResolver()

@lru_cache(maxsize=None) def call( self, declarations: str | frozenset[tuple[str, str]]

) -> dict[str, dict[str, str]]:

   return self._call_cached(declarations)

def _call_uncached(
```
   self, declarations: str | frozenset[tuple[str, str]]
```
) -> dict[str, dict[str, str]]: """ Convert CSS declarations to ExcelWriter style.

@@ -193,6 +201,7 @@ class CSSToExcelConverter: A style as interpreted by ExcelWriter when found in ExcelCell.style. """ properties = self.compute_css(declarations, self.inherited) return self.build_xlstyle(properties)

then it looks like it solves the issue:

In [14]: converter = CSSToExcelConverter("font-weight: bold")

In [15]: converter('vertical-align: top') I'm being computed (not looked up in cache) Out[15]: {'alignment': {'vertical': 'top'}, 'font': {'bold': True}}

In [16]: converter('vertical-align: top') # caching still works Out[16]: {'alignment': {'vertical': 'top'}, 'font': {'bold': True}}

In [17]: converter = None

In [18]: gc.collect() # now, it gets deleted! heeeelp I'm being deeeleteeeed Out[18]: 698