ENH: Implement pandas.read_iceberg by datapythonista · Pull Request #61383 · pandas-dev/pandas (original) (raw)

Curious if there is an open issue discussing including this new feature.

I think only the discussions you are aware of, bodo-ai/Bodo-Pandas-Collaboration#9 and the discussions in the calls that I know.

I had a look at using read_sql instead of read_iceberg, and to me it feels like the API would be too difficult to use. Considering the popularity of Iceberg, and that we already have specific connectors for much less popular formats such as feather, SPSS... I found this the most reasonable implementation. But happy to give a try at using the code in this PR with read_sql instead, if there is interest. But it also feels that our code will be more complex, so personally I don't see an advantage.

I'm also happy to revisit PDEP-9. There weren't objections to the general idea that I remember, the main blocker was that some people weren't happy that connectors could register with the name they wanted. And I don't think there is a good solution to this. To me it's not a problem, since at the end is the user who decides which Python dependencies are installed. In any case, to me it makes sense to move forward with this Iceberg, and surely this would be a good candidate to move as a third party with many others if we ever implement PDEP-9.