[Python-Dev] AIX to stable, what does that take? (original) (raw)
Michael Felt aixtools at felt.demon.nl
Thu Oct 4 05:13:51 EDT 2018
- Previous message (by thread): [Python-Dev] AIX to stable, what does that take?
- Next message (by thread): [Python-Dev] AIX to stable, what does that take?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On 10/4/2018 10:30 AM, INADA Naoki wrote:
Hello,
First of all, congratulations on passing all test on AIX.
My assumption is that it needs to (at least) pass all tests - and that is why I keep asking for attention. All the PRs to fix individual tests mean less if they are not merged, for whatever reason.
However, maybe there is another way, or even something additional needed. Maybe something I cannot provide and then I can adjust my expectations and goals. As a one of core developer, I don't know anything about AIX. If my change breaks AIX build, I can't investigate what's happened. So I think we need following in devguide: * Brief description about AIX, from developer's point of view. This I might be able to do. Bullet form:
- Committed to POSIX standard (valid when release came out, so AIX 5.3 confirms to a different standard than AIX 7.2)
- While Linux affinity is recognized - GNU (or GNP - GNU not POSIX) integration is not guaranteed. - GNU rte is not provided under support. There is a so-called Toolbox, GNU an other OSS utilities supplied by many packaged as RPMs. Unfortunately, different RPM providers (Michael Perlz, BULL Freeware, IBM, and others) have different numbering (the part after the package version, e.g., python-2.7.10-XXX so they do not mix well). Headache for both admins and developers trying to develop in a GNU-like environment.
- As a consultant, fedup with what is called the "RPM hell" by many AIX admins - I do not use any RPMs. I build everything myself, using xlc (gcc introduces the need for a GNU RTE, e.g., glibc). I package using installp (the "native") AIX package manager, and strive to make the packages independent (these days). When there are dependencies I try to build them as static libraries so that they do not become an additional install dependency.
- finally, a bit deeper: while the AIX linker loader supports svr4 shared libraries (it is the data, not the file name) it also supports having multiple shared libraries in a classic archive. So, rather that .../lib/libxxx.so and .../lib64/libxxx.so AIX prefers .../lib/libxxx.a with two so-called members, with same or different names. The one found is not it's name, but the symbol name and size of the ABI (32-bit or 64-bit)
- Hope that is enough of the key issues for now.
** In general, GNU autotools (autoreconf -f -v) works well, as does
configure.ac et al. for createing OSS Makefiles.
* How to run AIX on (VirtualBox, AWS EC2, Azure, GCP) easily. Easily! ? :) - well, on a POWER server it was easy enough for me to follow the step by step instructions for creating a buildbot. If I had a POWER server with more resources I would examine using the AIX internal WPAR - however, a POWER server configured to use PowerVC uses "EC2" aka cloud-init for creating a virtual machine. With that environment it should be "easy" to provide additional instructions to cloud-init-ec2.
Or, I provide you a login on my personal server that I run the buildbot on. etc. etc. - Where there is a will, there is a way.
* How to set up a development environment for Python. Again, follow the instructions for setting up a buildbot. * How to build Python. git clone ... autoreconf -v -f (as needed) ./configure --with-pydebug #gcc compiler ./configure --with-pydebug --without-computed-gotos # xlc compiler make make test * How to debug C code. I learned, 40 years ago, using adb (a debugger) - I do a lot of single-stepping. gdb is not the default debugger. If I were a developer I would probably dig into the AIX debuggers (there are at least two, kdb (kernel debugger, which I do use occaisionally for performance issues) and another I try to avoid. I add fprintf statements and am looking at learning how to use probevue.
In short, you probably have many much better ideas on how to debug C than I do :)
And even though there is a developer guide, it will take more long time than fixing issues on AIX, compared Linux, macOS, and Windows. But without this guide, it feels almost impossible to maintain AIX build to me. IMHO: The AIX build is stable, but this is unrecognized because it does have differences that cause tests to fail. I can think of one test that PASSes, but should fail. And another test that passes, but should have failed (in test_uuid) I have submitted a PR.
I tried to fix "all" in one PR, which confused people - so redid it as two (got _uuid working in Python 3.7 ! yes!!) but the "original" to fix uuid.py and test_uuid.py is still "awaiting change review".
My gut feeling to maintaining AIX is: a) all test pass so a potential regression is flagged; b) someone such as myself who knows the platform and can establish a "root cause" on why it is failing with AIX so that c) a developer becomes aware and can decide to ignore or adjust; d) alternatives - such as work around an implementation limitation (as I try to do, e.g., for test_time and the related _datetime.c) is yet another path.
In other words - it needs to be a shared responsibility - some volunteers with a passion for platform stability (in this specific case AIX and me as "passionate-person" and perhaps someone as youself who wants to focus on the language itself - ideally without deep (or any!) concern for platform differences.
Regards, My 6 bits!
- Previous message (by thread): [Python-Dev] AIX to stable, what does that take?
- Next message (by thread): [Python-Dev] AIX to stable, what does that take?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]