[Python-Dev] Support of the Android platform (original) (raw)

Xavier de Gaye xdegaye at gmail.com
Sun Dec 10 09:19:11 EST 2017

• Previous message (by thread): [Python-Dev] Proposed schedule for next 3.4 and 3.5 releases - end of January / early February
• Next message (by thread): [Python-Dev] Support of the Android platform
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

The following note is a proposal to add the support of the Android platform.

The note is easier to read with clickable links at https://github.com/xdegaye/cagibi/blob/master/doc/android_support.rst

Motivations

• Android is ubiquitous.
• This would be the first platform supported by Python that is cross-compiled, thanks to many contributors.
• Although the Android operating system is linux, it is different from most linux platforms, for example it does not use GNU libc and runs SELinux in enforcing mode. Therefore supporting this platform would make Python more robust and also would allow testing it on arm 64-bit processors.
• Python running on Android is also a handheld calculator, a successor of the slide rule and the HP 41_.

Current status

• The Python test suite succeeds when run on Android emulators using buildbot strenuous settings with the following architectures on API 24: x86, x86_64, armv7 and arm64.
• The Android build system_ is described in another section.
• The buildmaster-config PR 26_ proposes to update master.cfg to enable buildbots to run a given Android API and architecture on the emulators.
• The Android emulator is actually qemu, so the test suites for x86 and x86_64 last about the same time as the test suite run natively when the processor of the build system is of the x86 family. The test suites for the arm architectures last much longer: about 8 hours for arm64 and 10 hours for armv7 on a four years old laptop.
• The changes that have been made to achieve this status are listed in bpo-26865_, the Android meta-issue.
• Given the cpu resources required to run the test suite on the arm emulators, it may be difficult to find a contributed buildbot worker. So it remains to find the hardware to run these buildbots.

Proposal

Support the Android platform on API 24 [1]_ for the x86_64, armv7 and arm64 architectures built with NDK 14b.

API 24

• API 21 is the first version to provide usable support for wide characters and where SELinux is run in enforcing mode.

• API 22 introduces an annoying bug on the linker that prints something like this when python is started::

  WARNING: linker: libpython3.6m.so.1.0: unused DT entry: type 0x6ffffffe arg 0x14554.

  The termux_ Android terminal emulator describes this problem at the end of its termux-packages_ gitlab page and has implemented a termux-elf-cleaner tool to strip the useless entries from the ELF header of executables.

• API 24 is the first version where the adb_ shell is run on the emulator as a shell user instead of the root user previously, and the first version that supports arm64.

x86_64 It seems that no handheld device exists using that architecture. It is supported because the x86_64 Android emulator runs fast and therefore is a good candidate as a buildbot worker.

NDK 14b This release of the NDK is the first one to use Unified headers_ fixing numerous problems that had been fixed by updating the Python configure script until now (those changes have been reverted by now).

Android idiosyncrasies

• The default shell is /system/bin/sh.
• The file system layout is not a traditional unix layout, there is no /tmp for example. Most directories have user restricted access, /sdcard is mounted as noexec for example.
• The (java) applications are allocated a unix user id and a subdirectory on /data/data.
• SELinux is run in enforcing mode.
• Shared memory and semaphores are not supported.
• The default encoding is UTF-8.

Android build system

The Android build system is implemented at bpo-30386_ with PR 1629_ and is documented by its README_. It provides the following features:

• To build a distribution for a device or an emulator with a given API level and a given architecture.
• To start the emulator and
  • install the distribution
  • start a remote interactive shell
  • or run remotely a python command
  • or run remotely the buildbottest
• Run gdb on the python process that is running on the emulator with python pretty-printing.

The build system adds the Android/ directory and the configure-android script to the root of the Python source directory on the master branch without modifying any other file. The build system can be installed, upgraded (i.e. the SDK and NDK) and run remotely, through ssh for example.

The following external libraries, when they are configured in the build system, are downloaded from the internet and cross-compiled (only once, on the first run of the build system) before the cross-compilation of the extension modules:

• ncurses
• readline
• sqlite
• libffi
• openssl, the cross-compilation of openssl fails on x86_64 and arm64 and this step is skipped on those architectures.

The following extension modules are disabled by adding them to the *disabled* section of Modules/Setup:

• _uuid, Android has no uuid/uuid.h header.
• grp some grp.h functions are not declared.
• _crypt, Android does not have crypt.h.
• _ctypes on x86_64 where all long double tests fail (bpo-32202) and on arm64 (see bpo-32203).

.. [1] On Wikipedia Android version history_ lists the correspondence between API level, commercial name and version for each release. It also provides information on the global Android version distribution, see the two charts on top.

.. _README: https://github.com/xdegaye/cpython/blob/bpo-30386/Android/README.rst .. _bpo-26865: https://bugs.python.org/issue26865 .. _bpo-30386: https://bugs.python.org/issue30386 .. _bpo-32202: https://bugs.python.org/issue32202 .. _bpo-32203: https://bugs.python.org/issue32203 .. _PR 1629: https://github.com/python/cpython/pull/1629 .. _buildmaster-config PR 26: https://github.com/python/buildmaster-config/pull/26 .. _Android version history: https://en.wikipedia.org/wiki/Android_version_history .. _termux: https://termux.com/ .. _termux-packages: https://gitlab.com/jbwhips883/termux-packages .. _adb: https://developer.android.com/studio/command-line/adb.html .. _Unified headers: https://android.googlesource.com/platform/ndk.git/+/ndk-r14-release/docs/UnifiedHeaders.md .. _HP 41: https://en.wikipedia.org/wiki/HP-41C .. vim:filetype=rst:tw=78:ts=8:sts=2:sw=2:et:

• Previous message (by thread): [Python-Dev] Proposed schedule for next 3.4 and 3.5 releases - end of January / early February
• Next message (by thread): [Python-Dev] Support of the Android platform
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the Python-Dev mailing list