platform identifiers don't account for arm64

[ddelange]

both the dependency speficier recommendation in the isal README, and the dependency specifier in xopen pyproject.toml, do not account for new macs:

>>> from packaging.markers import default_environment
>>> default_environment()
{
    'implementation_name': 'cpython',
    'implementation_version': '3.12.8',
    'os_name': 'posix',
    'platform_machine': 'arm64',
    'platform_release': '24.6.0',
    'platform_system': 'Darwin',
    'platform_version': 'Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:40 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6041',
    'python_full_version': '3.12.8',
    'platform_python_implementation': 'CPython',
    'python_version': '3.12',
    'sys_platform': 'darwin'
}

hence it does not install isal with xopen:

$ pip install xopen
Collecting xopen
  Downloading xopen-2.0.2-py3-none-any.whl.metadata (15 kB)
Downloading xopen-2.0.2-py3-none-any.whl (17 kB)
Installing collected packages: xopen
Successfully installed xopen-2.0.2
$ pip install isal
Collecting isal
  Downloading isal-1.8.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (10 kB)
Downloading isal-1.8.0-cp312-cp312-macosx_11_0_arm64.whl (189 kB)
Installing collected packages: isal
Successfully installed isal-1.8.0

[ddelange]

why do you add/recommend the platform specifiers in the first place? better to let pip attempt a source install instead? and fix the sdist or add wheels if users run into problems?

[rhpvorderman]

Sorry for answering this so late. I have been out of commission for health stuff for a while. (Still limited in my work time, there will be more delays).

Thank you for telling me about the arm64 platform. I did not know that was a thing. MacOS arm64 should work. I can add that platform to the list. I do not use a MacOS machine myself so I often miss these things.

For why it is this way: ISA-L is not easy to install from source. You need a working C compiler in your environment, which is something only Linux users will have by default. Secondly you need to have a working version of nasm in your environment, which no one has by default. Except me, but I am the python-isal maintainer so that does not count. (Okay granted, maybe other people who write in x86 assembly all day, but I am going to assume that is quite the minority in the audience).

As a result all that will happen is that people will face install errors. And then they will go and complain. To me, even though I listed all requirements on the README. But they will have not read that README because they were installing xopen and do not care about its dependencies (rightfully so!). So they go straight to the python isal issues page and complain that they cannot install it.

So python-isal now has this on the README: https://github.com/pycompression/python-isal#python-isal-as-a-dependency-in-your-project

I shall update that statement to reflect other platforms as well.

[ddelange]

yes, that's very typical for the python ecosystem. two things I see in the wild that could be a path forward for isal without needing these nasty platform specifiers everywhere downstream:

1. don't publish .tar.gz to PyPI, remove platform specifiers, users on unsupported platforms will be greeted with an error (see pytorch for example), source install will require people to go to github
2. keep .tar.gz, remove platform specifiers, and publish all thinkable wheels (and add more when people complain about their esoteric platform). I maintain blake3-py for instance and this was our path forward (though we kept .tar.gz too), see https://pypi.org/project/blake3/#files. be sure to sync the python_requires with the actual python versions you build wheels for. alternatively, for c bindings especially, you can build ABI3 wheels which are 3.8+ forward compatible wheels, no need to publish new wheels when a new python version comes out provided that python_requires doesnt have an upper bound.

[rhpvorderman]

1. This does not change a thing. Users on unsupported platforms are greeted with an error now too. I'd rather have the specifiers in xopen so they do not get an error at all.
2. ISA-L is an Assembly code library, not a C code library. As a result some targets are simply unavailable. I wish I could build wheels for everybody, but that is simply impossible.

The specifiers are not ideal, but the best option at the moment. I have made an issue at python-isal to update the specifier to include the platforms for which builds are available.

[ddelange]

both my options would allow xopen to remove platform specifiers. current specifiers in xopen will silently skip installing isal on my system although it's available upstream (see original message above).

so at least a PR for xopen is required here with better specifiers to unblock installation on M-chip macs

[rhpvorderman]

both my options would allow xopen to remove platform specifiers.

No they won't. ISA-L is not necessary for xopen to function, and requiring its installation would disable users from installing it who would otherwise be able to.

current specifiers in xopen will silently skip installing isal on my system although it's available upstream (see original message above).

Yes. That is a problem. It will be fixed, don't worry. The reason that this issue exists in the first place is that it is very hard to publish wheels for platforms that do not have github action runners available. And MacOS arm64 runners were not available for a very long time.

[rhpvorderman]

Unfortunately another user pointed out that python-isal is slower on MacOS arm64. See pycompression/python-isal#254.

This is due to faulty compile defaults. So I changed the compile process back to the one that is used on conda-forge as well. Which does enable the correct optimizations. Unfortunately this does not work with MacOS for some reason. See pycompression/python-isal#255.

The error suggests using certain flags, but also that crashes the CI. At this point I don't know how to fix it. I don't own a Mac, errors aren't helpful. I can't do more without massively wasting my time I am affraid.

[rhpvorderman]

As a workaround: the conda package of xopen should work and should correctly load a correctly build ISA-L.