cmake: Add ENABLE_MULTILIB option to build a combined 8/10/12-bit library

[QShen3]

Summary

This PR adds an ENABLE_MULTILIB CMake option that builds the combined
multi-bit-depth library in a single cmake --build invocation, following the
recommendation in the
multi-library interface docs
and the existing build/linux/multilib.sh / vc*-x86_64/multilib.bat
scripts.

With -DENABLE_MULTILIB=ON:

• Two build targets (x265-multilib-10bit, x265-multilib-12bit) configure
  and build the 10-bit and 12-bit static libraries (EXPORT_C_API=OFF,
  namespaces x265_10bit / x265_12bit) into <build>/multilib/.
• The main library links them through the existing EXTRA_LIB /
  LINKED_10BIT / LINKED_12BIT mechanism, so the C API dispatches to the
  requested bit depth at runtime (x265_api_get(8/10/12)).
• For static builds, a POST_BUILD step merges the three archives into a single
  libx265.a / x265-static.lib using GNU ar (MRI), libtool (macOS) or
  lib.exe (MSVC), so consumers link one library.
• Shared builds embed the 10/12-bit archives via EXTRA_LIB as before.

New files

• source/cmake/multilib.cmake — defines the sub-build targets, inheriting
  the generator/toolchain/compiler settings from the parent build.
• source/cmake/x265-merge.cmake — cmake -P static archive merge script.

Testing

• Linux x86_64 (gcc): combined static lib, CLI and x265_api_get(8/10/12)
  all verified, assembly enabled.
• Windows x64 (MSVC): static and dynamic builds verified.
• Android x64: cross-compilation verified (ANDROID_ABI propagated to the
  nested builds).

Note

This is also being integrated into the vcpkg port as an opt-in multilib
feature.

[Akilan-Sivakumar]

Tested ENABLE_MULTILIB=ON across Linux, macOS (arm64), and Windows (MSVC).

Windows build fails

On MSVC, the nested sub-build archives land in a per-config subfolder (multilib\10bit\Release\x265-static.lib), but multilib.cmake hardcodes the expected path without it. EXTRA_LIB ends up pointing at a path that doesn't exist:

LINK : fatal error LNK1181: cannot open input file 'multilib\10bit\x265-static.lib'

Both x265-shared and cli fail to link. This contradicts this PR's testing note that Windows static/dynamic builds were verified.

Other correctness issues

• add_custom_command(OUTPUT ...) for the 10/12-bit sub-builds has no DEPENDS on source files, so once the archive exists, the outer build never re-invokes the nested build - editing source files won't trigger a rebuild.
• set(EXTRA_LIB ... CACHE STRING ... FORCE) silently overwrites any pre-existing EXTRA_LIB a user may have set.

Minor findings

• Linux & macOS: nested sub-builds don't inherit the parent Make jobserver (-j100 -> sub-builds run at -j1).
• macOS: the libtool -static merge step spams hundreds of "duplicate member name" warnings, since all 3 sub-builds produce identically-named object files. Harmless to the result, but noisy.
• Unused-variable CMake warnings for CMAKE_DISABLE_FIND_PACKAGE_VLD (Linux) and ENABLE_LIBNUMA (Windows) - passed to sub-builds unconditionally even where not applicable.
• Dead return() guard at top of multilib.cmake (unreachable).

[QShen3]

MSVC + Ninja works but MSVC + VS generators fails. I will fix it

[QShen3]

• Windows (MSVC) build failure — fixed
• Stale nested builds (no DEPENDS on sources) — fixed
• Unused-variable warnings — fixed
• EXTRA_LIB overwritten with FORCE — ENABLE_MULTILIB is an explicit opt-in; the combined library requires EXTRA_LIB to point at the sub-build archives (and LINKED_10BIT/LINKED_12BIT to enable the dispatch), so this is intentional. I can guard it if you'd prefer the user's value to win.
• Parallelism — improved. -j100 can not be forward in build stage
• macOS libtool "duplicate member name" warnings — cosmetic; the merged archive is correct. Could be silenced if preferred.
• Dead return() guard — removed

[Akilan-Sivakumar]

Thanks for the fixes. A few items remain.

Android / vcpkg propagation:
The rewrite dropped the automatic VCPKG_*/ANDROID_* forwarding, replaced by manual MULTILIB_CMAKE_ARGS. This PR's "Android verified" testing note was against the old auto-forwarding code - could you re-test Android against the current commit, and confirm the vcpkg port PR passes MULTILIB_CMAKE_ARGS through?

EXTRA_LIB:
Please guard the FORCE overwrite - if a user already had EXTRA_LIB set, silently discarding it could cause a confusing failure elsewhere.

macOS libtool warnings:
Please silence them since they don't affect the build's correctness.

VERSION / NASM_EXECUTABLE:
Both were forwarded to sub-builds before but are missing now, with no mention in the fix summary - was this intentional, or dropped during cleanup?

Parallelism:
Only helps when CMAKE_BUILD_PARALLEL_LEVEL is explicitly set - plain make -j100 won't propagate. Unlike multilib.sh, which builds sequentially, these sub-builds run concurrently with no ordering between them, so giving each the full core count would oversubscribe the machine. I'd suggest building 10-bit and 12-bit sequentially instead, like multilib.sh - which keeps the build log clean and easy to follow instead of interleaved output from two concurrent sub-builds.

[Akilan-Sivakumar]

Hi @QShen3.

Thanks for the fixes on this round - good progress overall.

Two things from before are still unaddressed: could you take a look at the Android/vcpkg propagation and VERSION/NASM_EXECUTABLE forwarding questions?

A few more findings:

1. ENABLE_SHARED used before it's defined

source/CMakeLists.txt:1089 checks NOT ENABLE_SHARED before option(ENABLE_SHARED ...) declares it. An undefined variable reads as false in CMake, so the static merge step runs even when a shared build is also being built. Can you move option(ENABLE_SHARED ...) above line 1089. This is a pre-existing ordering issue, not something you introduced - this PR is just the first thing to read it early.

2. Parallelism + log readability

I'd suggest include(ProcessorCount); ProcessorCount(_multilib_ncpu) as a fallback so sub-builds get a real -jN instead of -j1. Also, the log looks messy since the main 8-bit compile and the 10/12-bit sub-builds run concurrently and interleave - multilib.sh avoided this by building sequentially.

3. Unnecessary per-config warnings on single-config generators

multilib.cmake:93-102 always passes all four per-config CMAKE_ARCHIVE_OUTPUT_DIRECTORY_* variants, but those only matter for multi-config generators (VS, Xcode). On Unix Makefiles, only the generic + active-config ones are used - the rest just warn as unused.

4. Stale cache when toggling ENABLE_MULTILIB off in an existing build dir

multilib.cmake:121-129 force-writes EXTRA_LIB, LINKED_10BIT, LINKED_12BIT when ENABLE_MULTILIB=ON, but nothing resets them when it's turned back OFF in the same build dir. Verified directly - reconfiguring with OFF leaves EXTRA_LIB still pointing at the multilib archives, and LINKED_10BIT/LINKED_12BIT still ON.

[kvnloo]

@QShen3 Looking at b7dd75bc — sequential 10→12 chaining and the EXTRA_LIB overwrite guard look good.

A few remaining CMake nits (small / cherry-pickable):

1. ENABLE_SHARED order — if(ENABLE_MULTILIB AND NOT ENABLE_SHARED) runs before option(ENABLE_SHARED ...). Undefined reads as false, so the static merge POST_BUILD still fires when shared is also built. Move option(ENABLE_SHARED ...) above that block (pre-existing ordering; this PR is the first consumer that reads it early).

2. Toggle-off cache — ENABLE_MULTILIB=ON FORCE-writes EXTRA_LIB / LINKED_10BIT / LINKED_12BIT, but reconfiguring the same build dir with OFF leaves those cache entries sticky. Clear/unset them on the OFF path (or document wipe-cache / fresh build dir).

3. Single-config archive dirs — always passing all four CMAKE_ARCHIVE_OUTPUT_DIRECTORY_{DEBUG,RELEASE,MINSIZEREL,RELWITHDEBINFO} warns unused on Makefiles/Ninja. Gate the per-config variants on a multi-config generator check.

4. -j fallback — still only honors CMAKE_BUILD_PARALLEL_LEVEL. With 10/12 now sequential, include(ProcessorCount); ProcessorCount(_n) as a fallback would give nested builds a real -jN under plain make -jN without oversubscribing.

Android / MULTILIB_CMAKE_ARGS + VERSION/NASM forwarding are still open from Akilan's earlier notes — a short confirm that the vcpkg port passes MULTILIB_CMAKE_ARGS (e.g. -DANDROID_ABI=...) would close that loop.