Skip to content

Commit 852c21c

Browse files
committed
fix(libwebp): 用生成的 config.h 关掉 SSE4.1 —— clang 不像 cl.exe 那样白给 intrinsic
Windows 腿(llvm)红了,而且是个有价值的红: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be inlined into function 'VP8L32bToPlanar_SSE41' that is compiled without support for 'ssse3' libwebp 的 SSE4.1 门是 #if (defined(__SSE4_1__) || defined(WEBP_MSC_SSE41)) && \ (!defined(HAVE_CONFIG_H) || defined(WEBP_HAVE_SSE41)) 而 `WEBP_MSC_SSE41` **只**看 `_MSC_VER`。每个 MSVC ABI 编译器都定义它 —— clang 也 定义 —— 但只有 cl.exe 允许不带 target flag 使用任意 intrinsic。所以「变体自门控」 那句话对 MSVC 成立,对 targeting-MSVC 的 clang 不成立。原描述符(连同它的注释)是 在 cl.exe 上验证的,这一条是它没覆盖到的。 上游 CMake 的解法是 **per-file** `-msse4.1`,而描述符没有 per-file flag 字段。整包 加 `-msse4.1` 不是等价替代:clang 会在**基线** TU 里也发 SSE4.1,绕过 libwebp 自己 的 `VP8GetCPUInfo` 运行期分发 —— 那是在老 CPU 上 SIGILL,而不是回退。 所以改用上游同一套机制的另一半:定义 `HAVE_CONFIG_H` 并生成 `src/webp/config.h`。 这会把 cpu.h 里每个 `(!defined(HAVE_CONFIG_H) || defined(WEBP_HAVE_x))` 门从「默认开、 可否决」翻成「默认关、需允许」,于是这份生成的头就是白名单:只写 SSE2 与 NEON (两者分别是 x86-64 与 aarch64 的**基线**,不需要任何 flag),不写 SSE4.1。 上游本来就为这种配置准备好了退路:`dec_sse41.c` 等在 `#else` 分支里是 `WEBP_DSP_INIT_STUB(VP8DspInitSSE41)`,而调用点 `if (VP8GetCPUInfo(kSSE4_1)) VP8DspInitSSE41();` 由 `WEBP_HAVE_SSE41` 门控 —— 库照样链接、照样分发,只是低一档。 顺带用 `__GNUC__`/`__clang__` 声明三个 `HAVE_BUILTIN_BSWAP*`:HAVE_CONFIG_H 一开, 它们也从「默认有」变成「需声明」,不补就退回可移植实现。
1 parent 0aac70b commit 852c21c

3 files changed

Lines changed: 69 additions & 12 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ Two kinds of packages live here:
4242
| C++-source compat, one depending on the other | [`compat.abseil`](pkgs/c/compat.abseil.lua) (151 TUs; a wildcard over `absl/**` trimmed by upstream's test/benchmark naming conventions) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua) (the libprotobuf runtime, 79 TUs transcribed from upstream's own `src/file_lists.cmake`; declares `compat.abseil` as a dependency because protobuf's public headers include `absl/…`, and its `gzip` feature defines `HAVE_ZLIB` and pulls `compat.zlib`, while `upb` adds protobuf's 64-TU C runtime out of the same tarball. It also exposes **`protoc`** as a `kind = "bin"` target, so a consumer writing `tools = ["protoc"]` gets the compiler built for its own machine out of the same package it links — making a generator/runtime version mismatch inexpressible) · [`compat.re2`](pkgs/c/compat.re2.lua) (22 TUs, upstream's own `RE2_SOURCES`) · [`compat.redis-plus-plus`](pkgs/c/compat.redis-plus-plus.lua) (redis++ 1.3.13 — the sync client, 17 TUs + `patterns/redlock.cpp`, depends on `compat.hiredis`; the one header CMake would generate, `hiredis_features.h`, is snapshotted via `generated_files`, and the async/TLS TUs are left out so the base build stays a two-package pair. An `async` feature adds the libuv-backed `AsyncRedis` interface (the 9 async TUs + `compat.libuv`; `event_loop.cpp` runs `uv_run` on a background thread, and `<hiredis/adapters/libuv.h>` arrives through compat.hiredis' wrapper headers). Two versions, one on each side of the source-structure watershed, share this ONE source list: 1.3.13 (modern 17-TU layout) and 1.3.3 (pre-`redis_uri.cpp`/`redlock` 15-TU layout) — the union works because 1.3.3's TUs are a strict subset, so exactly two globs match nothing there (a warning, not an error; same trick as compat.catch2)) |
4343
| C++-source compat, zero-dep client + optional components | [`compat.websocket`](pkgs/c/compat.websocket.lua) (IXWebSocket 12.0.1 — a pure RFC 6455 client compiled from upstream's `IXWEBSOCKET_SOURCES` minus the four server TUs, so the **base build has zero external dependencies**: TLS off (the OpenSSL/MbedTLS/AppleSSL TUs aren't built) and `IXWEBSOCKET_USE_ZLIB` unset, so the gzip codec compiles to a no-op. Two optional features add on top: `server` (the four server TUs — `IXWebSocketServer`, `IXSocketServer`, `IXHttpServer`, `IXWebSocketProxyServer` — needing nothing external, and it **implies `zlib`** because upstream's server advertises permessage-deflate by default, which the transport negotiates regardless of the define) and `zlib` (deps `compat.zlib` and turns the codec into real per-message-deflate compression). The default-feature test brings its own minimal RFC 6455 echo server on loopback sockets (handshake, masking, fragmentation and close all exercised offline); a second member, `websocket-features`, runs a real `ix::WebSocketServer` and asserts the compression is observable on the wire — a 64 KiB repeated payload round-trips with `wireSize` = 80) |
4444
| Database client + the driver manager it needs, built from source | [`compat.nanodbc`](pkgs/c/compat.nanodbc.lua) (nanodbc 2.14.0, frozen upstream — one TU over the platform ODBC driver manager. Two fixes make the four-year-old source compile and RUN here: a force-included `char_traits<unsigned char>` shim for libc++ (the standard's own customization point, guarded on `_LIBCPP_VERSION` so libstdc++/MSVC are untouched; and note `-include` reaches C++ TUs only through `cxxflags`, never `cflags`), and a per-platform answer to the manager itself — windows links the SDK's odbc32, macOS the OS's iODBC, while linux takes `compat.unixodbc` because mcpp's runtime closure rejects a NEEDED `libodbc.so.2` that only the host has. The test asserts the manager's own diagnostics surface through the wrapper — including nanodbc's frozen off-by-one that drops the last SQL-state character) · [`compat.unixodbc`](pkgs/c/compat.unixodbc.lua) (unixODBC 2.3.14, Shape E over A — DM + odbcinst + ini/log/lst + libltdl compiled statically into one `odbc` target, exactly upstream's libodbc.a symbol set, so the consumer carries no `libodbc.so.2` NEEDED at all. The one non-obvious piece is libtool-free ltdl wiring: `-DLTDLOPEN=libltdlc` plus a generated `lt_libltdlc_LTX_preloaded_symbols` table (reconstructed from the libtool object's relocations) registers the dlopen loader. The frozen `config.h` merges ltdl's own configure output into the top-level one — ltdl sources never read the clashing identification macros, and the merge sidesteps a quoted `-DLT_CONFIG_H` that does not survive the pipeline. Verified against the libtool build of the same tarball: identical IM002 error path and identical `lt_dlopen` behaviour) |
45-
| C-source compat, SIMD variants that gate THEMSELVES | [`compat.libwebp`](pkgs/c/compat.libwebp.lua) (117 TUs as five directory globs rather than a transcribed file list. The glob is safe for a reason worth stating: every `src/dsp/*_sse2.c` / `*_sse41.c` / `*_neon.c` / `*_mips*.c` / `*_msa.c` variant keys off the COMPILER's own `__SSE2__` / `__SSE4_1__` / `__aarch64__` built-ins via `src/dsp/cpu.h`, so a variant that does not match the target compiles to an empty translation unit and the runtime probe falls back to scalar — no per-file flags, and nothing to keep in sync with a release. `-msse4.1` is deliberately NOT added: it would make the artifact require an SSE4.1 CPU. `src/demux` and `src/mux` are separate upstream libraries with their own public headers and stay out until something asks for them) |
45+
| C-source compat, an ISA tier turned off through the GENERATED config | [`compat.libwebp`](pkgs/c/compat.libwebp.lua) (117 TUs as five directory globs rather than a transcribed file list, and one real decision. libwebp's SSE4.1 gate is `(__SSE4_1__ || WEBP_MSC_SSE41) && (!HAVE_CONFIG_H || WEBP_HAVE_SSE41)`, and `WEBP_MSC_SSE41` keys off `_MSC_VER` **alone** — which every MSVC-ABI compiler defines, including clang, while only cl.exe lets an intrinsic be used without a target flag. Under clang the SSE4.1 sources fail on `always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3'`. Upstream answers with a PER-FILE `-msse4.1`, which no descriptor field expresses; adding it package-wide would let clang emit SSE4.1 in the BASELINE TUs too, past libwebp's own runtime dispatch — a SIGILL rather than a fallback. So the package uses the other half of upstream's own mechanism: `HAVE_CONFIG_H` plus a generated `src/webp/config.h` naming SSE2 and NEON and not SSE4.1, which turns `dec_sse41.c` and friends into upstream's `WEBP_DSP_INIT_STUB` and removes the matching `VP8DspInitSSE41()` call. `src/demux` and `src/mux` are separate upstream libraries with their own public headers and stay out until something asks for them) |
4646
| header-only (with `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) |
4747
| header-only, nothing to gate | [`compat.CLI11`](pkgs/c/compat.CLI11.lua) (a command line parser whose every definition is `CLI11_INLINE`, so the package is `*/include` plus an anchor TU. Upstream's two extras stay out: `src/Precompile.cpp` only means anything when `CLI11_COMPILE` also reaches the CONSUMER's translation units — an interface define, not a sources-only gate — and `src/modules/CLI11.cppm` is a module layer, which is a package shape of its own rather than a feature of the compat package) · [`compat.gtl`](pkgs/c/compat.gtl.lua) (Greg's Template Library — the Swiss-table `flat_hash_map` family plus btrees and a bit_vector. `*/include` exactly, not the tarball root: `tests/` and `examples/` carry headers of their own, and naming `include/` is what upstream's INTERFACE target exposes, so a consumer cannot accidentally resolve into test code) · [`compat.plf-hive`](pkgs/c/compat.plf-hive.lua) (the reference implementation of the proposed `std::hive`; the whole library is one file at the tarball root, so `*` plus an anchor TU is the entire package. Untagged upstream, so the version is a DATE over a commit archive — the compat.khrplatform precedent) |
4848
| single-header library + a GENERATED implementation TU | [`compat.nanosvg`](pkgs/c/compat.nanosvg.lua) (two stb-style headers where the implementation hides behind `NANOSVG_IMPLEMENTATION` / `NANOSVGRAST_IMPLEMENTATION`. Upstream ships no `.c` — its examples define the macros inline — so the package generates one that instantiates BOTH halves once. That is what turns a header drop into something linkable, and it moves the duplicate-symbol hazard from every consumer to a single place: consumers must NOT define those macros again, and the test links `nsvgParse` and `nsvgRasterize` together precisely so a package that instantiated only one half fails here rather than downstream) · [`compat.vulkan-memory-allocator`](pkgs/c/compat.vulkan-memory-allocator.lua) (VMA 3.4.0, same shape but the generated TU also has to make a POLICY choice. VMA defaults to `VMA_STATIC_VULKAN_FUNCTIONS 1`, which references `vkBindBufferMemory2` and seven siblings by name — eight undefined symbols against a headers-only dep. Pulling `compat.vulkan` to satisfy them would force a Vulkan loader on every consumer of a memory allocator and fight anyone dispatching through volk, so the generated TU selects the dynamic path instead and VMA resolves everything through `VmaVulkanFunctions`. Note the implementation is C++ despite the C-shaped API, so the generated file is `.cpp`) |

‎README.zh-CN.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ mcpp self config --mirror CN # 切换至国内镜像,默认使用 GLOBAL 上
3939
| C++ 源码 compat(彼此依赖) | [`compat.abseil`](pkgs/c/compat.abseil.lua)(151 TU;对 `absl/**` 取通配后,按上游自身的 test/benchmark 命名约定裁剪) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua)(libprotobuf 运行时,79 TU 逐条转录自上游 `src/file_lists.cmake`;因 protobuf 公开头文件 include 了 `absl/…`,故显式依赖 `compat.abseil`;`gzip` feature 定义 `HAVE_ZLIB` 并拉入 `compat.zlib`,`upb` feature 则从同一个 tarball 里再编出 protobuf 的 64 TU C 运行时;还以 `kind = "bin"` target 暴露 **`protoc`**,消费者写 `tools = ["protoc"]` 即可从「自己链接的那个包」拿到为本机构建的编译器,使生成器与运行时的版本错配无法表达) · [`compat.re2`](pkgs/c/compat.re2.lua)(22 TU,取自上游自身的 `RE2_SOURCES`) · [`compat.redis-plus-plus`](pkgs/c/compat.redis-plus-plus.lua)(redis++ 1.3.13 —— 同步客户端,17 TU + `patterns/redlock.cpp`,依赖 `compat.hiredis`;CMake 唯一会生成的头 `hiredis_features.h` 用 `generated_files` 快照,async/TLS TU 不收,基座保持两包成对。`async` feature 补齐 libuv 版 `AsyncRedis` 接口(9 个 async TU + `compat.libuv`;`event_loop.cpp` 在后台线程跑 `uv_run`,`<hiredis/adapters/libuv.h>` 经 compat.hiredis 的包装头到达)。两个版本分处源码结构分水岭两侧,共享同一份源列表:1.3.13(现代 17-TU 布局)与 1.3.3(缺 `redis_uri.cpp`/`redlock` 的 15-TU 旧布局)—— 并集之所以成立,是因为 1.3.3 的 TU 是 1.3.13 的严格子集,恰好两个 glob 在 1.3.3 上零命中(仅警告,非错误;与 compat.catch2 同款手法)) |
4040
| C++ 源码 compat(零依赖客户端 + 可选组件) | [`compat.websocket`](pkgs/c/compat.websocket.lua)(IXWebSocket 12.0.1 —— 从上游 `IXWEBSOCKET_SOURCES` 剔掉 4 个 server TU 后直编的纯 RFC 6455 客户端,**基座零外部依赖**:TLS 关闭(OpenSSL/MbedTLS/AppleSSL 三组 TU 均不编),`IXWEBSOCKET_USE_ZLIB` 不定义(gzip codec 编译为 no-op)。两个可选 feature 在基座上叠加:`server`(4 个 server TU —— `IXWebSocketServer`/`IXSocketServer`/`IXHttpServer`/`IXWebSocketProxyServer`,零新增外部依赖,且 **implies `zlib`** —— 因为上游 server 默认就宣称 permessage-deflate,而 transport 的协商不受宏门控)与 `zlib`(依赖 `compat.zlib`,把 codec 变成真正的 permessage-deflate 压缩)。默认构建的测试自带基于 loopback 原始 socket 的最小 RFC 6455 echo server(握手/掩码/分片/关闭全部离线实测);第二个成员 `websocket-features` 跑真实的 `ix::WebSocketServer`,并断言压缩在线路上可观测 —— 64 KiB 重复载荷往返,`wireSize` = 80) |
4141
| 数据库客户端 + 源码构建的驱动管理器 | [`compat.nanodbc`](pkgs/c/compat.nanodbc.lua)(nanodbc 2.14.0,上游已冻结 —— 单 TU 封装平台 ODBC 驱动管理器。两处修复让这份四年前的源码在此可编译、可运行:一个 force-include 的 `char_traits<unsigned char>` 补丁头(标准留给用户的定制点,以 `_LIBCPP_VERSION` 为界,不影响 libstdc++/MSVC;注意 `-include` 只能经 `cxxflags` 到达 C++ TU,`cflags` 够不着),以及对驱动管理器本身的分平台答案 —— windows 链 SDK 的 odbc32、macOS 链系统自带的 iODBC,linux 则依赖 `compat.unixodbc`,因为 mcpp 的运行时闭包检查不接受只有宿主才有的 `libodbc.so.2`。测试断言管理器自身的诊断能穿过封装层 —— 包括 nanodbc 已冻结的、会把 SQL state 末字符截掉的 off-by-one)· [`compat.unixodbc`](pkgs/c/compat.unixodbc.lua)(unixODBC 2.3.14,E 叠 A 形态 —— DM + odbcinst + ini/log/lst + libltdl 静态编入单一 `odbc` 目标,与上游 libodbc.a 符号集一致,消费者不带任何 `libodbc.so.2` NEEDED。唯一非常规之处是无 libtool 的 ltdl 接线:`-DLTDLOPEN=libltdlc` 加一张生成的 `lt_libltdlc_LTX_preloaded_symbols` 表(从 libtool 目标文件的重定位记录还原)注册 dlopen loader。冻结的 `config.h` 把 ltdl 自己的 configure 输出合并进顶层(冲突宏 ltdl 源码并不读),绕开了无法在管道中幸存的带引号 `-DLT_CONFIG_H`。已与同 tarball 的 libtool 构建对比验证:IM002 错误路径与 `lt_dlopen` 行为完全一致) |
42-
| C 源码 compat(SIMD 变体**自门控**) | [`compat.libwebp`](pkgs/c/compat.libwebp.lua)(117 个 TU 用五条目录通配写完,而不是逐条转录文件名。通配之所以安全值得写下来:`src/dsp/` 下每个 `*_sse2.c` / `*_sse41.c` / `*_neon.c` / `*_mips*.c` / `*_msa.c` 变体都经 `src/dsp/cpu.h` 依**编译器自己的** `__SSE2__` / `__SSE4_1__` / `__aarch64__` 内建宏自我门控 —— 不匹配当前 target 的变体编译成空 TU,运行期探测走标量回退,既不需要 per-file flag,也不会随版本漂移。`-msse4.1` 有意不加:加了产物就要求 SSE4.1 CPU。`src/demux` / `src/mux` 是上游各自独立、各带公开头文件的库,在有人需要之前不收) |
42+
| C 源码 compat(用**生成的 config** 关掉一档 ISA) | [`compat.libwebp`](pkgs/c/compat.libwebp.lua)(117 个 TU 用五条目录通配写完,外加一个真实的取舍。libwebp 的 SSE4.1 门是 `(__SSE4_1__ || WEBP_MSC_SSE41) && (!HAVE_CONFIG_H || WEBP_HAVE_SSE41)`,而 `WEBP_MSC_SSE41` **只**看 `_MSC_VER` —— 每个 MSVC ABI 编译器都定义它,clang 也不例外,但只有 cl.exe 允许不带 target flag 使用任意 intrinsic。于是 clang 下 SSE4.1 那批源码报 `always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3'`。上游用 **per-file** `-msse4.1` 解决,而描述符没有这个字段;整包加上去,clang 就会在**基线** TU 里也发 SSE4.1,绕过 libwebp 自己的运行期分发 —— 那是 SIGILL 而不是回退。所以本包用上游同一套机制的另一半:`HAVE_CONFIG_H` + 生成的 `src/webp/config.h`,只声明 SSE2 与 NEON、不声明 SSE4.1,`dec_sse41.c` 等随之变成上游的 `WEBP_DSP_INIT_STUB`,对应的 `VP8DspInitSSE41()` 调用点也一并消失。`src/demux` / `src/mux` 是上游各自独立、各带公开头文件的库,在有人需要之前不收) |
4343
| header-only(含 `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) |
4444
| header-only(无可门控组件) | [`compat.CLI11`](pkgs/c/compat.CLI11.lua)(命令行解析器,全部定义都是 `CLI11_INLINE`,故整包就是 `*/include` 加一个 anchor TU。上游两个额外件都不收:`src/Precompile.cpp` 只有在 `CLI11_COMPILE` 同时到达**消费者** TU 时才有意义 —— 那是 interface define,不是 sources 门控;`src/modules/CLI11.cppm` 属于模块层,是另一种包形态,而非 compat 包的 feature) · [`compat.gtl`](pkgs/c/compat.gtl.lua)(Greg's Template Library —— Swiss-table 的 `flat_hash_map` 家族,外加 btree 与 bit_vector。只取 `*/include` 而非 tarball 根:`tests/` 与 `examples/` 各自带头文件,而 `include/` 正是上游 INTERFACE target 暴露的范围,消费者不会误解析到测试代码) · [`compat.plf-hive`](pkgs/c/compat.plf-hive.lua)(提案中 `std::hive` 的参考实现;整库就是 tarball 根下一个文件,故全包即 `*` 加一个 anchor TU。上游不打 tag,于是版本用 commit 归档上的日期 —— 沿用 compat.khrplatform 的先例) |
4545
| 单头库 + **生成**实现 TU | [`compat.nanosvg`](pkgs/c/compat.nanosvg.lua)(两个 stb 风格头文件,实现藏在 `NANOSVG_IMPLEMENTATION` / `NANOSVGRAST_IMPLEMENTATION` 之后。上游不提供 `.c` —— 其示例是就地 define 宏 —— 故本包生成一个,把两半各实例化一次。这才让「一堆头文件」变成可链接的包,并把重复符号的风险从每个消费者收敛到唯一一处:消费者不得再次定义这两个宏。测试刻意同时链接 `nsvgParse` 与 `nsvgRasterize`,使「只实例化了一半」的包在此处就失败,而不是流到下游)· [`compat.vulkan-memory-allocator`](pkgs/c/compat.vulkan-memory-allocator.lua)(VMA 3.4.0,同一形态,但生成的 TU 还得做一个**策略**选择。VMA 默认 `VMA_STATIC_VULKAN_FUNCTIONS 1`,会按名字引用 `vkBindBufferMemory2` 等八个符号 —— 对「只依赖头文件」而言就是八个未定义引用。为此拉入 `compat.vulkan` 是错的:那会逼所有用内存分配器的消费者都链上 Vulkan loader,并与通过 volk 自行分发的项目冲突。故生成的 TU 改走动态路径,VMA 一律经 `VmaVulkanFunctions` 解析。注意其 API 形如 C 但实现是 C++,故生成文件为 `.cpp`)|

0 commit comments

Comments
 (0)