Skip to content

Commit 4dad97b

Browse files
0.23.0 --- an image placed at its base (kal_exec_alloc_at, KAL_EXEC_PROP_PLACE, the placement note, clause 12.4 version 1.1); a name with every link followed (kal_fs_resolved_name, KAL_FS_PROP_RESOLVED_NAME)
Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent ae400db commit 4dad97b

15 files changed

Lines changed: 287 additions & 12 deletions

File tree

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# openkal 0.23:映像在其基址上放置
2+
3+
2026-11-03。状态:**实施所依据的方案**。规范对照基于 `SPEC.md` 0.22,结果为 0.23.0。
4+
来源:mcpp 生态设计 2026-11-02 part 3 §16.8、§17.6;mcppls-clangd #16 的 warm-prime(Linux x64、macOS arm64)。
5+
6+
---
7+
8+
## 1. 要解决的问题
9+
10+
warm-prime 的冷阶段启动 176 个 worker。CI 的 Linux x64 上映像的处理器时间为原生的 1.27 倍、缺页为 1.42 倍,macOS arm64
11+
的缺页为 2.32 倍;本机(Linux x64)为 1.09 倍。差距按 worker 平均分摊,与模块大小无关:
12+
13+
| # | 测量 | 结果 |
14+
| --- | --- | --- |
15+
| P1 | 一次启动(`--version`)的缺页 | 原生 297(非 PIE,固定地址),映像 1285 |
16+
| P2 | 映像的重定位 | `.rela.dyn` 3.9 MB,约 164 000 条相对重定位,落在 2 MB 的 `.data.rel.ro` 上 |
17+
| P3 | 打包的相对重定位(`-z pack-relative-relocs`) | 表缩为 32 KiB,缺页 1285 → 1160;可写段仍被整体复制并逐项改写 |
18+
| P4 | 每次多出的缺页的代价 | 本机每个 worker 约 3.5 ms,CI 的 x64 虚拟机约 24 ms |
19+
20+
映像是位置无关的(12.2),加载器每次启动都要写入相对重定位算出的每个地址;为一个地址链接的原生程序不写这些地址,其只读
21+
数据是文件的页,按需缺页并预读相邻页。
22+
23+
## 2. 增补
24+
25+
| 接口 | 增补 | 层级 | 条文 |
26+
| --- | --- | --- | --- |
27+
| `openkal.exec` | `kal_exec_alloc_at(at, size)`、属性 `KAL_EXEC_PROP_PLACE` | 属性位 | 11 条 48、12.3 |
28+
| 映像格式 | 第 3 类 note(placement):基址与各个在文件中有字节的可写段(按地址升序)按该基址算好的字节在文件中的位置;本条文版本 1.1 | 可选 note | 12.4 |
29+
| 加载器 | 得到基址时,只读部分(`PT_GNU_RELRO`)中没有符号重定位的粒度直接映射文件的字节,其余复制,只做引用符号的重定位 | 陈述 | 12.3 |
30+
| `openkal.fs` | `kal_fs_resolved_name(base, name, len, out, cap)`、属性 `KAL_FS_PROP_RESOLVED_NAME` | 卷的属性位 | 11 条 49 |
31+
32+
`kal_exec_alloc_at` 在 `at` 处预留区域,该处已有内容或环境不允许程序选择地址时返回空指针、不预留任何东西;`at` 是
33+
`kal_memory_granularity` 的倍数。
34+
35+
同一次测量(warm-prime 的已有缓存阶段,`strace -c`)中,映像发出 87 万次 `newfstatat`,原生为 35 万次 `stat` 与 10 万次
36+
`readlink`:映像的 `realpath` 逐个组成部分询问是否为链接(`usr`、`usr/lib`、`usr/lib/gcc`……),原生程序对打开的每个文件只问
37+
环境一次(Linux 的 `/proc/self/fd`,macOS 的 `F_GETPATH`)。CI 的 x64 上映像的内核态时间因此多 20%。`kal_fs_resolved_name`
38+
回答名字跟随全部链接后的绝对名字:Linux 以 `O_PATH` 打开后读 `/proc/self/fd/N`,macOS 以 `O_EVTONLY` 打开后取 `F_GETPATH`,
39+
Windows 以 `GetFinalPathNameByHandleW`;openkal-musl 的 `realpath` 先用它,卷不声明时照旧逐个询问。
40+
41+
## 3. 方案中的决定
42+
43+
1. **一个操作,而不是 `kal_exec_alloc` 的提示。** 加载器在写入任何东西之前就要知道是否拿到了基址:拿到时取存储的字节,拿不到
44+
时按文件重定位。提示得到的地址需要再比较,比较之后的回退正是这个操作的语义,所以由操作陈述。
45+
2. **属性位,而不是必需。** 有的环境替程序决定每块内存的位置。Linux 用 `MAP_FIXED_NOREPLACE`(4.17 之前的内核把它当作
46+
提示,结果不同则释放并拒绝),macOS 用提示并比较(`MAP_FIXED` 会替换已有内容),Windows 用 `VirtualAlloc` 在该地址预留;
47+
三个实现都声明该位。Windows 不声明 `KAL_EXEC_PROP_FILE`,得到基址时复制存储的字节,省去重定位但不省去复制。
48+
3. **基址由映像决定,在封装时选定。** 同一映像的所有进程放在同一地址,重定位后的只读数据在各进程中相同,可以是同一文件的页。
49+
基址取自映像内容的摘要,落在用户地址空间的中段(16 TiB 起,4 GiB 一档),与系统分配器自上而下的放置相距很远;一个进程中
50+
多个映像各有其基址,冲突时后者照常重定位。
51+
4. **只存可写段,只跳过相对重定位。** 指令与只读数据本来就是文件的字节(0.21);引用符号的重定位指向加载器绑定的实现入口,
52+
因加载器而异,仍在加载时完成。只读部分(`PT_GNU_RELRO`)中没有这类重定位的粒度不再被写,可以映射文件;其余粒度复制后
53+
改写。
54+
5. **格式版本 1.1。** 新 note 是可选的;1.0 的加载器不读它,照常重定位,行为不变。一个要求新加载器才能运行的映像不由此产生。
55+
6. **先放置,后签名。** 签名覆盖整个文件(12.4 第 2 类),存储的段也在其中;`okld place` 拒绝已签名的映像。基址取自 ELF
56+
所描述的字节(placement 描述符读作零),同一映像无论放置几次都得到同一基址与同一字节。
57+
58+
## 4. 版本组
59+
60+
| 包 | 版本 | 内容 |
61+
| --- | --- | --- |
62+
| openkal | 0.23.0 | 本文 |
63+
| openkal-linux | 0.24.0 | `kal_exec_alloc_at` |
64+
| openkal-macos | 0.21.0 | `kal_exec_alloc_at` |
65+
| openkal-windows | 0.19.0 | `kal_exec_alloc_at` |
66+
| openkal-emscripten | 0.12.0 | 依 0.23 编写,无其他改动 |
67+
| openkal-image | 0.4.0 | `okld place`(封装步骤)、placement note、加载器按 12.3 放置 |
68+
| openkal-musl | 0.27.0 | 加载器为 openkal-ld 0.4.0(另含 0.22 的 `/dev/null`) |
69+
| openkal-llvm-runtime | 0.18.0 | 建在 openkal-musl 0.27.0 上 |
70+
| openkal-platform | 0.23.0 | 版本组;封装步骤增加 `okld place`,链接打包相对重定位 |
71+
| mcpplibs.os | 0.2.8 | openkal 0.23 的实现 |
72+
73+
## 5. 验证
74+
75+
- 一致性套件:未对齐的地址被拒绝;已有区域的地址被拒绝且原区域不变;刚释放的地址被重新预留并可写;未声明该位的实现拒绝
76+
所有请求。openkal-linux 0.24 的本地一致性:328 项全部成立。
77+
- openkal-image:同一映像在基址上、在任意地址上(`OKRUN_PLACE=0`)与未放置时观察到的内存相同;在基址上不做相对重定位;
78+
封装后的映像通过 `okld check`,存储段的一个字节被改动时被拒绝;重复放置不改变字节;openkal-image 0.3.1 的 okrun 照常运行
79+
0.4.0 封装的映像。本机(Linux x64)clangd 映像一次启动(`--version`)的缺页:在基址上 606,在任意地址上 1 047;
80+
`PT_GNU_RELRO` 的 2 MB 中除首尾两个 64 KiB 外都是文件的页。
81+
- warm-prime:CI 的 Linux x64、arm64、macOS arm64 上映像对原生的冷阶段处理器时间与缺页。

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,16 +77,16 @@ conditional on the target.
7777

7878
```toml
7979
[dependencies]
80-
openkal = "0.22.0"
80+
openkal = "0.23.0"
8181

8282
[target.'cfg(os = "linux")'.dependencies]
83-
openkal-linux = "0.23.0"
83+
openkal-linux = "0.24.0"
8484

8585
[target.'cfg(os = "macos")'.dependencies]
86-
openkal-macos = "0.20.0"
86+
openkal-macos = "0.21.0"
8787

8888
[target.'cfg(windows)'.dependencies]
89-
openkal-windows = "0.18.0"
89+
openkal-windows = "0.19.0"
9090
```
9191

9292
The program imports the interface and names no implementation.

‎SPEC.md‎

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# openkal Specification, version 0.22
1+
# openkal Specification, version 0.23
22

33
## 1. Scope
44

@@ -92,7 +92,7 @@ provides an interface in whole or not at all.
9292

9393
Version 0.15 specifies the core and optional interfaces, version 0.18 adds
9494
`openkal.map` (clause 11, entry 40), and version 0.20 adds `openkal.region`,
95-
`openkal.prof` and `openkal.debug` (entries 44 and 45); versions 0.21 and 0.22 add no interface. The reserved row is not specified, and its
95+
`openkal.prof` and `openkal.debug` (entries 44 and 45); versions 0.21 to 0.23 add no interface. The reserved row is not specified, and its
9696
name shall not be used for other purposes.
9797

9898
The S, L and X columns state which boundaries an interface's declarations can
@@ -1920,6 +1920,35 @@ The following are recorded so that they are not mistaken for oversights.
19201920
writes the device itself is served by its C library without the
19211921
environment. It is a position and not required, because an environment
19221922
that starts no programs has nothing to give it to.
1923+
48. **An image placed at its base.** **Specified in 0.23**, as
1924+
`kal_exec_alloc_at` and `KAL_EXEC_PROP_PLACE` in `openkal.exec`
1925+
(`exec.h`), with the placement note (clause 12.4, version 1.1) and its use
1926+
by a loader (clause 12.3). An image is position-independent (12.2), so a
1927+
loader wrote, on every start, each address its relative relocations
1928+
compute: measured for a language server's image, 164 000 relocations over
1929+
two mebibytes of read-only-after-relocation data, made resident in each of
1930+
the 176 programs a build of its modules starts --- about 900 page faults a
1931+
program more than the same server built for its environment at one
1932+
address, which on a virtual machine of a continuous-integration service
1933+
cost a fifth of the build's processor time. An image computed once for one
1934+
base and placed there is the file's pages, as such a program is. The
1935+
reservation at an address is an operation and not a hint to
1936+
`kal_exec_alloc`, because a loader that is refused must know it before it
1937+
writes anything; it is a position, because an environment may decide where
1938+
every program's memory lies; and the address is the image's, chosen when it
1939+
is sealed, not the environment's.
1940+
49. **A name with every link followed.** **Specified in 0.23**, as
1941+
`kal_fs_resolved_name` and `KAL_FS_PROP_RESOLVED_NAME` in `openkal.fs`
1942+
(`fs.h`). A C library's `realpath` asked about each component of a name in
1943+
turn whether it is a link: measured in a language server's session run as
1944+
an image, half a million enquiries, where the program of its environment
1945+
asked the environment once for each file it opened (`/proc/self/fd` on one
1946+
system, `F_GETPATH` on another). Every environment that keeps links answers
1947+
the question for a name it has resolved, and nothing above it can compose
1948+
the enquiries into one. The name is absolute and in the environment's
1949+
spelling, as `kal_env_image` states one: a C library maps it onto its own
1950+
names as it maps that one. It is a position of the volume, as the other
1951+
operations of names are.
19231952

19241953
## 12. Images
19251954

@@ -2000,6 +2029,19 @@ zero, where the image's bytes are not a file's at a multiple of
20002029
`KAL_MAP_ALIGNMENT` (an image a program carries in its own bytes), and where a
20012030
relocation writes the segment.
20022031

2032+
Where the implementation sets `KAL_EXEC_PROP_PLACE` (version 0.23) and the image
2033+
carries a placement note (12.4), a loader asks for the image's span at the
2034+
note's base (`kal_exec_alloc_at`). Given it, the loader takes each stored
2035+
segment's bytes instead of the file's and applies only the relocations that
2036+
name a symbol: a part of a stored segment that is read-only after relocation
2037+
(the image's `PT_GNU_RELRO`) and that no such relocation writes is the file's
2038+
bytes (`kal_exec_map_part`) where `KAL_EXEC_PROP_FILE` is set, and the rest is
2039+
copied. Refused, or where the position is zero, the loader reserves the span
2040+
anywhere and relocates the file's segments as before. The program observes the
2041+
same memory either way; what differs is that a program placed at its base writes
2042+
no address its relative relocations compute, as a program of its environment
2043+
linked for one address does not.
2044+
20032045
Where an environment's own convention differs from the image's --- on x86-64,
20042046
one environment passes arguments in other registers --- the loader presents each
20052047
entry in the image's convention, and a function the image passes to the
@@ -2030,8 +2072,11 @@ An image carries notes whose owner is `OPENKAL`, in a section `.note.openkal`:
20302072
| --- | --- |
20312073
| 1, requirements | `kal_u32` major and `kal_u32` minor version of this clause (1 and 0), then a `kal_u64` of `KAL_IFACE_*` positions: **the interfaces the image requires**, derived from the names it imports other than weakly |
20322074
| 2, signature | 32 bytes of an Ed25519 public key, then 64 bytes of the signature, over the whole file with these 96 bytes read as zero |
2075+
| 3, placement | a `kal_u64` base, a multiple of 65536; a `kal_u32` count and a `kal_u32` zero; then, for each of `count` writable loadable segments that have bytes in the file, in ascending order of address, three `kal_u64`: the segment's address in the image, the file offset of its stored bytes, congruent to that address modulo 65536, and their length, the segment's size in the file. The stored bytes are the segment's file bytes with each relative relocation, and each of the packed relative form, applied for `base`; a relocation that names a symbol is not applied. Version 1.1 of this clause |
20332076

2034-
The first is required and the second is not. A loader compares the required interfaces with `kal_interfaces` and
2077+
The first is required and the others are not. A count of zero states no
2078+
placement, and a loader reads a placement note only after the requirements note
2079+
states version 1.1 or higher. A loader compares the required interfaces with `kal_interfaces` and
20352080
refuses to load an image that requires one the implementation does not provide,
20362081
naming the interface. That is clause 6.1 in this arrangement: the loader binds
20372082
at load what a linker binds at link, so it reports at load what a linker reports

‎SURFACE.txt‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# The C surface of openkal 0.22.0, one name per line.
1+
# The C surface of openkal 0.23.0, one name per line.
22
#
33
# This file is normative and is the single source consulted by clause 9. A
44
# conforming implementation exports the names of the interfaces it provides
@@ -74,6 +74,7 @@ kal_fs_read_at
7474
kal_fs_remove
7575
kal_fs_rename
7676
kal_fs_rename_flags
77+
kal_fs_resolved_name
7778
kal_fs_seek
7879
kal_fs_set_executable_at
7980
kal_fs_set_modified
@@ -115,6 +116,7 @@ kal_task_wake
115116
kal_task_yield
116117
# openkal.exec
117118
kal_exec_alloc
119+
kal_exec_alloc_at
118120
kal_exec_free
119121
kal_exec_granularity
120122
kal_exec_map_part

‎conformance/src/declarations.c‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ void okc_declarations_c(void)
4545
(void)sizeof(&kal_env_var_at);
4646
(void)sizeof(&kal_env_var_count);
4747
(void)sizeof(&kal_exec_alloc);
48+
(void)sizeof(&kal_exec_alloc_at);
4849
(void)sizeof(&kal_exec_free);
4950
(void)sizeof(&kal_exec_granularity);
5051
(void)sizeof(&kal_exec_map_part);
@@ -60,6 +61,7 @@ void okc_declarations_c(void)
6061
(void)sizeof(&kal_fs_info);
6162
(void)sizeof(&kal_fs_link_create);
6263
(void)sizeof(&kal_fs_link_read);
64+
(void)sizeof(&kal_fs_resolved_name);
6365
(void)sizeof(&kal_fs_list_begin);
6466
(void)sizeof(&kal_fs_list_next);
6567
(void)sizeof(&kal_fs_lock);

‎conformance/src/macros_reach.cpp‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,5 +45,7 @@ static_assert(kal::macros::KAL_SPAWN_NULL_IN_M == 4u);
4545
static_assert(kal::macros::KAL_SPAWN_NULL_OUT_M == 8u);
4646
static_assert(kal::macros::KAL_SPAWN_NULL_ERR_M == 16u);
4747
static_assert(kal::macros::KAL_PROCESS_PROP_NULL_STREAMS_M == 8192u);
48+
static_assert(kal::macros::KAL_EXEC_PROP_PLACE_M == 8u);
49+
static_assert(kal::macros::KAL_FS_PROP_RESOLVED_NAME_M == 2048u);
4850

4951
} // namespace

‎conformance/src/sections/exec.cpp‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ module okc.exec;
22

33
import openkal.types;
44
import openkal.exec;
5+
import openkal.memory;
56
import openkal.macros;
67
import openkal.stream;
78
import okc.report;
@@ -70,6 +71,43 @@ bool make_file() {
7071
return ok;
7172
}
7273

74+
// A region at the address asked for (0.23): an address just released is given
75+
// again, and one a region holds is refused with nothing reserved; where the
76+
// position is zero, every request is refused. An address that is not a multiple
77+
// of the reservation quantum is refused either way.
78+
void at_an_address() {
79+
const kal_uintptr q = kal_memory_granularity();
80+
const kal_uintptr size = 4 * q;
81+
void* r = kal_exec_alloc(size);
82+
if (r == nullptr) {
83+
unobserved(kind::behaviour, "a region is reserved at the address asked for",
84+
"no region could be reserved to find a free address");
85+
return;
86+
}
87+
auto* bytes = static_cast<unsigned char*>(r);
88+
observe(kind::behaviour, kal_exec_alloc_at(bytes + 1, size) == nullptr,
89+
"an address that is not a multiple of the reservation quantum is refused");
90+
if (!kal::exec::has(kal::exec::place)) {
91+
kal_exec_free(r, size);
92+
observe(kind::behaviour, kal_exec_alloc_at(r, size) == nullptr,
93+
"a region at an address is refused where the implementation does not claim it");
94+
return;
95+
}
96+
observe(kind::behaviour, kal_exec_alloc_at(r, size) == nullptr,
97+
"an address a region holds is refused");
98+
bytes[0] = 5;
99+
observe(kind::behaviour, bytes[0] == 5, "and the region there is as it was");
100+
kal_exec_free(r, size);
101+
void* again = kal_exec_alloc_at(r, size);
102+
observe(kind::behaviour, again == r, "a region is reserved at an address just released");
103+
if (again == r) {
104+
auto* b = static_cast<unsigned char*>(again);
105+
b[size - 1] = 6;
106+
observe(kind::behaviour, b[size - 1] == 6, "and is written as a region kal_exec_alloc returned");
107+
kal_exec_free(again, size);
108+
}
109+
}
110+
73111
void from_a_file() {
74112
const kal_uintptr g = kal_exec_granularity();
75113
if (g == 0 || kAlign % g != 0) {
@@ -154,7 +192,7 @@ void run() {
154192

155193
claim("kal_exec_props()", kal_exec_props());
156194
observe(kind::abi,
157-
(kal_exec_props() & ~(kal::exec::republish | kal::exec::available | kal::exec::file).bits) == 0,
195+
(kal_exec_props() & ~(kal::exec::republish | kal::exec::available | kal::exec::file | kal::exec::place).bits) == 0,
158196
"no position the specification has not assigned is reported");
159197

160198
// WHETHER THE INTERFACE CAN BE EXERCISED BY THIS ARTIFACT IS READ BEFORE
@@ -286,6 +324,8 @@ void run() {
286324
}
287325

288326
claim("KAL_EXEC_PROP_FILE", kal_exec_props() & kal::exec::file.bits);
327+
claim("KAL_EXEC_PROP_PLACE", kal_exec_props() & kal::exec::place.bits);
328+
at_an_address();
289329
#if defined(MCPP_FEATURE_FS)
290330
from_a_file();
291331
#else

0 commit comments

Comments
 (0)