Moon Library Format
Part of the Sun ABI draft. These are proposed requirements, not guarantees of current releases. Open decisions must be resolved before stabilization.
Role of a Moon bundle
A .moon file packages compiled code with the declarations needed to import it.
One bundle contract covers the container format, interface schema, generic
semantics, machine ABI, and embedded dependencies. Its requirements are defined in the
compatibility model.
The sections below describe the current format and proposed changes. The draft does not yet specify every byte needed to implement a reader or writer.
Current container structure
For a small geometry library, picture the file as:
[header][geometry bitcode][optional native archive][MoonLibrary protobuf]
↑ header points hereA module record points back to its bitcode bytes. Two module records can point to exactly the same image; this does not require storing that image twice. The table below is a byte-layout reference for that picture.
The container contains a small header, compiled payload bytes, and one
MoonLibrary protobuf. The library record owns the module and archive indexes;
there are no separate custom index tables. Several modules may reference the
same bitcode image.
| Byte offset | Size in bytes | Contents |
|---|---|---|
0 | 4 | Moon magic, 0x53554E4C |
4 | 4 | Container revision |
8 | 8 | Offset of the MoonLibrary protobuf |
16 | 8 | Length of the MoonLibrary protobuf |
H | Variable | LLVM bitcode and native archive payloads |
| Library offset | Library length | Complete MoonLibrary protobuf, ending at the end of the file |
H is sizeof(MoonHeader), normally 24 bytes. Header integers still use native
byte order and C++ structure layout. A portable header encoding remains separate
work; protobuf encoding does not make the existing header portable.
Container revision 15 introduces the complete library record. The reader requires
an exact match with MoonHeader::VERSION; earlier bundles must be rebuilt.
A matching container revision does not authorize semantic import: compatibility
validation still precedes use of declarations or compiled code.
Library record
Suppose the header points to the final protobuf correctly, but a module claims its bitcode extends beyond the file. The reader must reject that range before trying to decode it. A correct outer header does not validate every inner record.
include/moon_bundling/moon.proto defines the following structure:
| Message | Contents |
|---|---|
MoonLibrary | Library content identity, bundle compatibility contract, ordered module records, and ordered native archive records |
LibraryModule | Complete ModuleMetadata, bitcode offset, and bitcode length |
LibraryArchive | Archive-set identity, archive name, byte offset, and byte length |
Offsets refer to positions in the container file. Large bitcode and archive bytes remain outside protobuf. Module metadata is nested directly in the library record, so declarations and the payload references describing the library are available through one schema.
The reader bounds the library protobuf before parsing, then checks record counts, identities, duplicate keys, archive names, and payload ranges before authorizing semantic use. Exact shared bitcode ranges are allowed; unrelated overlapping ranges and payload references into the library protobuf are rejected. Module content identities must agree with the library identity. A missing contract is an error; the former module contract field remains reserved.
Producer identity remains inside the contract message as diagnostic metadata, excluded from the compatibility hash. The library content identity is separate from that hash and continues to identify the artifact used for symbol isolation.
Remaining format proposals
Portable header integers, optional complete source inputs, and native Sun object packaging remain separate work. The compatibility model still requires complete semantic and runtime requirements; changing the container does not establish stable ABI support.
Module metadata
For area(Rectangle), metadata tells the importer about the function and its
rectangle declaration; bitcode holds the compiled body. A retained generic body
may additionally name a private helper, so that helper’s identity must remain
available even though application source cannot name it.
include/moon_bundling/moon.proto defines the current protobuf metadata, reusing
AST schemas. It carries module names and visibility, declaration records and
portable references, artifact identities, dependencies, target triple, static
initialization order, declarations, retained generic bodies, globals, and C
binding assertions.
Currently, ordinary compiled function bodies are not exported as generic source bodies. Global initializers are omitted from declaration metadata; their compiled storage and initialization code reside in the payload. Private declarations may still be needed by generic bodies and must preserve their identities and visibility.
Reading a protobuf field does not mean the consumer understands its meaning. The proposed reader checks required language features as well as field encoding; see generic semantics.
Optional complete source
Imagine these rebuild inputs (illustrative filenames):
geometry.sun public entrypoint
rectangle.sun private implementation
init.sun global initialization
shape_data.sun generated input, or a reproducible recipe to generate itEmbedding only geometry.sun is not complete source. A consumer rebuilding for
another target needs the private implementation and initialization too. This
source-packaging feature is proposed, not an existing rebuild guarantee.
A producer may embed all source code so a consumer can choose to rebuild the bundle with its own compiler and target settings. This can make a library usable when its existing compiled contract is unsupported. Source embedding and consumer recompilation are both optional; bundles containing only compiled code and the usual import metadata remain supported.
Complete source means more than exported generic bodies. It includes private implementations, global initializers, and source for embedded dependencies, plus the inputs needed to build them. The consumer still needs a toolchain that understands the recorded source language and can satisfy external requirements. Recompilation does not promise that arbitrary future compilers accept old source or produce identical behavior.
| Minimal example | Level | Requirement when offering complete source recompilation |
|---|---|---|
| A public wrapper calls a private helper that was omitted from the source package. | MUST | Include all source needed to rebuild the claimed scope, including private code, initializers, and embedded dependency code. Identify any binary-only components rather than describe them as rebuildable. |
| The library needs a generated file and an exact external dependency. | MUST | Record source language versions, module paths, exact dependency identities, relevant build settings, generated inputs or their reproducible generation recipe, resources, and required external tools or libraries. |
| The caller explicitly chooses to rebuild an otherwise compatible library. | MUST | Let the consumer choose recompilation, including when the existing compiled contract is supported. |
| The bundle contains source, but the caller chooses its compiled payload. | MUST NOT | Recompile merely because source is present. A consumer may explicitly configure recompilation as a fallback after compiled-contract rejection. |
| A required source-language feature or generated input is unavailable. | MUST | Check source and build requirements independently of the old compiled ABI. Reject missing inputs or unsupported source semantics with a clear error. |
| Sun code rebuilds for AArch64, but its retained archive is x86-64. | MUST | Rebuild native components when their source and toolchain are available, or check the compatibility of any retained binaries. |
| Rebuilding geometry produces new code with a new contract. | MUST | Produce a distinct artifact with one newly generated bundle contract and hash covering rebuilt code and any retained binaries. Preserve the original bundle. |
The source-package encoding, build-description schema, and consumer options remain to be specified. Source files and build metadata do not introduce per-artifact ABI contracts; the resulting compiled bundle still has one contract.
Dependency and native archive handling
If geometry embeds math revision A, installing math revision B alongside it does not replace the already embedded instructions. Similarly, isolating two archives’ symbol names prevents a name collision; it does not make their allocators interchangeable.
Current bundles include linked bitcode from compiled dependencies. The dependency list is not a dynamic-loader instruction to replace that code with the newest available library. Exact referenced declaration identities remain significant when importing metadata and specializing generic bodies.
Native static archives are carried with an archive-set identity. Their symbols are isolated using that identity, and the same carried archive set can be deduplicated. This isolates names; it does not establish compatible C runtimes, allocation families, process-wide state, or platform requirements. Those remain part of the runtime and ownership contracts.
Bundle producer and compatibility contract
geometry.moon
artifact identity: which exact library implementation is this?
bundle contract: what does all its compiled code require?
producer metadata: which compiler built it?Changing a producer label alone does not change the required layouts or calls. Changing an embedded dependency’s required runtime behavior does.
The metadata stores one bundle contract hash
and the information needed to derive its complete contract. Existing type,
declaration, and target metadata can supply part of that description. The experimental schema is include/abi/compatibility.proto, referenced once
through MoonLibrary.bundle_contract. The container header references that
bundle metadata directly. External native symbol references are part of this
same protobuf (native_link_symbols) as diagnostic metadata, excluded from the
compatibility hash and import authorization. Embedded dependency diagnostics are
retained; no per-archive compatibility hash is introduced.
| Minimal example | Level | Requirement |
|---|---|---|
| Geometry embeds math code requiring an extra runtime service. | MUST | Cover the bundle's code, generic bodies, and all embedded dependency requirements with one contract and hash. |
| Only the recorded compiler build name changes. | MUST | Store the bundle producer's build identity and ordering separately from the contract. |
| Three embedded libraries are described by the outer bundle’s contract. | MUST NOT | Require separate compatibility manifests, hashes, or compiler-age metadata for embedded artifacts. |
| An old generic refers to a helper from one exact artifact. | MUST | Preserve exact identities and references needed by declarations and generic bodies. |
The compatibility procedure accepts or rejects the complete bundle contract. Older compatible bundles produce warnings. Differing experimental compiler fingerprints also warn after validation. Unsupported required contracts and concrete mismatches reject the compiled payload. If the consumer selects source recompilation, it validates the rebuilt artifact instead. The producer's version does not authorize an import.
Proposed encoding requirements
Compare two new fields: an optional build label can be skipped without changing execution; a new required interpretation of a generic operation cannot. The schema must distinguish those cases rather than reject every unfamiliar field or accept all of them.
The proposed encoding uses fixed-width little-endian container integers, byte-counted UTF-8 text, and defined offsets. Current files do not yet follow this proposal. Target payload byte order is a separate property.
| Minimal example | Level | Requirement |
|---|---|---|
| A reader on another host must interpret the same header integers. | MUST | Define an encoding independent of C++ padding and producer host byte order. |
| A reader needs to know whether a length counts bytes or entries. | MUST | Define header bytes, existing indexes, bundle metadata fields, shared ranges, strings, resource limits, and integrity checks. |
| An unknown debug label differs from an unknown required call rule. | MUST | Provide a way to mark required capabilities without making every unknown field mandatory. |
| An older reader skips an added diagnostic label. | MUST | Use tagged, skippable metadata fields with stable defaults. Older readers may ignore an unknown optional field only when doing so leaves import and execution behavior unchanged. |
| Ignoring a new field would change how a generic body is interpreted. | MUST | Mark new semantics as required when ignoring them would change interpretation or behavior. Readers that cannot satisfy those requirements must reject the bundle. |
| A field formerly storing a name must not start storing a layout. | MUST NOT | Reuse a field identifier for a different meaning or change an existing field's meaning. |
| A producer adds only a safely skippable diagnostic field. | MUST NOT | Require a new container revision solely for additive optional metadata that existing readers can safely skip. |
| A compiled-code reader does not understand the embedded source format. | MUST | Allow compiled-code consumers to skip an unsupported optional source package without changing the compiled contract. |
| An old generic reader encounters a required unknown operation. | MUST | Reject unknown required features under the generic semantics contract. |
| A tiny file claims to contain billions of module records. | MUST | Validate them before allocation. |
| A payload begins inside the file but ends past its last byte. | MUST | Check ranges without integer overflow; reject truncation, invalid references, and conflicting duplicate identities. |
| Two modules share exactly one bitcode image; an archive overlaps only half of it. | MUST NOT | Accept arbitrary overlap between unrelated records. Shared bitcode ranges are allowed; exact overlap rules still need a specification. |
| An input hash stays fixed while serialized output bytes are corrupted. | MUST NOT | Treat build-input identity as a checksum of serialized bytes. |
| Check the actual payload bytes independently of the build-input identity. | MUST | Define payload-integrity digests separately from artifact identity. |
A digest does not prove that an untrusted producer followed Sun's safety rules.
Import validation order
For the geometry file, validate its byte ranges before reading its declarations,
then validate those declarations and its payload before app uses them. Reading
the file must not run geometry’s startup code as a side effect.
For compiled-code imports, the proposed reader performs these checks before generated code uses the bundle:
- Validate the header, container version, index bounds, and record encoding.
- Decode supported metadata and reject missing required capabilities.
- Validate declaration references, exact dependencies, and duplicate identities.
- Validate the complete bundle contract and its hash against supported implementations and the target.
- Validate imported declarations, layouts, and signatures under that contract before lowering dependent code.
- Load or link supported payloads and specialize supported generic bodies.
| Minimal example | Level | Requirement |
|---|---|---|
| Lazy loading must validate a function before its first use. | MUST | Complete the checks above before using the compiled bundle. For selected source recompilation, validate the source inputs and resulting artifact before use. Lazy loading must reject incompatible data before use. |
| Listing module names must not execute library initializers. | MUST NOT | Run startup code merely because the container is being parsed. |
Format decisions still required
- Exact byte-level specification and migration from native header encoding.
- Bounds, resource limits, overlap rules, and integrity coverage.
- Whether future bundles carry native Sun objects, optional bitcode, or both.
- Stable field identifiers, extension defaults, older readers, and supported revision lifetimes.
- Versioned interface and generic-body schemas with reference validation.
- Optional source-package encoding, completeness rules, build descriptions, and consumer recompilation options.
| Minimal example | Level | Required tests |
|---|---|---|
| Keep a saved old bundle instead of rebuilding it with today’s compiler. | MUST | Retain old artifacts in the conformance suite. |
| Truncate a valid file, then try an overflowing payload length. | MUST | Cover malformed offsets, overflowing lengths, truncation, shared blobs, duplicate records, unknown capabilities, and unsupported payloads. |
Compiled payloads
Moon currently carries LLVM bitcode, declarations, and generic bodies. The initial design can keep this model.
| Minimal example | Level | Requirement |
|---|---|---|
| LLVM reads the module, but its Sun signature disagrees with the metadata. | MUST | Cover bitcode reader compatibility, Sun ABI compatibility, and generic semantics in the single bundle contract; matching one alone is insufficient. |
LLVM's backward bitcode compatibility helps newer consumers load older code; it does not define Sun layouts or permit older readers to consume arbitrary future bitcode. See the LLVM compatibility policy (opens in a new tab).
Native objects with optional bitcode are a possible future packaging design. They would reduce dependence on bitcode readers while preserving the need for compatible metadata, generic specialization, platform ABI, and runtime services. No payload-format migration is required by this draft.