cinder/core guide

cinder/core is the lowest Mojo layer of the engine: logging, assertions, a bump arena, generational handles and pools, a bounded ring, byte packing and hashing, file helpers, a monotonic clock with a fixed-step accumulator, CPU images, the job system, Tracy zones, Boxed heap cells and the SIMD math package. It performs no GPU, window or audio I/O. Everything above it (platform, gfx, ecs, render, ...) depends on it; it depends only on the generated bindings in cinder/ffi and, for two modules that call into the shim (jobs, image, profile), on libcinder_native being loaded.

Note on layering: core/profile.mojo and core/image.mojo import NativePtr from cinder.platform.native so they can call the shim through the single loaded library. That is the one upward reference in the module map of docs/ARCHITECTURE.md; core/jobs.mojo takes a raw Pointer[CinderNative, MutUntrackedOrigin] instead and does not import platform.

Source: cinder/core/*.mojo, cinder/core/math/*.mojo. Reference: docs/api/core.md. Tests: tests/core/test_core.mojo, test_math.mojo, test_jobs.mojo, test_image.mojo, test_profile.mojo.

Key types #

Logging #

Logger[S] filters on min_level and forwards a LogRecord to its sink. Sinks implement the LogSink trait (emit(mut self, record: LogRecord)) and must not raise. Records print as one line, [LEVEL] category: message key=value .... The convenience methods trace, debug, info, warn, error take a category and a message; the full log overload also takes a List[LogField], which it consumes.

from cinder.core.log import LogField, LogLevel, Logger, MemorySink

var logger = Logger[MemorySink](MemorySink(), LogLevel.info())
logger.debug("gfx", "dropped")           # below min_level, filtered
logger.info("gfx", "device created")
var fields = List[LogField]()
fields.append(LogField("adapter", "lavapipe"))
logger.log(LogLevel.warn(), "gfx", "slow", fields^)
# logger.sink.records[1] prints as "[WARN] gfx: slow adapter=lavapipe"
logger.min_level = LogLevel.trace()      # the filter can be changed at any time

Other subsystems do not hold a logger. Platform, Gfx and ShaderManager collect LogRecords in a diagnostics list and hand them over through take_diagnostics(); the owner forwards them to its logger once per frame.

Assertions #

cinder_assert(condition, message) is compiled out when the build defines CINDER_ASSERTS=0 (mojo build -D CINDER_ASSERTS=0); cinder_check is never compiled out and guards memory safety (handle validation, FFI preconditions). Both call std.os.abort with cinder assertion failed: ... or cinder check failed: .... Recoverable conditions raise instead; a failed assertion is a bug. Tests verify the abort in a child process with std.testing.assert_aborts._assert_aborts (the module name shadows the function, so the underscore form is called directly).

Arena #

LinearArena(capacity) allocates one block (plus one cache line so any alignment up to 64 can be served) and bumps an offset. alloc_bytes(size, alignment=16) returns a Pointer[UInt8, MutUntrackedOrigin]; alloc[T](count) returns uninitialised, T-aligned storage. reset() makes everything available again and high_water() is the number to size the arena by.

from cinder.core.arena import LinearArena

var arena = LinearArena(1024)
var p16 = arena.alloc_bytes(10, 16)        # 16-byte aligned
var f = arena.alloc[Float64](4)            # aligned for Float64
f[] = 1.5
f[unsafe_offset=3] = 2.5
arena.reset()                              # every earlier pointer now dangles
var again = arena.alloc_bytes(3, 1)        # same address as the first allocation

Raises on a non-positive capacity, on an alignment that is not a power of two in [1, 64], on a negative size and when the block is exhausted (LinearArena exhausted: ...). Filling it exactly is allowed.

Handles and pools #

Handle[Tag] is a 64-bit generational handle (32-bit slot, 32-bit generation) whose Tag is a phantom type: a Handle[TextureTag] cannot be passed where a Handle[BufferTag] is expected. packed() and from_packed() round-trip through one UInt64 for serialisation. Pool[T, Tag](capacity) is a slot map with O(1) insert, get, update, remove; generations start at 1 and skip 0 on wrap, so the zero handle is never valid. Slots are never reallocated: the pool holds at most capacity values and insert raises Pool is full instead of growing, which keeps steady-state frames allocation-free.

from cinder.core.handles import Handle, Pool

struct TextureTag:
    var _unused: UInt8

var pool = Pool[String, TextureTag](4)
var a = pool.insert(String("a"))
var removed = pool.remove(a)               # "a"; a is now stale
var c = pool.insert(String("c"))           # reuses a's slot with a new generation
# c.index == a.index, c.generation != a.generation
# pool.get(a) raises "invalid handle Handle(0@1)"
pool.update(c, String("c2"))

get returns a copy (pooled values are small records); handles() allocates and is for tools and tests. is_valid is the non-raising check. cinder/gfx/resources.mojo builds every GPU resource pool on this type.

Ring buffer, bytes, hashing #

RingBuffer[T](capacity) requires a power-of-two capacity (>= 2). push returns False when full without dropping anything queued, pop returns None when empty, peek raises when empty. It is single-threaded by design: cross-thread queues live in the shim (CnEventQueue), see ADR-0003.

ByteWriter appends little-endian u8, u32, i32, f32, vec2, vec3, vec3_padded, vec4 and column-major mat4 values for GPU uploads; pad_to(16) aligns for std140 rules (the layout itself is the caller's job). pad_to needs a positive alignment: 0 or a negative value is a bug that cinder_assert aborts on (with assertions compiled out it pads nothing). reset() keeps capacity for frame scratch; take() consumes the writer. fnv1a64 / fnv1a64_string are the stable content hashes used by the asset cooker; hex64 formats them. decode_base64 accepts standard and URL-safe alphabets, skips whitespace and treats padding as optional; it raises on any other character.

Filesystem and time #

read_bytes, read_text, write_bytes, write_text wrap open and raise FileError(path, message), declared as raises FileError, so a caller sees the path in the error text (FileError(path: message)). file_exists never raises.

Clock is a monotonic nanosecond clock; tick() returns seconds since the previous tick and is never negative. FixedStep(hz=60.0, max_steps_per_frame=8) is the accumulator that drives the simulation:

from cinder.core.time import FixedStep

var step = FixedStep(hz=60.0, max_steps_per_frame=4)
var n = step.advance(frame_seconds)        # simulation steps to run this frame, capped at 4
var alpha = step.alpha()                   # [0, 1): interpolation between the last two states

When the cap is reached with more than a step still owed, the backlog is dropped rather than stalling rendering. Negative frame times count as zero.

Images #

Image(width, height) is a tightly packed RGBA8 buffer. load_image(lib, path) and decode_image(lib, bytes) go through the shim's stb wrappers and raise with cn_image_failure_reason() in the message; save_png raises when the file cannot be written. Image.from_rows(width, height, rows, stride) repacks a GPU readback whose rows are padded to the 256-byte copy alignment; swap_rb() converts BGRA swapchain readbacks. pixel(x, y) and set_pixel(x, y, v) abort through cinder_check (in every build) when x is outside [0, width) or y outside [0, height). compare(a, b, tolerance=2, window=8) returns an ImageDiff with SSIM on luma over non-overlapping windows plus max_channel_delta and differing_pixels; it raises when the sizes differ. Golden-image tests bound all three.

Job system #

JobSystem(lib, worker_count, job_capacity=1 << 14) starts worker_count threads (cinder-worker, created through cn_thread_create) each owning a Chase-Lev deque of job indices; deque 0 belongs to the main thread. A job is a thin function pointer JobFn = def(Pointer[NoneType, MutUntrackedOrigin]) thin -> None plus an untyped data pointer and an optional Counter. submit queues from the main thread and returns False when the job table or the main deque is full: the caller must then run the job inline. wait(counter) runs jobs on the calling thread until the counter reaches zero, so waiting never idles a core. reset() recycles the job table and is legal only when nothing is queued or running (frame start). With worker_count == 0 everything runs on the thread that waits.

from cinder.core.jobs import Counter, JobFn, JobSystem

def _sum_chunk(data: Pointer[NoneType, MutUntrackedOrigin]) -> None:
    # SAFETY: data was submitted as a ChunkJob kept alive until the counter reaches zero.
    var job = data.unsafe_bitcast[ChunkJob]()[]
    ...

var lib = NativeLibrary()
var jobs = JobSystem(lib.ptr(), 3)                     # three worker threads
var counter = Counter()
var counter_ptr = Pointer(to=counter).unsafe_mut_cast[True]().unsafe_origin_cast[MutUntrackedOrigin]()
for c in range(CHUNKS):
    if not jobs.submit(_sum_chunk, desc_ptr.unsafe_offset(c).unsafe_bitcast[NoneType](), Optional(counter_ptr)):
        _sum_chunk(desc_ptr.unsafe_offset(c).unsafe_bitcast[NoneType]())   # saturated: run inline
jobs.wait(counter_ptr)
keep(descriptors)
keep(counter)

Construction raises on a worker count outside [0, 64], a non-power-of-two capacity, or a failed semaphore or thread creation (workers already started are stopped and joined first). The orderings are the conservative ones from ADR-0008; executed() is a diagnostic count.

Profiling #

Profiler(lib) wraps the shim's Tracy client. Nothing listens on the network until start(), which is idempotent and returns False when the shim was built without Tracy (available()). While stopped, zone, plot, message and frame_mark are no-ops and Zone.id is 0. A zone is one line:

from cinder.core.profile import COLOR_RENDER, Profiler

var profiler = Profiler(lib.ptr())
_ = profiler.start()
var z = profiler.zone(String("frame"), COLOR_RENDER)
z.text(String("frame body"))
_ = z^                      # ends the zone; otherwise it ends at the last use of `z`
profiler.frame_mark()

Tracy copies zone names, zone text and messages during the call, but keys plots by the name pointer and reads it until the process exits. plot(name, value) therefore interns each distinct name once (a never-freed copy shared by every Profiler copy; plot_name_count() reports how many) and hands Tracy that copy, so a temporary String is safe and repeated plots of a name do not allocate. Plots sent while profiling is stopped are dropped without interning.

C strings #

string_from_c(ptr, max_len=-1) turns text returned by C (SDL errors, driver names, SDL log lines, stb failure reasons) into a String. It decodes the bytes as UTF-8, replacing invalid sequences with U+FFFD; building a String one chr(byte) at a time would double-encode every non-ASCII character. max_len caps the scan for fixed-size C buffers without a guaranteed terminator.

Boxed #

Boxed[T](value^) owns T at a stable heap address. Subsystems that hand out pointers to themselves (Gfx, Resources, JobSystem, the renderer) must not move after those pointers exist, so the App runtime keeps them in Boxed cells and passes box.ptr(); box[] is a mutable reference.

Math #

Fixed conventions (docs/ARCHITECTURE.md): right-handed, Y up, -Z forward, metres, column-major matrices, reverse-Z depth with an infinite far plane. Vec3 keeps its fourth SIMD lane at zero. Mat4 stores columns c0..c3; m[row, col] indexes an element, column(c) and row(r) return Vec4s. Indices outside 0..3 are bugs: cinder_assert aborts (with assertions compiled out the index wraps to i & 3). Constructors: identity, zero, from_rows, translation, scaling, rotation(q), trs, look_at, perspective_reverse_z_infinite(fov_y, aspect, near), orthographic_reverse_z(l, r, b, t, near, far). Point and vector transforms are transform_point, transform_vector, project_point (perspective divide). Quat offers from_axis_angle, from_euler(yaw, pitch, roll), rotate, nlerp, slerp, inverse (the zero quaternion returns the zero quaternion, like Mat4.inverse of a singular matrix); Transform is a TRS triple with to_mat4, from_mat4 (no shear; a negative determinant folds into the x scale and the rotation is taken from the un-mirrored basis, so to_mat4() rebuilds the input), __mul__ (parent * child) and lerp.

from cinder.core.math import Mat4, Vec3, approx_eq, radians

var proj = Mat4.perspective_reverse_z_infinite(radians(60.0), 16.0 / 9.0, 0.1)
var on_near = proj.project_point(Vec3(0.0, 0.0, -0.1))     # depth 1.0
var far = proj.project_point(Vec3(0.0, 0.0, -100000.0))    # depth approaches 0.0
var view = Mat4.look_at(Vec3(1.0, 2.0, 3.0), Vec3(1.0, 2.0, -7.0), Vec3.unit_y())

Comparisons use approx_equal(other, eps=EPSILON); approx_eq is absolute near zero and relative elsewhere. tests/core/test_math.mojo checks every operation against Float64 reference loops over Philox-seeded inputs.

Threading and error handling #

• Everything in core except JobSystem is single-threaded and owned by one thread. RingBuffer, LinearArena, Pool and the loggers have no internal synchronisation.
• JobSystem is the only place Mojo code runs on a thread Cinder did not inherit from the OS loader: _worker_entry is an abi("C") function invoked by the shim's thread (ADR-0008, spike 4). Job data must stay valid until the counter it was submitted with reaches zero; std.benchmark.keep the descriptors and the counter after wait. submit and reset are main-thread calls.
• Raises (recoverable, Error unless noted): LinearArena construction and allocation, Pool construction, insert, get, update, remove, RingBuffer construction and peek, the fs functions (FileError), Image construction from a wrong-sized buffer, from_rows, load_image, decode_image, save_png, compare, decode_base64, JobSystem construction.
• Aborts: cinder_assert, cinder_check. Out-of-range List indexing in Mojo also aborts; the handle and ring types check their indices before touching storage.
• Returns a status instead of raising: RingBuffer.push (False when full), JobSystem.submit (False when saturated), Profiler.start (False without Tracy).
• Logging never raises; a sink that fails drops the record.

Pitfalls #

• Untracked pointers do not keep their pointee alive (docs/MOJO_NOTES.md). Everything handed to C or to a job by pointer (Image.pixels, a String turned into a C string, job descriptors) must be followed by keep(value) after the last call that can read it. profile.mojo and image.mojo do this internally for their own temporaries; callers do it for job data.
• Pointer(to=x) cast to MutUntrackedOrigin ends the compiler's view of x; _ = x is not a read.
• A struct that stores pointers to its own fields breaks when moved. Box it (Boxed) and hand out box.ptr().
• String has inline small-string storage; a C pointer from as_c_string_span() dies when the owning List[String] reallocates. Hand long-lived C strings as List[UInt8].
• Pool.get copies; write back with update. A Handle whose slot was removed and reused fails validation rather than aliasing the new value.
• LinearArena.reset() invalidates every pointer it ever returned; nothing checks this at run time.
• JobSystem.reset() while jobs are queued corrupts the job table. The frame loop calls it once at frame start.
• FixedStep drops backlog beyond max_steps_per_frame steps; if simulation must never skip, raise the cap, do not loop on advance.
• List[...] = [x] literals allocate every time; List.clear() keeps capacity. Per-frame scratch is a persistent field cleared each frame, never a fresh list (docs/MOJO_NOTES.md, Phase 8).
• A destructor that aborts on leaks hides the real failure when a test raises; count and assert in tests instead.

Related documents #

• docs/ARCHITECTURE.md (module map, conventions), docs/THREADING.md (threads and shared state).
• docs/MOJO_NOTES.md (lifetime rules that every unsafe_ call here relies on).
• ADR-0003 docs/adr/0003-native-shim-pull-model.md: why cross-thread queues are in C and RingBuffer is single-threaded.
• ADR-0008 docs/adr/0008-job-system-threads.md: job system design.
• Tests: tests/core/test_core.mojo (handles, arena, ring, time, fs, logging, assertions, ByteWriter.pad_to bounds, UTF-8 C strings), tests/core/test_math.mojo, tests/core/test_jobs.mojo (contention, saturation, zero workers), tests/core/test_image.mojo (PNG round trip, SSIM), tests/core/test_profile.mojo.