igneum-network/igneum
1
0
Fork 0
Code Issues Pull requests Releases Activity
igneum/igneum-pow/src/derive.rs
igneum-labs 50ebdd64d9 Counter ASIC 3.0 item 2: the per-day item-derivation program (class dr736), its interpreter, emitter and packs 
A prototype behind a new LoadClass field (derive_len) and Shape field, Shape::for_class_day: every mixer slot of
the item derivation runs a straight-line program of 736 instructions drawn from the day key stream (the same
SplitMix64 stream, after the 40 mixer draws), twelve two-register forms, the chain rule of SuperscalarHash made
strict (every instruction reads the register the previous one wrote), an acceptance test with the x8 mixer's
operation and multiply counts from the code as floors (72 x 144 as written, 72 x 128 hoisted, 1,152 multiplies).
The verifier runs the program with a word-major (SoA) interpreter over the 32 items of a load, dispatching on
instruction pairs; no JIT. The emitter writes mh_round_0..8 into memhard.h, memhard.metal and kernel.cl. Packs
dr736-genesis and dr736-devnet-epoch0 under proto-cuda/packs-ca3-derive. The v2 and v3 paths are untouched: every
pinned pack re-exports byte for byte (tests/packs.rs), cargo test -p igneum-pow 58 + 7 + 4 + 19 + 7 green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 07:41:23 +00:00

978 lines
43 KiB
Rust
Raw Blame History

//! The per-day item-derivation program (Counter ASIC 3.0 item 2, `docs/plans/counter-asic-3-derivation.md`):
//! RandomX's SuperscalarHash idea (`vendor/RandomX/src/superscalar.cpp`, read 6 October 2026 at commit 7607fb2)
//! rebuilt for a 16-word item on a GPU. In place of the fixed-shape mixer `M_r` of spec 01 section 1.8.4, each of
//! the nine mixer slots of an item (one before each of the 8 dependent cache reads, one after the last) runs a
//! straight-line program of [`DERIVE_LEN`] instructions drawn once a day from the day key stream, from a fixed
//! set of twelve two-register forms. The 8 dependent cache reads per item are untouched.
//!
//! Rules of the draw (the dependency chain of SuperscalarHash, made strict):
//! * every instruction reads the chain register `c`, the register the previous instruction wrote (`s[0]`, the
//! address word, at the start of each round program), and writes a register `d != c`, which becomes the chain;
//! so no two instructions of a program can run in parallel, and no two consecutive instructions write one
//! register (the "ror r,C1; ror r,C2" and "xor r,r2; xor r,r2" merges of SuperscalarHash's `selectDestination`
//! cannot arise);
//! * every form is a bijection on the 16-word state (the old `d` enters through `+=`, `-=`, `^=`, an odd multiply,
//! or a rotation of itself), so a program loses no entropy, the property `M_r` has;
//! * the forms are integer only, modulo 2^32, with rotations by 1..31: bit-exact on Metal, CUDA and OpenCL by the
//! same argument as the lottery hash's families (spec 01 section 1.14); no division, no float, no branch;
//! * four draws per instruction in a fixed order, so the stream position of every draw is fixed by the index.
//!
//! The acceptance test ([`DeriveProgram::check`]) rejects a degenerate draw and the next attempt is drawn from the
//! continuation of the stream, the rule the program generator uses (spec 01 section 1.4.6).
use crate::memhard::ITEM_ROUNDS;
use crate::seed::SplitMix64;
/// Registers of the item state (the item is 16 words).
pub const DERIVE_REGS: usize = 16;
/// Round programs per item: one before each cache read and one after the last (`ITEM_ROUNDS + 1`).
pub const DERIVE_PROGRAMS: usize = ITEM_ROUNDS + 1;
/// Instructions per round program for the x8-equivalent operation count (the candidate, class "dr736"): 9 x 736
/// = 6,624 instructions per item at a mean of 1.62 GPU operations (1.52 chip operations) each, about 10,730 GPU
/// operations, 10,070 chip operations and 1,460 multiplies per item. The x8 mixer, counted from the code
/// (`memhard::mixer`, 16 x (xor, add, mul) + 8 quarter rounds x 12 = 144 operations as written, 128 with the
/// `RC + rk` adds hoisted as constants, 16 multiplies; `chip-model-v3.md` section 1 prices 130 from the spec text):
/// 72 applications = 10,368 as written, 9,216 hoisted, 1,152 multiplies. The floors below are those three.
pub const DERIVE_LEN_X8: u32 = 736;
/// Draws per instruction: the op roll, the destination roll, the second-source roll and the immediate.
pub const DRAWS_PER_INSTR: u64 = 4;
/// The floor of chip operations per item (the x8 mixer with its constants hoisted: 72 x 128).
pub const OPS_FLOOR_X8: u64 = 9_216;
/// The floor of GPU operations per item (the x8 mixer as written: 72 x 144).
pub const GPU_OPS_FLOOR_X8: u64 = 10_368;
/// The floor of multiplies per item (the x8 mixer's 72 x 16).
pub const MULS_FLOOR_X8: u64 = 1_152;
/// Distinct rotation amounts an item's programs must use, at least.
pub const DISTINCT_ROTS_FLOOR: usize = 8;
/// Attempts before the generator gives up (never reached: see [`DeriveProgram::draw`]).
pub const MAX_ATTEMPTS: u32 = 64;
/// The twelve forms. `c` is the chain register (the previous destination), `d` the destination (`d != c`), `b` a
/// third register (`b != d`, `b != c`), `k` a rotation in 1..31, `i` a 32-bit constant (odd for `MulC`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
#[repr(u8)]
pub enum DOp {
/// `d += c`
Add = 0,
/// `d -= c`
Sub = 1,
/// `d ^= c`
Xor = 2,
/// `d *= (c OR 1)`: the multiply-lo form, odd so it is a bijection on `d`
Mul = 3,
/// `d = rotl(d, k) + c`
Rot = 4,
/// `d = rotl(d ^ c, k)`
XRot = 5,
/// `d += c + i`
AddC = 6,
/// `d ^= c ^ i`
XorC = 7,
/// `d = (d ^ c) * i`, `i` odd: the per-word form of `M_r` with the chain in place of the round constant
MulC = 8,
/// `d = d * i + c`, `i` odd
MulC2 = 9,
/// `d ^= (c AND b)`
AndX = 10,
/// `d += (c OR b)`
OrX = 11,
}
/// The op weights in percent, in draw order (sum 100). Fixed at genesis; only the order, the registers and the
/// constants are drawn.
pub const DOP_WEIGHTS: [(DOp, u64); 12] = [
(DOp::Add, 14),
(DOp::Sub, 10),
(DOp::Xor, 14),
(DOp::Mul, 10),
(DOp::Rot, 10),
(DOp::XRot, 10),
(DOp::AddC, 6),
(DOp::XorC, 6),
(DOp::MulC, 8),
(DOp::MulC2, 4),
(DOp::AndX, 4),
(DOp::OrX, 4),
];
impl DOp {
pub fn from_u8(v: u8) -> Option<DOp> {
DOP_WEIGHTS.iter().map(|(o, _)| *o).find(|o| *o as u8 == v)
}
pub fn name(self) -> &'static str {
match self {
DOp::Add => "add",
DOp::Sub => "sub",
DOp::Xor => "xor",
DOp::Mul => "mul",
DOp::Rot => "rot",
DOp::XRot => "xrot",
DOp::AddC => "addc",
DOp::XorC => "xorc",
DOp::MulC => "mulc",
DOp::MulC2 => "mulc2",
DOp::AndX => "andx",
DOp::OrX => "orx",
}
}
/// Integer operations as a GPU executes the form (every `|`, `&`, `+`, `^`, `*`, rotate counts one).
pub fn gpu_ops(self) -> u64 {
match self {
DOp::Add | DOp::Sub | DOp::Xor => 1,
_ => 2,
}
}
/// Integer operations as the chip model counts them (`c OR 1` is a wire on a chip, so `Mul` is one multiply;
/// a constant folded into a chain value is still an add or an xor, so every other two-op form stays two).
pub fn chip_ops(self) -> u64 {
match self {
DOp::Add | DOp::Sub | DOp::Xor | DOp::Mul => 1,
_ => 2,
}
}
pub fn is_mul(self) -> bool {
matches!(self, DOp::Mul | DOp::MulC | DOp::MulC2)
}
pub fn has_rot(self) -> bool {
matches!(self, DOp::Rot | DOp::XRot)
}
pub fn has_third(self) -> bool {
matches!(self, DOp::AndX | DOp::OrX)
}
pub fn has_imm(self) -> bool {
matches!(self, DOp::AddC | DOp::XorC | DOp::MulC | DOp::MulC2)
}
/// The op of a roll in 0..99.
pub fn for_roll(roll: u64) -> DOp {
let mut acc = 0u64;
for (op, w) in DOP_WEIGHTS {
acc += w;
if roll < acc {
return op;
}
}
DOp::OrX
}
}
/// One instruction. `src` is the chain register (carried so the interpreter and the emitter need no state).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct DInstr {
pub op: DOp,
pub dst: u8,
pub src: u8,
/// The third register of `AndX` and `OrX`; 0 on every other form (drawn and unused).
pub src2: u8,
/// The rotation 1..31 of `Rot` and `XRot`, else 0.
pub rot: u8,
/// The constant of `AddC`, `XorC` (any), `MulC` and `MulC2` (odd); 0 on every other form.
pub imm: u32,
}
/// The nine round programs of an item for one day, with the attempt that passed the acceptance test.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct DeriveProgram {
pub len: u32,
pub attempt: u32,
pub rounds: Vec<Vec<DInstr>>,
}
/// Why a candidate was rejected.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum DeriveReject {
/// A register no instruction of round program `round` writes.
RegisterNeverWritten { round: usize, reg: u8 },
/// Fewer than [`DISTINCT_ROTS_FLOOR`] distinct rotation amounts over the item's programs.
RotationsDegenerate { distinct: usize },
/// Chip operations per item under [`OPS_FLOOR_X8`] scaled to the length.
OpsUnderFloor { ops: u64, floor: u64 },
/// GPU operations per item under [`GPU_OPS_FLOOR_X8`] scaled to the length.
GpuOpsUnderFloor { ops: u64, floor: u64 },
/// Multiplies per item under [`MULS_FLOOR_X8`] scaled to the length.
MulsUnderFloor { muls: u64, floor: u64 },
}
impl std::fmt::Display for DeriveReject {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
DeriveReject::RegisterNeverWritten { round, reg } => write!(f, "register {reg} never written in round program {round}"),
DeriveReject::RotationsDegenerate { distinct } => write!(f, "only {distinct} distinct rotation amounts"),
DeriveReject::OpsUnderFloor { ops, floor } => write!(f, "{ops} chip operations per item, floor {floor}"),
DeriveReject::GpuOpsUnderFloor { ops, floor } => write!(f, "{ops} GPU operations per item, floor {floor}"),
DeriveReject::MulsUnderFloor { muls, floor } => write!(f, "{muls} multiplies per item, floor {floor}"),
}
}
}
impl DeriveProgram {
/// Draw one candidate of `len` instructions per round program from `rng` (four draws per instruction).
pub fn draw_candidate(rng: &mut SplitMix64, len: u32, attempt: u32) -> DeriveProgram {
let mut rounds = Vec::with_capacity(DERIVE_PROGRAMS);
for _ in 0..DERIVE_PROGRAMS {
let mut prog = Vec::with_capacity(len as usize);
let mut chain = 0u8;
for _ in 0..len {
let op = DOp::for_roll(rng.below(100));
// the destination: the 15 registers other than the chain, in ascending order
let d_roll = rng.below((DERIVE_REGS - 1) as u64) as u8;
let dst = if d_roll >= chain { d_roll + 1 } else { d_roll };
// the third register: the 14 registers other than dst and the chain, in ascending order
let b_roll = rng.below((DERIVE_REGS - 2) as u64) as u8;
let (lo, hi) = if dst < chain { (dst, chain) } else { (chain, dst) };
let mut b = b_roll;
if b >= lo {
b += 1;
}
if b >= hi {
b += 1;
}
let x = rng.next() as u32;
let mut ins = DInstr { op, dst, src: chain, src2: 0, rot: 0, imm: 0 };
if op.has_third() {
ins.src2 = b;
}
if op.has_rot() {
ins.rot = 1 + (x % 31) as u8;
}
if op.has_imm() {
ins.imm = if matches!(op, DOp::MulC | DOp::MulC2) { x | 1 } else { x };
}
prog.push(ins);
chain = dst;
}
rounds.push(prog);
}
DeriveProgram { len, attempt, rounds }
}
/// Draw the program of a day: candidates from `rng` in turn until one passes [`DeriveProgram::check`].
/// Panics after [`MAX_ATTEMPTS`] (the floors sit more than 7 standard deviations under the expected counts, so
/// a rejection is a rare event and 64 in a row is not one that happens).
pub fn draw(rng: &mut SplitMix64, len: u32) -> DeriveProgram {
for attempt in 0..MAX_ATTEMPTS {
let p = Self::draw_candidate(rng, len, attempt);
if p.check().is_ok() {
return p;
}
}
panic!("derivation program: {MAX_ATTEMPTS} candidates rejected in a row");
}
/// The floors for this length (chip operations, GPU operations, multiplies): the x8 floors scaled by
/// `len / DERIVE_LEN_X8`, so a shorter class, measured as a fallback, has its own proportional floors.
pub fn floors(len: u32) -> (u64, u64, u64) {
let scale = |f: u64| f * len as u64 / DERIVE_LEN_X8 as u64;
(scale(OPS_FLOOR_X8), scale(GPU_OPS_FLOOR_X8), scale(MULS_FLOOR_X8))
}
/// The acceptance test: every register written in every round program; at least [`DISTINCT_ROTS_FLOOR`]
/// distinct rotation amounts; chip operations, GPU operations and multiplies per item at or above the floors
/// (the x8 mixer's counts from the code). The structural
/// rules (`dst != src`, the third register distinct, rotations in 1..31, odd multiplier constants) hold by
/// construction and are asserted.
pub fn check(&self) -> Result<(), DeriveReject> {
let mut rots = [false; 32];
for (r, prog) in self.rounds.iter().enumerate() {
let mut written = [false; DERIVE_REGS];
let mut chain = 0u8;
for ins in prog {
assert!(ins.src == chain && ins.dst != ins.src && (ins.dst as usize) < DERIVE_REGS, "chain rule");
if ins.op.has_third() {
assert!(ins.src2 != ins.dst && ins.src2 != ins.src && (ins.src2 as usize) < DERIVE_REGS, "third register");
}
if ins.op.has_rot() {
assert!((1..=31).contains(&ins.rot), "rotation");
rots[ins.rot as usize] = true;
}
if matches!(ins.op, DOp::MulC | DOp::MulC2) {
assert!(ins.imm & 1 == 1, "odd multiplier");
}
written[ins.dst as usize] = true;
chain = ins.dst;
}
if let Some(reg) = written.iter().position(|w| !w) {
return Err(DeriveReject::RegisterNeverWritten { round: r, reg: reg as u8 });
}
}
let distinct = rots.iter().filter(|r| **r).count();
if distinct < DISTINCT_ROTS_FLOOR {
return Err(DeriveReject::RotationsDegenerate { distinct });
}
let (ops_floor, gpu_floor, muls_floor) = Self::floors(self.len);
let ops = self.chip_ops();
if ops < ops_floor {
return Err(DeriveReject::OpsUnderFloor { ops, floor: ops_floor });
}
let gpu = self.gpu_ops();
if gpu < gpu_floor {
return Err(DeriveReject::GpuOpsUnderFloor { ops: gpu, floor: gpu_floor });
}
let muls = self.muls();
if muls < muls_floor {
return Err(DeriveReject::MulsUnderFloor { muls, floor: muls_floor });
}
Ok(())
}
pub fn instr_count(&self) -> u64 {
self.rounds.iter().map(|p| p.len() as u64).sum()
}
pub fn gpu_ops(&self) -> u64 {
self.rounds.iter().flatten().map(|i| i.op.gpu_ops()).sum()
}
pub fn chip_ops(&self) -> u64 {
self.rounds.iter().flatten().map(|i| i.op.chip_ops()).sum()
}
pub fn muls(&self) -> u64 {
self.rounds.iter().flatten().filter(|i| i.op.is_mul()).count() as u64
}
/// Count per op, in [`DOP_WEIGHTS`] order.
pub fn op_counts(&self) -> [u64; 12] {
let mut c = [0u64; 12];
for i in self.rounds.iter().flatten() {
c[i.op as usize] += 1;
}
c
}
/// "add=887 sub=..." in weight order.
pub fn op_mix(&self) -> String {
let c = self.op_counts();
DOP_WEIGHTS.iter().map(|(o, _)| format!("{}={}", o.name(), c[*o as usize])).collect::<Vec<_>>().join(" ")
}
/// FNV-1a 64 over the instruction stream (op, dst, src, src2, rot, imm as bytes): the program's fingerprint
/// for packs and logs.
pub fn fingerprint(&self) -> u64 {
let mut b = Vec::with_capacity(self.instr_count() as usize * 9);
for i in self.rounds.iter().flatten() {
b.push(i.op as u8);
b.push(i.dst);
b.push(i.src);
b.push(i.src2);
b.push(i.rot);
b.extend_from_slice(&i.imm.to_le_bytes());
}
crate::seed::fnv1a64(&b)
}
}
/// Lanes of the SoA interpreter: the verifier derives up to 32 distinct items per load (one per lane of the
/// unit), so each instruction runs across 32 item states at once and the dispatch is paid once per 32 items.
pub const SOA_LANES: usize = 32;
/// The item states of a batch, word-major: `st[reg][lane]`.
pub type SoaState = [[u32; SOA_LANES]; DERIVE_REGS];
#[inline(always)]
fn rotl(x: u32, n: u32) -> u32 {
x.rotate_left(n)
}
/// The twelve forms over a batch, one function each, every one a straight loop over the lanes the compiler
/// vectorises. The destination row and the source rows are distinct by the chain rule (`dst != src`, and the third
/// register distinct from both: asserted by [`DeriveProgram::check`] and checked here in debug builds), so the
/// rows are addressed through raw pointers rather than copied out of the state.
mod forms {
use super::{rotl, DInstr, SoaState, SOA_LANES};
#[inline(always)]
pub fn add(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = d.wrapping_add(*c);
}
}
}
#[inline(always)]
pub fn sub(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = d.wrapping_sub(*c);
}
}
}
#[inline(always)]
pub fn xor(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d ^= *c;
}
}
}
#[inline(always)]
pub fn mul(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = d.wrapping_mul(*c | 1);
}
}
}
#[inline(always)]
pub fn rot(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let r = ins.rot as u32;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = rotl(*d, r).wrapping_add(*c);
}
}
}
#[inline(always)]
pub fn xrot(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let r = ins.rot as u32;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = rotl(*d ^ *c, r);
}
}
}
#[inline(always)]
pub fn addc(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let i = ins.imm;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = d.wrapping_add(c.wrapping_add(i));
}
}
}
#[inline(always)]
pub fn xorc(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let i = ins.imm;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d ^= *c ^ i;
}
}
}
#[inline(always)]
pub fn mulc(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let i = ins.imm;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = (*d ^ *c).wrapping_mul(i);
}
}
}
#[inline(always)]
pub fn mulc2(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src);
let i = ins.imm;
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
*d = d.wrapping_mul(i).wrapping_add(*c);
}
}
}
#[inline(always)]
pub fn andx(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src && ins.src2 != ins.dst && ins.src2 != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
let bp = st.as_ptr().add(ins.src2 as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
let b = &*bp.add(k);
*d ^= *c & *b;
}
}
}
#[inline(always)]
pub fn orx(ins: &DInstr, st: &mut SoaState) {
debug_assert!(ins.dst != ins.src && ins.src2 != ins.dst && ins.src2 != ins.src);
// SAFETY: dst, src (and src2) are distinct registers below DERIVE_REGS, so the rows do not alias and the
// pointers stay inside `st`.
unsafe {
let dp = st.as_mut_ptr().add(ins.dst as usize) as *mut u32;
let cp = st.as_ptr().add(ins.src as usize) as *const u32;
let bp = st.as_ptr().add(ins.src2 as usize) as *const u32;
for k in 0..SOA_LANES {
let d = &mut *dp.add(k);
let c = &*cp.add(k);
let b = &*bp.add(k);
*d = d.wrapping_add(*c | *b);
}
}
}
}
/// One instruction over the batch (the single dispatch; [`run_round`] dispatches on pairs).
#[inline(always)]
pub fn run_instr(ins: &DInstr, st: &mut SoaState) {
match ins.op {
DOp::Add => forms::add(ins, st),
DOp::Sub => forms::sub(ins, st),
DOp::Xor => forms::xor(ins, st),
DOp::Mul => forms::mul(ins, st),
DOp::Rot => forms::rot(ins, st),
DOp::XRot => forms::xrot(ins, st),
DOp::AddC => forms::addc(ins, st),
DOp::XorC => forms::xorc(ins, st),
DOp::MulC => forms::mulc(ins, st),
DOp::MulC2 => forms::mulc2(ins, st),
DOp::AndX => forms::andx(ins, st),
DOp::OrX => forms::orx(ins, st),
}
}
/// Run one round program over the batch. The dispatch is on PAIRS of instructions (144 arms, one indirect branch
/// per two instructions): the op sequence of a drawn program is random, so the branch predictor misses most
/// dispatches, and the miss (about 3.7 ns of the 7.2 ns an instruction cost per batch on one M5 Max core, measured
/// with `examples/derive_perf.rs` on 6 October 2026, a functional run) is paid once per pair instead of once per
/// instruction. The result is bit for bit that of [`run_instr`] in sequence.
#[inline(never)]
pub fn run_round(prog: &[DInstr], st: &mut SoaState) {
let mut it = prog.chunks_exact(2);
for pair in &mut it {
let (a, b) = (&pair[0], &pair[1]);
match (a.op as u8) * 12 + b.op as u8 {
0 => { forms::add(a, st); forms::add(b, st); }
1 => { forms::add(a, st); forms::sub(b, st); }
2 => { forms::add(a, st); forms::xor(b, st); }
3 => { forms::add(a, st); forms::mul(b, st); }
4 => { forms::add(a, st); forms::rot(b, st); }
5 => { forms::add(a, st); forms::xrot(b, st); }
6 => { forms::add(a, st); forms::addc(b, st); }
7 => { forms::add(a, st); forms::xorc(b, st); }
8 => { forms::add(a, st); forms::mulc(b, st); }
9 => { forms::add(a, st); forms::mulc2(b, st); }
10 => { forms::add(a, st); forms::andx(b, st); }
11 => { forms::add(a, st); forms::orx(b, st); }
12 => { forms::sub(a, st); forms::add(b, st); }
13 => { forms::sub(a, st); forms::sub(b, st); }
14 => { forms::sub(a, st); forms::xor(b, st); }
15 => { forms::sub(a, st); forms::mul(b, st); }
16 => { forms::sub(a, st); forms::rot(b, st); }
17 => { forms::sub(a, st); forms::xrot(b, st); }
18 => { forms::sub(a, st); forms::addc(b, st); }
19 => { forms::sub(a, st); forms::xorc(b, st); }
20 => { forms::sub(a, st); forms::mulc(b, st); }
21 => { forms::sub(a, st); forms::mulc2(b, st); }
22 => { forms::sub(a, st); forms::andx(b, st); }
23 => { forms::sub(a, st); forms::orx(b, st); }
24 => { forms::xor(a, st); forms::add(b, st); }
25 => { forms::xor(a, st); forms::sub(b, st); }
26 => { forms::xor(a, st); forms::xor(b, st); }
27 => { forms::xor(a, st); forms::mul(b, st); }
28 => { forms::xor(a, st); forms::rot(b, st); }
29 => { forms::xor(a, st); forms::xrot(b, st); }
30 => { forms::xor(a, st); forms::addc(b, st); }
31 => { forms::xor(a, st); forms::xorc(b, st); }
32 => { forms::xor(a, st); forms::mulc(b, st); }
33 => { forms::xor(a, st); forms::mulc2(b, st); }
34 => { forms::xor(a, st); forms::andx(b, st); }
35 => { forms::xor(a, st); forms::orx(b, st); }
36 => { forms::mul(a, st); forms::add(b, st); }
37 => { forms::mul(a, st); forms::sub(b, st); }
38 => { forms::mul(a, st); forms::xor(b, st); }
39 => { forms::mul(a, st); forms::mul(b, st); }
40 => { forms::mul(a, st); forms::rot(b, st); }
41 => { forms::mul(a, st); forms::xrot(b, st); }
42 => { forms::mul(a, st); forms::addc(b, st); }
43 => { forms::mul(a, st); forms::xorc(b, st); }
44 => { forms::mul(a, st); forms::mulc(b, st); }
45 => { forms::mul(a, st); forms::mulc2(b, st); }
46 => { forms::mul(a, st); forms::andx(b, st); }
47 => { forms::mul(a, st); forms::orx(b, st); }
48 => { forms::rot(a, st); forms::add(b, st); }
49 => { forms::rot(a, st); forms::sub(b, st); }
50 => { forms::rot(a, st); forms::xor(b, st); }
51 => { forms::rot(a, st); forms::mul(b, st); }
52 => { forms::rot(a, st); forms::rot(b, st); }
53 => { forms::rot(a, st); forms::xrot(b, st); }
54 => { forms::rot(a, st); forms::addc(b, st); }
55 => { forms::rot(a, st); forms::xorc(b, st); }
56 => { forms::rot(a, st); forms::mulc(b, st); }
57 => { forms::rot(a, st); forms::mulc2(b, st); }
58 => { forms::rot(a, st); forms::andx(b, st); }
59 => { forms::rot(a, st); forms::orx(b, st); }
60 => { forms::xrot(a, st); forms::add(b, st); }
61 => { forms::xrot(a, st); forms::sub(b, st); }
62 => { forms::xrot(a, st); forms::xor(b, st); }
63 => { forms::xrot(a, st); forms::mul(b, st); }
64 => { forms::xrot(a, st); forms::rot(b, st); }
65 => { forms::xrot(a, st); forms::xrot(b, st); }
66 => { forms::xrot(a, st); forms::addc(b, st); }
67 => { forms::xrot(a, st); forms::xorc(b, st); }
68 => { forms::xrot(a, st); forms::mulc(b, st); }
69 => { forms::xrot(a, st); forms::mulc2(b, st); }
70 => { forms::xrot(a, st); forms::andx(b, st); }
71 => { forms::xrot(a, st); forms::orx(b, st); }
72 => { forms::addc(a, st); forms::add(b, st); }
73 => { forms::addc(a, st); forms::sub(b, st); }
74 => { forms::addc(a, st); forms::xor(b, st); }
75 => { forms::addc(a, st); forms::mul(b, st); }
76 => { forms::addc(a, st); forms::rot(b, st); }
77 => { forms::addc(a, st); forms::xrot(b, st); }
78 => { forms::addc(a, st); forms::addc(b, st); }
79 => { forms::addc(a, st); forms::xorc(b, st); }
80 => { forms::addc(a, st); forms::mulc(b, st); }
81 => { forms::addc(a, st); forms::mulc2(b, st); }
82 => { forms::addc(a, st); forms::andx(b, st); }
83 => { forms::addc(a, st); forms::orx(b, st); }
84 => { forms::xorc(a, st); forms::add(b, st); }
85 => { forms::xorc(a, st); forms::sub(b, st); }
86 => { forms::xorc(a, st); forms::xor(b, st); }
87 => { forms::xorc(a, st); forms::mul(b, st); }
88 => { forms::xorc(a, st); forms::rot(b, st); }
89 => { forms::xorc(a, st); forms::xrot(b, st); }
90 => { forms::xorc(a, st); forms::addc(b, st); }
91 => { forms::xorc(a, st); forms::xorc(b, st); }
92 => { forms::xorc(a, st); forms::mulc(b, st); }
93 => { forms::xorc(a, st); forms::mulc2(b, st); }
94 => { forms::xorc(a, st); forms::andx(b, st); }
95 => { forms::xorc(a, st); forms::orx(b, st); }
96 => { forms::mulc(a, st); forms::add(b, st); }
97 => { forms::mulc(a, st); forms::sub(b, st); }
98 => { forms::mulc(a, st); forms::xor(b, st); }
99 => { forms::mulc(a, st); forms::mul(b, st); }
100 => { forms::mulc(a, st); forms::rot(b, st); }
101 => { forms::mulc(a, st); forms::xrot(b, st); }
102 => { forms::mulc(a, st); forms::addc(b, st); }
103 => { forms::mulc(a, st); forms::xorc(b, st); }
104 => { forms::mulc(a, st); forms::mulc(b, st); }
105 => { forms::mulc(a, st); forms::mulc2(b, st); }
106 => { forms::mulc(a, st); forms::andx(b, st); }
107 => { forms::mulc(a, st); forms::orx(b, st); }
108 => { forms::mulc2(a, st); forms::add(b, st); }
109 => { forms::mulc2(a, st); forms::sub(b, st); }
110 => { forms::mulc2(a, st); forms::xor(b, st); }
111 => { forms::mulc2(a, st); forms::mul(b, st); }
112 => { forms::mulc2(a, st); forms::rot(b, st); }
113 => { forms::mulc2(a, st); forms::xrot(b, st); }
114 => { forms::mulc2(a, st); forms::addc(b, st); }
115 => { forms::mulc2(a, st); forms::xorc(b, st); }
116 => { forms::mulc2(a, st); forms::mulc(b, st); }
117 => { forms::mulc2(a, st); forms::mulc2(b, st); }
118 => { forms::mulc2(a, st); forms::andx(b, st); }
119 => { forms::mulc2(a, st); forms::orx(b, st); }
120 => { forms::andx(a, st); forms::add(b, st); }
121 => { forms::andx(a, st); forms::sub(b, st); }
122 => { forms::andx(a, st); forms::xor(b, st); }
123 => { forms::andx(a, st); forms::mul(b, st); }
124 => { forms::andx(a, st); forms::rot(b, st); }
125 => { forms::andx(a, st); forms::xrot(b, st); }
126 => { forms::andx(a, st); forms::addc(b, st); }
127 => { forms::andx(a, st); forms::xorc(b, st); }
128 => { forms::andx(a, st); forms::mulc(b, st); }
129 => { forms::andx(a, st); forms::mulc2(b, st); }
130 => { forms::andx(a, st); forms::andx(b, st); }
131 => { forms::andx(a, st); forms::orx(b, st); }
132 => { forms::orx(a, st); forms::add(b, st); }
133 => { forms::orx(a, st); forms::sub(b, st); }
134 => { forms::orx(a, st); forms::xor(b, st); }
135 => { forms::orx(a, st); forms::mul(b, st); }
136 => { forms::orx(a, st); forms::rot(b, st); }
137 => { forms::orx(a, st); forms::xrot(b, st); }
138 => { forms::orx(a, st); forms::addc(b, st); }
139 => { forms::orx(a, st); forms::xorc(b, st); }
140 => { forms::orx(a, st); forms::mulc(b, st); }
141 => { forms::orx(a, st); forms::mulc2(b, st); }
142 => { forms::orx(a, st); forms::andx(b, st); }
143 => { forms::orx(a, st); forms::orx(b, st); }
_ => unreachable!(),
}
}
for ins in it.remainder() {
run_instr(ins, st);
}
}
/// The scalar reference: one instruction on one 16-word state, the text the kernels carry (`emit.rs`,
/// `derive_instr_text`) restated in Rust. The tests pin the SoA interpreter against it.
pub fn run_round_scalar(prog: &[DInstr], s: &mut [u32; DERIVE_REGS]) {
for ins in prog {
let d = ins.dst as usize;
let c = s[ins.src as usize];
match ins.op {
DOp::Add => s[d] = s[d].wrapping_add(c),
DOp::Sub => s[d] = s[d].wrapping_sub(c),
DOp::Xor => s[d] ^= c,
DOp::Mul => s[d] = s[d].wrapping_mul(c | 1),
DOp::Rot => s[d] = rotl(s[d], ins.rot as u32).wrapping_add(c),
DOp::XRot => s[d] = rotl(s[d] ^ c, ins.rot as u32),
DOp::AddC => s[d] = s[d].wrapping_add(c.wrapping_add(ins.imm)),
DOp::XorC => s[d] ^= c ^ ins.imm,
DOp::MulC => s[d] = (s[d] ^ c).wrapping_mul(ins.imm),
DOp::MulC2 => s[d] = s[d].wrapping_mul(ins.imm).wrapping_add(c),
DOp::AndX => s[d] ^= c & s[ins.src2 as usize],
DOp::OrX => s[d] = s[d].wrapping_add(c | s[ins.src2 as usize]),
}
}
}
/// The source text of one instruction in the C-family dialects (the same text in Metal, CUDA C and OpenCL C:
/// `s` is the 16-word state, `mh_rotl` the rotate of the memhard core).
pub fn instr_text(ins: &DInstr) -> String {
let (d, c, b) = (ins.dst, ins.src, ins.src2);
match ins.op {
DOp::Add => format!("s[{d}] += s[{c}];"),
DOp::Sub => format!("s[{d}] -= s[{c}];"),
DOp::Xor => format!("s[{d}] ^= s[{c}];"),
DOp::Mul => format!("s[{d}] *= (s[{c}] | 1u);"),
DOp::Rot => format!("s[{d}] = mh_rotl(s[{d}], {}u) + s[{c}];", ins.rot),
DOp::XRot => format!("s[{d}] = mh_rotl(s[{d}] ^ s[{c}], {}u);", ins.rot),
DOp::AddC => format!("s[{d}] += s[{c}] + {:#010x}u;", ins.imm),
DOp::XorC => format!("s[{d}] ^= s[{c}] ^ {:#010x}u;", ins.imm),
DOp::MulC => format!("s[{d}] = (s[{d}] ^ s[{c}]) * {:#010x}u;", ins.imm),
DOp::MulC2 => format!("s[{d}] = s[{d}] * {:#010x}u + s[{c}];", ins.imm),
DOp::AndX => format!("s[{d}] ^= (s[{c}] & s[{b}]);"),
DOp::OrX => format!("s[{d}] += (s[{c}] | s[{b}]);"),
}
}
/// One instruction as a line of program.json: `"add d=3 c=0"`, `"mulc d=5 c=3 imm=0x..."`.
pub fn instr_line(ins: &DInstr) -> String {
let mut s = format!("{} d={} c={}", ins.op.name(), ins.dst, ins.src);
if ins.op.has_third() {
s.push_str(&format!(" b={}", ins.src2));
}
if ins.op.has_rot() {
s.push_str(&format!(" k={}", ins.rot));
}
if ins.op.has_imm() {
s.push_str(&format!(" imm={:#010x}", ins.imm));
}
s
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn weights_sum_and_ops() {
assert_eq!(DOP_WEIGHTS.iter().map(|(_, w)| w).sum::<u64>(), 100);
for (i, (op, _)) in DOP_WEIGHTS.iter().enumerate() {
assert_eq!(*op as usize, i);
assert_eq!(DOp::from_u8(i as u8), Some(*op));
}
assert_eq!(DOp::for_roll(0), DOp::Add);
assert_eq!(DOp::for_roll(13), DOp::Add);
assert_eq!(DOp::for_roll(14), DOp::Sub);
assert_eq!(DOp::for_roll(99), DOp::OrX);
// the expected chip operations per instruction, 1.52 (1.62 on a GPU), put 736 x 9 over the x8 floors
let mean: f64 = DOP_WEIGHTS.iter().map(|(o, w)| o.chip_ops() as f64 * *w as f64 / 100.0).sum();
assert!((mean - 1.52).abs() < 1e-9, "{mean}");
let gpu_mean: f64 = DOP_WEIGHTS.iter().map(|(o, w)| o.gpu_ops() as f64 * *w as f64 / 100.0).sum();
assert!((gpu_mean - 1.62).abs() < 1e-9, "{gpu_mean}");
assert!(9.0 * DERIVE_LEN_X8 as f64 * mean > OPS_FLOOR_X8 as f64);
assert!(9.0 * DERIVE_LEN_X8 as f64 * gpu_mean > GPU_OPS_FLOOR_X8 as f64);
assert_eq!(DeriveProgram::floors(DERIVE_LEN_X8), (9_216, 10_368, 1_152));
assert_eq!(DeriveProgram::floors(368), (4_608, 5_184, 576));
let mul_share: f64 = DOP_WEIGHTS.iter().filter(|(o, _)| o.is_mul()).map(|(_, w)| *w as f64 / 100.0).sum();
assert!(9.0 * DERIVE_LEN_X8 as f64 * mul_share > MULS_FLOOR_X8 as f64);
}
#[test]
fn draw_is_structural_and_accepted() {
let mut rng = SplitMix64::new(0x1234_5678_9abc_def0);
let p = DeriveProgram::draw(&mut rng, DERIVE_LEN_X8);
assert_eq!(p.attempt, 0, "the first candidate of this seed passes");
assert_eq!(p.rounds.len(), DERIVE_PROGRAMS);
assert_eq!(p.instr_count(), 9 * DERIVE_LEN_X8 as u64);
assert!(p.check().is_ok());
assert!(p.chip_ops() >= OPS_FLOOR_X8 && p.gpu_ops() >= GPU_OPS_FLOOR_X8 && p.muls() >= MULS_FLOOR_X8);
assert!(p.gpu_ops() > p.chip_ops());
// every instruction consumes the newest result
for prog in &p.rounds {
let mut chain = 0u8;
for ins in prog {
assert_eq!(ins.src, chain);
assert_ne!(ins.dst, chain);
chain = ins.dst;
}
}
// four draws per instruction: the same program again from the same seed, and a different one one draw on
let mut rng2 = SplitMix64::new(0x1234_5678_9abc_def0);
assert_eq!(DeriveProgram::draw(&mut rng2, DERIVE_LEN_X8), p);
let mut rng3 = SplitMix64::new(0x1234_5678_9abc_def0);
rng3.next();
assert_ne!(DeriveProgram::draw(&mut rng3, DERIVE_LEN_X8), p);
}
#[test]
fn soa_matches_scalar_and_is_a_bijection() {
let mut rng = SplitMix64::new(7);
let p = DeriveProgram::draw(&mut rng, 64);
let mut st: SoaState = [[0u32; SOA_LANES]; DERIVE_REGS];
let mut scalars = [[0u32; DERIVE_REGS]; SOA_LANES];
let mut x = SplitMix64::new(99);
for k in 0..SOA_LANES {
for r in 0..DERIVE_REGS {
let v = x.next() as u32;
st[r][k] = v;
scalars[k][r] = v;
}
}
let before = scalars;
for prog in &p.rounds {
run_round(prog, &mut st);
for k in 0..SOA_LANES {
run_round_scalar(prog, &mut scalars[k]);
}
}
for k in 0..SOA_LANES {
for r in 0..DERIVE_REGS {
assert_eq!(st[r][k], scalars[k][r], "lane {k} reg {r}");
}
}
// distinct inputs stay distinct (a bijection on the state, spot-checked: 32 lanes, no collision)
for a in 0..SOA_LANES {
for b in a + 1..SOA_LANES {
assert_ne!(scalars[a], scalars[b]);
assert_ne!(before[a], before[b]);
}
}
}
#[test]
fn acceptance_rejects_degenerate_draws() {
let mut rng = SplitMix64::new(3);
let mut p = DeriveProgram::draw(&mut rng, 64);
// a register never written: make every write of round 2 go to the chain's neighbour
let mut q = p.clone();
for ins in q.rounds[2].iter_mut() {
ins.dst = if ins.src == 1 { 2 } else { 1 };
}
let mut chain = 0u8;
for ins in q.rounds[2].iter_mut() {
ins.src = chain;
ins.dst = if chain == 1 { 2 } else { 1 };
chain = ins.dst;
}
assert!(matches!(q.check(), Err(DeriveReject::RegisterNeverWritten { round: 2, .. })));
// all rotations equal
let mut q = p.clone();
for ins in q.rounds.iter_mut().flatten() {
if ins.op.has_rot() {
ins.rot = 5;
}
}
assert!(matches!(q.check(), Err(DeriveReject::RotationsDegenerate { distinct: 1 })));
// every op an add apart from the xor-rotates (so the rotations stay distinct): under the ops floor
for ins in p.rounds.iter_mut().flatten() {
if ins.op != DOp::XRot {
ins.op = DOp::Add;
ins.rot = 0;
ins.imm = 0;
ins.src2 = 0;
}
}
assert!(matches!(p.check(), Err(DeriveReject::OpsUnderFloor { .. })), "{:?}", p.check());
// no multiplies at all but the ops floor met: under the multiply floor
let mut q = DeriveProgram::draw(&mut SplitMix64::new(11), 64);
for ins in q.rounds.iter_mut().flatten() {
if ins.op.is_mul() {
ins.op = DOp::AddC;
}
}
assert!(matches!(q.check(), Err(DeriveReject::MulsUnderFloor { .. })), "{:?}", q.check());
}
#[test]
fn text_forms() {
let i = DInstr { op: DOp::MulC, dst: 5, src: 3, src2: 0, rot: 0, imm: 0x9e37_79b9 };
assert_eq!(instr_text(&i), "s[5] = (s[5] ^ s[3]) * 0x9e3779b9u;");
assert_eq!(instr_line(&i), "mulc d=5 c=3 imm=0x9e3779b9");
let i = DInstr { op: DOp::XRot, dst: 0, src: 15, src2: 0, rot: 17, imm: 0 };
assert_eq!(instr_text(&i), "s[0] = mh_rotl(s[0] ^ s[15], 17u);");
let i = DInstr { op: DOp::AndX, dst: 2, src: 9, src2: 14, rot: 0, imm: 0 };
assert_eq!(instr_text(&i), "s[2] ^= (s[9] & s[14]);");
}
}
Reference in a new issue View git blame Copy permalink