Skip to content

mohitmishra786/low-level-dev-skills / skills/compiler-internals/jit-compilation

jit-compilation

Verified

JIT compilation skill for runtime code generation. Use when building LLVM ORC JIT, LLJIT, Cranelift JIT, inline caches, trampolines, or Rust dynasm codegen. Activates on queries about ORC JIT, LLJIT, Cranelift, ExecutionSession, inline cache, W^X, or dynasm.

JIT Compilation

Purpose

Guide agents through just-in-time compilation: LLVM ORC JIT v2 (ExecutionSession, IRLayer, ObjectLayer), LLJIT for simpler use cases, Cranelift JIT, inline caches for dynamic dispatch, trampolines for lazy compilation, security considerations (W^X, code signing), and Rust dynasm for x86 codegen.

When to Use

  • Building an interpreter with a JIT hot-path
  • Implementing lazy function compilation on first call
  • Embedding dynamic code generation in a REPL or game engine
  • Prototyping codegen without writing a full linker
  • Creating inline caches for polymorphic call sites
  • Generating x86 machine code from Rust with dynasm

Workflow

1. JIT architecture overview

Source/AST/Bytecode
    → IR (LLVM IR or Cranelift CLIF)
    → Object file (in memory)
    → Runtime linker (RTDyldObjectLinkingLayer)
    → Executable code in R+X memory
    → Function pointer call

2. LLJIT — simplest LLVM JIT

#include "llvm/ExecutionEngine/Orc/LLJIT.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/IRBuilder.h"

using namespace llvm;
using namespace llvm::orc;

int main() {
    auto JIT = cantFail(LLJITBuilder().create());

    LLVMContext Context;
    auto M = std::make_unique<Module>("jit", Context);
    IRBuilder<> Builder(Context);

    // int add(int a, int b) { return a + b; }
    Function *AddFn = Function::Create(
        FunctionType::get(Builder.getInt32Ty(),
                          {Builder.getInt32Ty(), Builder.getInt32Ty()}, false),
        Function::ExternalLinkage, "add", M.get());

    BasicBlock *BB = BasicBlock::Create(Context, "entry", AddFn);
    Builder.SetInsertPoint(BB);
    auto Args = AddFn->arg_begin();
    Value *Sum = Builder.CreateAdd(Args, Args + 1);
    Builder.CreateRet(Sum);

    cantFail(JIT->addIRModule(ThreadSafeModule(std::move(M), std::make_unique<LLVMContext>())));

    auto AddSym = JIT->lookup("add");
    auto *AddPtr = (int (*)(int, int))AddSym->getValue();
    int result = AddPtr(3, 4);  // 7
    return 0;
}
clang++ -std=c++17 jit.cpp $(llvm-config --cxxflags --ldflags --libs core orcjit native) -o jit

3. ORC JIT v2 layers

ExecutionSession ES;
auto &MainJD = ES.createBareJITDylib("main");

RTDyldObjectLinkingLayer ObjectLayer(
    ES, []() { return std::make_unique<SectionMemoryManager>(); });

IRCompileLayer CompileLayer(
    ES, ObjectLayer, std::make_unique<TargetMachineBuilder>());

// Add IR module to JITDylib
ThreadSafeModule TSM = ...;
CompileLayer.add(MainJD, std::move(TSM));

// Resolve symbol
auto Sym = ES.lookup({&MainJD}, "my_func");

Layers:

  • IRLayer — compiles LLVM IR to object
  • ObjectLayer — links relocatable objects
  • ExecutionSession — symbol lookup and JITDylib management

4. Lazy compilation with trampolines

First call → trampoline → compile function → patch trampoline → direct call
// Simplified lazy compile on first invocation
void *LazyCompile(const std::string &Name) {
    if (!Compiled.count(Name)) {
        auto Fn = CompileFromAST(Name);
        Compiled[Name] = Fn;
        // Patch call site or update function pointer table
    }
    return Compiled[Name];
}

ORC supports lazy reexports and lazy compilation via LazyCallThroughManager.

5. Inline caches

// Concept: monomorphic call site caches resolved target
// Pseudocode for dynamic language
function call_site(obj, method, args) {
    if (obj.class_id === cache.class_id) {
        return cache.fn_ptr(args);  // fast path
    }
    cache.class_id = obj.class_id;
    cache.fn_ptr = resolve_method(obj, method);
    return cache.fn_ptr(args);
}

JIT generates specialized code per cached type; deoptimize on cache miss.

6. Cranelift JIT

use cranelift::prelude::*;
use cranelift_jit::{JITBuilder, JITModule};
use cranelift_module::{Linkage, Module};

let isa = cranelift_native::builder().finish(settings::Flags::new(settings::builder()))?;
let jit_builder = JITBuilder::with_isa(isa, cranelift_module::default_libcall_names());
let mut module = JITModule::new(jit_builder);

let mut ctx = module.make_context();
ctx.func = Function::with_name_signature(
    module.declare_function("add", Linkage::Export, &sig)?,
    sig,
);
// ... build IR in ctx.func ...
module.define_function(func_id, &mut ctx)?;
module.finalize_definitions()?;
let code = module.get_finalized_function(func_id);
let add_fn: fn(i32, i32) -> i32 = unsafe { std::mem::transmute(code) };

Cranelift: faster compile times than LLVM, good for embeddable JITs.

7. Security — W^X policy

W^X (Write XOR Execute)
├── Memory page is writable OR executable, never both
├── JIT: allocate RW → write code → mprotect(RX)
└── macOS hardened runtime requires signed JIT pages
#include <sys/mman.h>

void *mem = mmap(NULL, size, PROT_READ | PROT_WRITE,
                 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
// write machine code to mem
mprotect(mem, size, PROT_READ | PROT_EXEC);

Linux: MAP_JIT on Apple platforms; sealed memfd on hardened systems.

8. Rust dynasm (x86)

use dynasm::dynasm;
use dynasmrt::{Assembler, ExecutableBuffer};

let mut asm = Assembler::new().unwrap();
dynasm!(asm
    ; .arch x86_64
    ; add:
    ; add eax, ecx
    ; ret
);
let buf = asm.finalize().unwrap();
let add_fn: fn(i32, i32) -> i32 = unsafe { std::mem::transmute(buf.ptr(0)) };

Use for lightweight asm snippets without LLVM dependency.

9. Decision tree

JIT backend choice?
├── Need LLVM optimizations → ORC JIT / LLJIT
├── Fast compile, embeddable → Cranelift
├── Tiny asm snippets → dynasm
└── ML workloads → MLIR → ORC pipeline

Common Problems

SymptomCauseFix
Symbol not found on lookupName mangling or not exportedUse C linkage; LLVMExternalLinkage
Segfault calling JIT codeABI mismatchMatch calling convention and types
W^X mmap failedSELinux/grsecurityUse MAP_JIT; check dmesg
Stale code after recompileOld function pointerInvalidate caches; use trampolines
LLVM JIT slow compile-O2 in JITUse -O0 for JIT; optimize hot paths only
Cranelift verify errorInvalid CLIFEnable cranelift_codegen::verify_function

Related Skills

  • skills/compiler-internals/llvm-passes — optimize before JIT
  • skills/compiler-internals/compiler-frontend — AST to IR for JIT input
  • skills/compiler-internals/mlir — MLIR lowering to LLVM for JIT
  • skills/compilers/llvm — LLVM IR fundamentals
  • skills/low-level-programming/interpreters — bytecode interpreters using JIT
  • skills/low-level-programming/assembly-x86 — hand-written asm context
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 319kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 319kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 319kUpdated Claude CodeCodexVerifiedInstall