GPU-Accelerated LLM Training in C and CUDA — SaaS, AI & Subscription Products · Tetrees EX

SaaS, AI & Subscription ProductsSinglePhiên bản 1.0.0

GPU-Accelerated LLM Training in C and CUDA

bởi Rowan E.

Về sản phẩm này

Train GPT-2 and GPT-3 class language models in pure C and CUDA with no PyTorch or Python runtime required. Designed for researchers and engineers who want fast, readable, low-dependency LLM pretraining from scratch.

Công nghệ & tương thích

PythonC

Phiên bản

v1.0.01 thg 3, 2025

Usage - llm.c: LLM Training in Pure C/CUDA

Overview

This block provides a pure C and CUDA implementation of GPT-2 and GPT-3 pretraining and fine-tuning, with no dependency on PyTorch or Python at runtime. It targets backend engineers and ML infrastructure teams who need a minimal, auditable, high-performance training loop they can embed in a C/CUDA build pipeline or invoke as a subprocess from a Node.js orchestration layer. The CPU path runs in ~1,000 lines of C; the CUDA path is faster than PyTorch Nightly on single-GPU benchmarks.

What you get

.github/ - CI workflow definitions for CPU, GPU, and test pipelines
dev/ - Developer utilities: data downloaders, CUDA benchmarks, evaluation scripts
doc/ - Documentation and worked examples (e.g. layernorm derivation in C/Python)
llmc/ - Core header-only C/CUDA library: dataloader, tokenizer, sampler, schedulers, logger, CUDA/cuDNN helpers
scripts/ - Shell scripts for launching single- and multi-node GPT-2/GPT-3 training runs
LICENSE - MIT license
README.md - Project overview, quick-start instructions, and architecture notes
profile_gpt2cu.py - Python profiling harness for the CUDA training binary
requirements.txt - Python dependencies for data preparation and evaluation scripts
test_gpt2.c - Unit test for the GPT-2 C implementation
train_gpt2.c - CPU fp32 reference training implementation (~1,000 lines)
train_gpt2.py - PyTorch reference implementation (nanoGPT-derived)
train_llama3.py - PyTorch reference implementation for LLaMA-3 variant

Required dependencies

This is a native C/CUDA project. There are no npm packages to install. The build toolchain requirements are:

# No npm install required - this is a native C/CUDA build
# Ensure the following native tools are available:

# Ubuntu / Debian
sudo apt-get install build-essential libgomp1 cuda-toolkit-12-x

# macOS (CPU only)
xcode-select --install

# Python deps for data prep and evaluation only
pip install -r source/requirements.txt

Native build steps required:

Build the CPU training binary: make -C source train_gpt2
Build the CUDA training binary: make -C source train_gpt2cu (requires CUDA toolkit ≥ 12.0)
Build the fp32 CUDA binary (simpler, educational):

make -C source train_gpt2fp32cu

Copy the source/ directory into your project root, e.g. vendor/llmc/.

Download starter data and model weights:

chmod u+x vendor/llmc/dev/download_starter_pack.sh
./vendor/llmc/dev/download_starter_pack.sh

Build the binary you need from vendor/llmc/:

cd vendor/llmc
make train_gpt2        # CPU fp32
make train_gpt2cu      # CUDA bf16/fp32 mixed
make train_gpt2fp32cu  # CUDA fp32 only (simpler)

Set environment variables for OpenMP thread count:
```
export OMP_NUM_THREADS=8
```
From your Node.js/TypeScript project, invoke the compiled binary as a child process. No tsconfig path aliases are needed since this is a native subprocess integration, not a JS module import.
Place any .bin data files (tokenized datasets, model checkpoints) in a directory accessible to the binary. Default paths expected by the binary are relative to its working directory.

import { spawn, ChildProcess } from "child_process";

function trainGpt2(options: {
  binaryPath: string;    // path to compiled train_gpt2 or train_gpt2cu
  dataPath: string;      // path to tokenized .bin dataset
  ompThreads?: number;   // OMP_NUM_THREADS, default 8
  extraArgs?: string[];  // additional CLI flags passed to binary
}): ChildProcess;

function evaluateGpt2(options: {
  binaryPath: string;
  checkpointPath: string;
  evalScript: string;    // path to dev/eval/run_eval.sh
}): Promise<{ hellaswag?: number; mmlu?: number }>;

function dataPrep(options: {
  pythonBin: string;     // e.g. "python3"
  script: string;        // e.g. "vendor/llmc/dev/data/tinyshakespeare.py"
  outputDir: string;
}): Promise<void>;

import { spawn } from "child_process";
import * as path from "path";

const binaryPath = path.resolve("vendor/llmc/train_gpt2");

const proc = spawn(binaryPath, [], {
  cwd: path.resolve("vendor/llmc"),
  env: { ...process.env, OMP_NUM_THREADS: "8" },
});

proc.stdout.on("data", (chunk: Buffer) => {
  const line = chunk.toString();
  // Parse loss lines, e.g.: "step 10: loss 3.821147 (took 1234.56 ms)"
  const match = line.match(/step (\d+): loss ([0-9.]+)/);
  if (match) {
    console.log(`Step ${match[1]}, Loss: ${match[2]}`);
  }
});

proc.stderr.on("data", (chunk: Buffer) => {
  console.error("stderr:", chunk.toString());
});

proc.on("exit", (code) => {
  console.log("Training exited with code", code);
});

import { execFile, ExecFileException } from "child_process";
import * as path from "path";
import * as util from "util";

const execFileAsync = util.promisify(execFile);

async function prepareData(): Promise<void> {
  const script = path.resolve("vendor/llmc/dev/data/tinyshakespeare.py");
  const { stdout, stderr } = await execFileAsync("python3", [script]);
  if (stdout) console.log(stdout);
  if (stderr) console.error(stderr);
  console.log("Data preparation complete. .bin files written.");
}

prepareData().catch(console.error);

import { exec } from "child_process";
import * as path from "path";
import * as util from "util";

const execAsync = util.promisify(exec);

async function runEval(checkpointDir: string): Promise<void> {
  const evalScript = path.resolve("vendor/llmc/dev/eval/run_eval.sh");
  const cmd = `bash ${evalScript} ${checkpointDir}`;

  const { stdout, stderr } = await execAsync(cmd, {
    cwd: path.resolve("vendor/llmc"),
  });

  // dev/eval/summarize_eval.py format: "hellaswag: 0.2941"
  const match = stdout.match(/hellaswag:\s*([0-9.]+)/i);
  if (match) {
    console.log("HellaSwag accuracy:", parseFloat(match[1]));
  } else {
    console.log("Raw eval output:", stdout);
  }
  if (stderr) console.error(stderr);
}

runEval("./checkpoints/gpt2_124M").catch(console.error);

I have purchased the "llm.c: LLM Training in Pure C/CUDA" source block.
The source is located at vendor/llmc/ in my project.
There is a USAGE.md at vendor/llmc/USAGE.md with full integration instructions.

My project is a Node.js/TypeScript backend service (Express, TypeScript 5, Node 20).

Please help me integrate llm.c step by step:

1. Read USAGE.md and the file listing under vendor/llmc/ to understand
   the structure. Do not invent any APIs or filenames not listed there.

2. Add a build step in my package.json scripts section that runs
   `make -C vendor/llmc train_gpt2` for CPU and optionally
   `make -C vendor/llmc train_gpt2cu` for CUDA.

3. Create a TypeScript module at src/llmc/runner.ts that exports functions
   to: (a) prepare data by spawning the relevant dev/data Python script,
   (b) launch a training run via the compiled binary with configurable
   OMP_NUM_THREADS and working directory, and (c) run post-training evaluation
   via dev/eval/run_eval.sh and return parsed accuracy metrics.

4. Wire error handling so non-zero exit codes from child processes throw
   typed errors in TypeScript.

5. Add a route in my Express app at POST /api/train that accepts a JSON body
   with { dataset, threads, cuda } and kicks off a training job, streaming
   stdout loss lines back to the client via Server-Sent Events.

Use only the real file paths and binary names from USAGE.md and the file listing.
The upstream project is llm.c (https://github.com/karpathy/llm.c).

GPU-Accelerated LLM Training in C and CUDA

Về sản phẩm này

Công nghệ & tương thích

Phiên bản

Usage - llm.c: LLM Training in Pure C/CUDA

Overview

What you get

Required dependencies

Chạy mã này

Báo cáo xác minh

Cài đặt & tích hợp

Tương thích với

Triển khai vào không gian làm việc

Sử dụng sản phẩm này qua MCP

Đánh giá

Discussion

Project setup

Public API

trainGpt2 (subprocess)

evaluateGpt2 (subprocess)

dataPrep (subprocess)

Working examples

Scenario - Launch CPU training from a Node.js job runner

Scenario - Prepare tinyshakespeare dataset before training

Scenario - Run post-training HellaSwag evaluation

File walkthrough

Common pitfalls and fixes

Vibe-coding prompt template

License and credit

Ax: TypeScript DSPy Framework for Production LLM Apps

Groq TypeScript SDK for Fast LLM APIs

LangChain.js TypeScript Framework for LLM Applications

SupremeMemory for LLM solutions (Sate of the Art memory Features)