GPU-Accelerated LLM Training in C and CUDA — SaaS, AI & Subscription Products · Tetrees EX

SaaS, AI & Subscription ProductsSingle版本 1.0.0

GPU-Accelerated LLM Training in C and CUDA

由 Rowan E. 出售

产品介绍

Train GPT-2 and GPT-3 class language models in pure C and CUDA with no PyTorch or Python runtime required. Designed for researchers and engineers who want fast, readable, low-dependency LLM pretraining from scratch.

技术栈与兼容性

PythonC

版本

v1.0.02025年3月1日

Usage - llm.c: LLM Training in Pure C/CUDA

Overview

This block provides a pure C and CUDA implementation of GPT-2 and GPT-3 pretraining and fine-tuning, with no dependency on PyTorch or Python at runtime. It targets backend engineers and ML infrastructure teams who need a minimal, auditable, high-performance training loop they can embed in a C/CUDA build pipeline or invoke as a subprocess from a Node.js orchestration layer. The CPU path runs in ~1,000 lines of C; the CUDA path is faster than PyTorch Nightly on single-GPU benchmarks.

What you get

.github/ - CI workflow definitions for CPU, GPU, and test pipelines
dev/ - Developer utilities: data downloaders, CUDA benchmarks, evaluation scripts
doc/ - Documentation and worked examples (e.g. layernorm derivation in C/Python)
llmc/ - Core header-only C/CUDA library: dataloader, tokenizer, sampler, schedulers, logger, CUDA/cuDNN helpers
scripts/ - Shell scripts for launching single- and multi-node GPT-2/GPT-3 training runs
LICENSE - MIT license
README.md - Project overview, quick-start instructions, and architecture notes
profile_gpt2cu.py - Python profiling harness for the CUDA training binary
requirements.txt - Python dependencies for data preparation and evaluation scripts
test_gpt2.c - Unit test for the GPT-2 C implementation
train_gpt2.c - CPU fp32 reference training implementation (~1,000 lines)
train_gpt2.py - PyTorch reference implementation (nanoGPT-derived)
train_llama3.py - PyTorch reference implementation for LLaMA-3 variant

Required dependencies

This is a native C/CUDA project. There are no npm packages to install. The build toolchain requirements are:

# No npm install required - this is a native C/CUDA build
# Ensure the following native tools are available:

# Ubuntu / Debian
sudo apt-get install build-essential libgomp1 cuda-toolkit-12-x

# macOS (CPU only)
xcode-select --install

# Python deps for data prep and evaluation only
pip install -r source/requirements.txt

Native build steps required:

Build the CPU training binary: make -C source train_gpt2
Build the CUDA training binary: make -C source train_gpt2cu (requires CUDA toolkit ≥ 12.0)
Build the fp32 CUDA binary (simpler, educational):

make -C source train_gpt2fp32cu

Copy the source/ directory into your project root, e.g. vendor/llmc/.

Download starter data and model weights:

chmod u+x vendor/llmc/dev/download_starter_pack.sh
./vendor/llmc/dev/download_starter_pack.sh

Build the binary you need from vendor/llmc/:

cd vendor/llmc
make train_gpt2        # CPU fp32
make train_gpt2cu      # CUDA bf16/fp32 mixed
make train_gpt2fp32cu  # CUDA fp32 only (simpler)

Set environment variables for OpenMP thread count:
```
export OMP_NUM_THREADS=8
```
From your Node.js/TypeScript project, invoke the compiled binary as a child process. No tsconfig path aliases are needed since this is a native subprocess integration, not a JS module import.
Place any .bin data files (tokenized datasets, model checkpoints) in a directory accessible to the binary. Default paths expected by the binary are relative to its working directory.

import { spawn, ChildProcess } from "child_process";

function trainGpt2(options: {
  binaryPath: string;    // path to compiled train_gpt2 or train_gpt2cu
  dataPath: string;      // path to tokenized .bin dataset
  ompThreads?: number;   // OMP_NUM_THREADS, default 8
  extraArgs?: string[];  // additional CLI flags passed to binary
}): ChildProcess;

function evaluateGpt2(options: {
  binaryPath: string;
  checkpointPath: string;
  evalScript: string;    // path to dev/eval/run_eval.sh
}): Promise<{ hellaswag?: number; mmlu?: number }>;

function dataPrep(options: {
  pythonBin: string;     // e.g. "python3"
  script: string;        // e.g. "vendor/llmc/dev/data/tinyshakespeare.py"
  outputDir: string;
}): Promise<void>;

import { spawn } from "child_process";
import * as path from "path";

const binaryPath = path.resolve("vendor/llmc/train_gpt2");

const proc = spawn(binaryPath, [], {
  cwd: path.resolve("vendor/llmc"),
  env: { ...process.env, OMP_NUM_THREADS: "8" },
});

proc.stdout.on("data", (chunk: Buffer) => {
  const line = chunk.toString();
  // Parse loss lines, e.g.: "step 10: loss 3.821147 (took 1234.56 ms)"
  const match = line.match(/step (\d+): loss ([0-9.]+)/);
  if (match) {
    console.log(`Step ${match[1]}, Loss: ${match[2]}`);
  }
});

proc.stderr.on("data", (chunk: Buffer) => {
  console.error("stderr:", chunk.toString());
});

proc.on("exit", (code) => {
  console.log("Training exited with code", code);
});

import { execFile, ExecFileException } from "child_process";
import * as path from "path";
import * as util from "util";

const execFileAsync = util.promisify(execFile);

async function prepareData(): Promise<void> {
  const script = path.resolve("vendor/llmc/dev/data/tinyshakespeare.py");
  const { stdout, stderr } = await execFileAsync("python3", [script]);
  if (stdout) console.log(stdout);
  if (stderr) console.error(stderr);
  console.log("Data preparation complete. .bin files written.");
}

prepareData().catch(console.error);

import { exec } from "child_process";
import * as path from "path";
import * as util from "util";

const execAsync = util.promisify(exec);

async function runEval(checkpointDir: string): Promise<void> {
  const evalScript = path.resolve("vendor/llmc/dev/eval/run_eval.sh");
  const cmd = `bash ${evalScript} ${checkpointDir}`;

  const { stdout, stderr } = await execAsync(cmd, {
    cwd: path.resolve("vendor/llmc"),
  });

  // dev/eval/summarize_eval.py format: "hellaswag: 0.2941"
  const match = stdout.match(/hellaswag:\s*([0-9.]+)/i);
  if (match) {
    console.log("HellaSwag accuracy:", parseFloat(match[1]));
  } else {
    console.log("Raw eval output:", stdout);
  }
  if (stderr) console.error(stderr);
}

runEval("./checkpoints/gpt2_124M").catch(console.error);

I have purchased the "llm.c: LLM Training in Pure C/CUDA" source block.
The source is located at vendor/llmc/ in my project.
There is a USAGE.md at vendor/llmc/USAGE.md with full integration instructions.

My project is a Node.js/TypeScript backend service (Express, TypeScript 5, Node 20).

Please help me integrate llm.c step by step:

1. Read USAGE.md and the file listing under vendor/llmc/ to understand
   the structure. Do not invent any APIs or filenames not listed there.

2. Add a build step in my package.json scripts section that runs
   `make -C vendor/llmc train_gpt2` for CPU and optionally
   `make -C vendor/llmc train_gpt2cu` for CUDA.

3. Create a TypeScript module at src/llmc/runner.ts that exports functions
   to: (a) prepare data by spawning the relevant dev/data Python script,
   (b) launch a training run via the compiled binary with configurable
   OMP_NUM_THREADS and working directory, and (c) run post-training evaluation
   via dev/eval/run_eval.sh and return parsed accuracy metrics.

4. Wire error handling so non-zero exit codes from child processes throw
   typed errors in TypeScript.

5. Add a route in my Express app at POST /api/train that accepts a JSON body
   with { dataset, threads, cuda } and kicks off a training job, streaming
   stdout loss lines back to the client via Server-Sent Events.

Use only the real file paths and binary names from USAGE.md and the file listing.
The upstream project is llm.c (https://github.com/karpathy/llm.c).

GPU-Accelerated LLM Training in C and CUDA

产品介绍

技术栈与兼容性

版本

Usage - llm.c: LLM Training in Pure C/CUDA

Overview

What you get

Required dependencies

运行此代码

验证报告

安装与集成

支持工具

部署到你的工作区

通过 MCP 使用此产品

评价

Discussion

Project setup

Public API

trainGpt2 (subprocess)

evaluateGpt2 (subprocess)

dataPrep (subprocess)

Working examples

Scenario - Launch CPU training from a Node.js job runner

Scenario - Prepare tinyshakespeare dataset before training

Scenario - Run post-training HellaSwag evaluation

File walkthrough

Common pitfalls and fixes

Vibe-coding prompt template

License and credit

Ax: TypeScript DSPy Framework for Production LLM Apps

Groq TypeScript SDK for Fast LLM APIs

LangChain.js TypeScript Framework for LLM Applications

SupremeMemory for LLM solutions (Sate of the Art memory Features)