Production LLMs

LLMs from research to product:

Training stages:

1. Pretraining: massive corpus, next-token prediction. Foundation model.
2. Supervised fine-tuning (SFT): instruction-following examples.
3. RLHF / DPO: align with human preferences.
4. Constitutional AI: train to refuse harmful outputs.

Open-source LLMs:

• Llama 2/3 (Meta): 7B, 13B, 70B, 405B params.
• Mistral: 7B, Mixtral 8x7B.
• Falcon: TII UAE.
• Qwen: Alibaba.
• DeepSeek: efficient training pioneer.
• Yi: 01.AI.

Closed:

• GPT-4 / 4o: OpenAI.
• Claude 3 / 4: Anthropic.
• Gemini: Google.
• Grok: xAI.

Fine-tuning:

• Full fine-tune: update all params. Expensive.
• LoRA: low-rank adapters. ~1% of params trained.
• QLoRA: quantize base model + LoRA. Run on consumer GPUs.

Quantization:

• FP32 → FP16 → INT8 → INT4.
• 4-bit quantization: 4x memory savings, minor quality loss.
• bitsandbytes, GPTQ, AWQ, GGUF.

Serving:

• vLLM: high-throughput serving with paged attention.
• TGI: Hugging Face Text Generation Inference.
• TensorRT-LLM: NVIDIA optimized.
• llama.cpp: CPU + GPU, GGUF format.
• MLX: Apple Silicon.

Latency vs throughput:

• Latency: time to first token.
• Throughput: tokens/sec across all users.
• Trade-offs: bigger batches → higher throughput but worse latency.

Costs (approximate):

• GPT-4 API: ~$10-30 per million tokens.
• Self-hosted Llama 70B: ~$1-2 per million tokens (compute amortized).
• 7B model on GPU: ~$0.10/M tokens.

RAG (Retrieval-Augmented Generation):

• Index docs as embeddings.
• Retrieve relevant chunks.
• Inject into prompt.
• LLM grounds in retrieved data.
• Reduces hallucination.

Tool use / function calling:

• LLM emits JSON: {"function": "search", "args": {...}}.
• Code executes function, returns result.
• LLM continues with result in context.
• Foundation of "agents".

Safety:

• Prompt injection: user prompts override system prompts.
• Jailbreaks: rephrased to bypass safety training.
• PII leakage: model may emit training data.
• Bias: outputs may reflect dataset biases.
• Modern: constitutional AI, content filters, monitoring.

Don't roll your own LLM. Pretrain costs millions. Fine-tune Llama or use API.