YA Yohannes Azeze

Writing Outline

Tutorial - GenAI and Retrieval Systems

Using Open-Weights LLMs Locally and Effectively

A practical guide to running open-weights models locally with realistic expectations around hardware, quantization, latency, privacy, and quality.

Draft Outline

  • When local models make sense: privacy, cost control, offline work, and experimentation
  • Model selection tradeoffs: size, license, context length, reasoning quality, and domain fit
  • Quantization, memory, batching, and latency basics
  • Evaluation workflow before trusting a local model in a real application