Llama

“I built thaw because forking an LLM agent is absurdly wasteful today. When an agent explores N branches — RL rollouts, best-of-N, parallel coding attempts — each branch re-runs prefill over the same shared context. You pay for the same prompt N times.thaw snapshots a live inferen...”

“I've been running Qwen3.6-35B-A3B locally on llama.cpp and noticed that prompt processing throughput gets too low with MTP. I got nerd-sniped.I'm not a C++ dev, I know almost nothing about ML, and I'm only scratching the surface of how LLMs work. What started as curiosity turned ...”

“I built minLlama because I wanted a Llama implementation that was easy to understand and hack for KV cache compression research. There is also a PyTorch and Jax version in ~140 lines.Would be interested in feedback from people who have written transformer implementations before, ...”

“Hi HN,I wanted to share EzziwebTools, a hub of 30+ free tools. The site is built with Next.js and React, and includes both standard utilities and a few AI-powered tools (using Groq/Llama 3), such as:An API Response Time Comparator (visualizes TTFB vs Download time) An AI-powered ...”

“See my comment to parent. I've been using local LLMs for practical, personal tasks for a few months now very successfuly.You can run fantastic local models if you have either:- M-series Apple device with ideally >= 24GB of VRAM- RTX [345]090 GPUI'm fortunate enough to have both a...”