OpenAI’s Jalapeño chip challenges GPU inference
Summary
The dominant shift is from treating AI as a single-model problem to designing complete operating systems around inference and agents: chips are being specialized around prefill, decode, memory locality, and network topology, while software teams build harnesses around context and verification. Yet the operational lesson is consistent across hardware, evaluation, and coding practice: speed and autonomy only matter when bounded by measurable end-to-end outcomes, reproducible checks, and retained human accountability.
OpenAI Jalapeno Custom AI ASIC at Hot Chips 2026
OpenAI presented Jalapeño as a full inference platform, pairing a 700W ASIC with host and rack design rather than optimizing a chip in isolation. On the public InferenceX benchmark, it reports 1.9× higher peak mixed-token throughput per kilowatt and 1.7× lower end-to-end latency than GB200 on GPT-OSS 120B; on DeepSeek R1, the matched-point claims are 1.7× and 3.6× respectively. The architecture keeps KV state local and varies active compute, memory, and network resources by inference phase, aiming to avoid the data movement and idle-power costs of specialized fleets. OpenAI says AI-assisted design cut the RTL-to-tapeout path to roughly nine months and plans deployment in its infrastructure by year-end.
Read the source →Google’s TPUv8s for Training and Inference at Hot Chips 2026
Google is splitting TPUv8 into separate 8t training and 8i inference chips, arguing that MoE traffic and agentic long contexts make a single compromise design inefficient. The inference-oriented 8i carries more HBM and SRAM per unit of compute and uses a BoardFly network with at most seven hops, versus 16 for the prior torus topology. The 8t training superpod is specified at 9,600 chips, 2PB of shared HBM, 121 EFLOPS of FP4 compute, and roughly twice TPUv7 Ironwood’s performance per watt. It also adopts a dedicated Virgo network intended to span 134,000 TPUs at 47 Pbit/s and uses optical switching to reshape slices and work around failed chips.
Read the source →SambaNova’s SN50 RDU for AI at Hot Chips 2026
SambaNova argues that agentic inference is chiefly a decode and bandwidth problem: it says decode consumes 97% of DeepSeek V3 runtime, while GPU model-bandwidth utilization falls as clusters grow. Its SN50 RDU uses a software-managed dataflow design with on-chip SRAM, claims five times SN40’s FLOPS, and scales beyond 256 chips through 800GbE scale-up and 400GbE scale-out networking. The company reports model bandwidth utilization of 45% at 256 SN50s and more than 350 TB/s aggregate model bandwidth at 512 RDUs. Its approach is explicitly heterogeneous: pair SN50s for decode with NVIDIA H200s for prefill and move work over RoCE.
Read the source →Microsoft’s Maia 200 AI Accelerator at Hot Chips 2026
Microsoft’s second-generation Maia 200 is a 3nm, 140-billion-transistor accelerator with six HBM3e stacks, 7 TB/s of HBM bandwidth, 10,000 TFLOPS FP4 peak, and a 750W TDP. Its Software Defined Local Access architecture makes dataflow compile-time explicit while keeping data local to tiles, reducing cross-talk and fabric pressure. Microsoft combines this with an all-Ethernet scale-up network that it says can link 6,000 chips across 128 racks, rather than using a separate scale-out fabric. The tradeoff is a hardware-specific programming model: kernels must be tuned through MCCL and the architecture’s explicit data movement to reach its intended efficiency.
Read the source →Cerebras Talks Going Rack-Scale with Their WSEs at Hot Chips 2026
Cerebras’s CS-4 is its first dedicated rack-scale system, combining three higher-clocked WSE-3 Turbo wafers in one scale-up domain. The company claims twice the token rate and 10× tokens per watt versus CS-3, driven by on-chip SRAM bandwidth quoted at 43,000 TB/s and direct wafer links with as little as two microseconds of latency. Its Nexus rack redesign brings power, liquid cooling, I/O, metering, and leak detection into pluggable “backpacks,” while using 50% fewer components than CS-3. Cerebras expects the platform to support mixed systems with AMD MI455X GPUs and to underpin CS-5 and CS-6, the latter adding stacked DRAM to trade some SRAM for more compute.
Read the source →NVIDIA’s Groq 3 LPU Accelerators for Heterogeneous AI Compute at Hot Chips 2026
NVIDIA described Groq-derived LPUs as a decode-focused complement to Vera Rubin GPUs, which remain responsible for prefill and attention in a disaggregated inference flow. A 256-LPU LPX rack is specified at 128GB of SRAM, 40 PB/s aggregate SRAM bandwidth, 315 PFLOPS FP8, and 11,000 decoded tokens per second on Gemma 4 31B; NVIDIA cites a third-party benchmark showing four times the next public competitor’s output rate. The deterministic LPU design puts instruction and network scheduling in software, which NVIDIA says also enables power smoothing and thermal-aware placement. The stated tradeoff is efficiency: increasing LPU offload can improve high-interactivity performance by up to 5×, but reduces total-throughput efficiency relative to GPU-only operation.
Read the source →Claude for Word: Turn a draft into a finished document
Claude for Word adds a Claude panel inside Word on the web, Windows, and Mac, where it can read document text, comments, and linked material such as a Box file. The demonstration has Claude consolidate reviewer feedback, identify conflicting asks, fact-check performance claims against a source document, restructure content, and propose all edits as review cards or tracked changes. Sensitive or irreversible actions, including changing editing mode and accepting revisions, require confirmation; a user can inspect the reasoning behind suggestions and accept or amend them selectively. It also supports reusable slash-command skills such as a copy-edit or company brand-guidelines check, and is included with paid Claude plans through Microsoft AppSource.
Read the source →BMAD's Founder on the Future of AI Coding (And the Slop Apocalypse)
The discussion warns that an agentic “slop apocalypse” may show up not only as broken code but as rising token spend, repeated cycles, and slower delivery. BMAD’s founder argues for moving from human-in-the-loop toward human-on-the-loop work: agents should guide and execute, while people remain in control of the direction and decisions. The practical implication is to treat agent productivity as a systems problem—maintain oversight and a clear workflow instead of maximizing unattended generation. The transcript frames the conversation as a workshop on BMAD, coding-agent practices, and where engineering work is heading.
Read the source →Apple introduces new Mac Studio with M5 Max and M5 Ultra - up to 512GB of unified memory
The extracted Reddit entry contains only the submitter attribution and link/comments markers, with no discussion or substantive details to summarize. The linked item’s title identifies Apple’s new Mac Studio hardware and a claimed 512GB unified-memory ceiling. No additional claims from the Reddit post are included here.
Read the source →Funding better evaluations of AI’s impact on wellbeing
Anthropic is launching a $5 million program for independent, open-source evaluations of AI’s effects on user wellbeing, offering funding, model access, and technical support. It argues that wellbeing cannot be judged reliably from isolated answers because risk often emerges across a long conversation and depends on user history—for example, diet advice may be harmful in the context of disordered eating. The initiative seeks work from clinicians, psychologists, and methodologists to create benchmarks that can be used across the industry, while grantees retain independence and publish their work openly. Applications are due September 21, with invitations for full proposals scheduled by October 5.
Read the source →GitHub Copilot app Customize tab is generally available
GitHub has made a Customize tab generally available in the Copilot app, centralizing MCP servers, plugins, skills, and canvases. A Featured view is intended to help teams discover additions when they do not already know which extension type they need, while dedicated sections support browsing by type. GitHub positions canvases as a way to keep context visible while acting on work such as issue triage, backlog prioritization, follow-ups, investigation, implementation, and review preparation. The update is a discoverability and workflow-composition layer rather than a new underlying model capability.
Read the source →How to evaluate LLMs before production
GitHub’s central recommendation is to start evaluations from the product decision, not a generic question of whether an LLM is accurate. In its secret-scanning example, false-positive reduction and precision are the primary user outcomes, while recall is a non-negotiable safety guardrail and latency, cost, reliability, and integration are deployment constraints. Teams should rerun versioned, end-to-end offline evaluations after meaningful changes to prompts, models, context construction, or surrounding logic, changing one major variable at a time. Stronger models may make simpler prompts viable, but model upgrades still need the same checks because quality gains can conceal regressions, cost, latency, or format changes.
Read the source →5 ways to upgrade your home decor with Google Search
Google promotes Search features for turning décor inspiration into purchases and DIY work, noting that searches for “home decor inspo” rose 300% in the preceding month. AI Mode can take a room photo and dimensions to suggest and mock up furniture; Lens and Circle to Search can identify a seen item or locate similar products. Search Live offers voice-and-video guidance for installation tasks, while product listings expose cross-store prices, price history, and alerts. The examples are consumer shopping aids rather than a claim that the tools can guarantee fit, quality, or availability.
Read the source →v2.1.246
Claude Code v2.1.246 adds a warning for overly broad Bash allow rules, an Auto mode classifier-rule editor under /permissions, and completion timestamps in end-of-turn output. It fixes numerous reliability and usability defects, including slow or blank transcripts, background-session startup failures, MCP interruption and argument-type errors, plugin installation and discovery issues, and session-resume failures with third-party API proxies. Security-relevant fixes include always requiring approval for malformed Bash commands and preventing a third-party gateway key from being sent to Anthropic telemetry or metrics endpoints. The release also improves /cd so new project settings, MCP servers, skills, hooks, and agents take effect immediately, and marks max-turn subagent results as partial instead of silently complete.
Read the source →Granite 4.2 LLMs: How They're Built
IBM released Apache-2.0 Granite 4.2 dense reasoning models in 3B, 8B, and 30B sizes, each with thinking, low-effort thinking, non-thinking modes, native tool calling, and a 512K context window. The family is trained from scratch on roughly 15T tokens, then supervised on about 7.2 million examples and post-trained through staged reinforcement learning; the 8B and 30B variants additionally receive agentic RL in sandboxed software-engineering, terminal, and search tasks. The SFT mixture is 31.6% agentic data, predominantly software engineering, and IBM filters it with LLM judges, heuristics, and global deduplication. The implementation is designed to work with OpenAI-compatible function calling through vLLM and is also supported in SGLang.
Read the source →AI At Home Part 2: Multi GPU Drifting
This practical account explains why local LLM generation is normally constrained by memory bandwidth rather than raw math: each next token requires reading model weights and the existing context. Mixture-of-experts models reduce activated weights per token but still require the full model to reside in fast memory because routing varies token by token. For a four-GPU home server with 32GB per GPU, straightforward layer parallelism fits a larger model but processes layers serially and is bounded roughly by one GPU’s bandwidth minus inter-GPU transfer overhead. The article sets up tensor parallelism as an alternative that divides each layer across GPUs, trading synchronization and interconnect behavior against the chance to use their bandwidth concurrently.
Read the source →A Manifesto for Responsible Agentic Coding
The manifesto argues that cheap code generation makes established engineering disciplines more important, not less: every production line should be read, understood, debugged, and owned by a human. It recommends using LLMs for prototypes, repetitive refactors, tests, and technical-debt work, while keeping iterations and pull requests small enough for meaningful human review. Passing CI is not treated as sufficient evidence for an overnight agent change, because review also preserves shared architectural understanding and limits cognitive debt. It additionally calls for transparency about AI use, no agent access to PII, credentials, trade secrets, or production systems, and human accountability for consequential decisions.
Read the source →Apple's new desktop computers are designed specifically for local AI development
Apple refreshed the Mac mini and Mac Studio around local inference, pairing a new M6 mini with a M5 Ultra Studio that can reach 512GB unified memory and 1.2TB/s memory bandwidth. Apple says macOS 26.2 enabled low-latency Thunderbolt 5 communication for distributed MLX inference, which has led users to chain machines to run models too large for one mainstream device. The M6 starts at $899 with 16GB, while an M5 Ultra Studio starts at $5,499; the 512GB configuration is expected in late October. The article’s key caveat is that this is primarily a specifications refresh and that local large-model workflows still need substantially more memory than an ordinary developer laptop provides.
Read the source →Super-intelligence or Superstition? Exploring Psychological Factors Influencing Belief in AI Predictions about Personal Behavior
The extracted content contains only arXivLabs boilerplate and no paper abstract, methods, results, or conclusions. The title indicates a study of psychological factors associated with believing AI predictions about personal behavior. A substantive summary would require the paper text, which is not present in the supplied material.
Read the source →Why Ramp built its own in-house coding agent, Inspect
Ramp built Inspect as a remote, sandboxed coding-agent platform after finding third-party tools too limited for high parallelism, internal integrations, and frontend verification. The system gives agents access to organization-specific APIs and MCP context, can run tests, inspect telemetry and feature flags, and visually verify frontend work with screenshots and live previews. Since its November 2025 v2 release, Inspect has reached one million sessions; Ramp says it now authors 75% of merged PRs and spins up a fully provisioned environment in under five seconds. The build-versus-buy lesson is conditional: a small 5.5-person team can justify a custom harness when proprietary context, centralized environments, and closed-loop verification are the differentiators.
Read the source →The full stack behind abundant intelligence
The extracted item provides only a one-sentence description: OpenAI CFO Sarah Friar argues that improvements in chips, compute, models, and products compound to make useful intelligence cheaper and more scalable. It frames the company’s strategy as full-stack coordination rather than a model-only advance. No further evidence, figures, or argument detail is present in the supplied content.
Read the source →Jalapeño’s first results show industry-leading speed and efficiency in AI inference
OpenAI says its first custom inference platform, Jalapeño, delivers 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T on the public InferenceX benchmark. For highly interactive workloads it reports 2.1–4.1× higher performance, while sustained measured power was at or below 550W despite a 700W rating. The core design claim is that keeping KV cache local and co-designing chip, memory, networking, and serving software handles both compute-heavy prefill and bandwidth-heavy decode without transferring state among specialized systems. OpenAI plans deployment by year-end while continuing to use NVIDIA and other external accelerators, and says later Jalapeño generations are already in development.
Read the source →Introducing the Admin plugin for ChatGPT Work and Codex
OpenAI introduced an Admin plugin for ChatGPT Work and Codex. It is intended to let workspace administrators analyze usage, manage members and permissions, adjust limits, and handle other administrative requests. The supplied content does not specify rollout timing, access controls, or exact supported commands.
Read the source →EVE Online: The Move to Python 3 Begins!
EVE Online is beginning a migration from Stackless Python 2.7, its last major runtime upgrade in 2010, after running on Stackless Python since the game launched in 2003. The plan starts with futurize across 2.4 million lines of code and then manually reviews roughly 20,000 sites where Python 2 and 3 semantics differ, such as integer division. The linked note says the announcement does not yet explain how Stackless will be replaced. It points to EVE Frontier’s Carbon engine, which has already replaced Stackless with the now-open-source carbonengine/scheduler library, as relevant precedent.
Read the source →