← All digests

🖥️ Hardware & Infra

Chips, GPUs, datacenters, and compute buildouts.

Subscribe to Hardware & Infra · RSS
📝 Article ServeTheHome

Apple Mac Studio M5 Ultra and Mac Mini M6 Launched

Apple’s new Mac mini uses the company’s first 2nm desktop chip, M6, with a 12-core CPU, 12-core GPU, dual 16-core Neural Engine, up to 32GB unified memory, and 170GB/s bandwidth; Apple claims 1.2× multithreaded CPU performance and nearly 30% more peak GPU AI compute than M5. The new Mac Studio centers on a quad-die M5 Ultra that combines two dual-die M5 Max chips via an inter-die fabric rated above 4.4TB/s, scaling to a 36-core CPU, 80-core GPU, 32-core Neural Engine, up to 512GB unified memory, and 1.2TB/s memory bandwidth. That capacity makes it a desk-side option for models that cannot fit on common discrete GPUs, though Apple’s performance claims are from preproduction hardware and the author expects bandwidth to be decisive for local inference speed. Price is the tradeoff: a 256GB M5 Ultra is already over $11,000, while the 512GB version is due in October; the article contrasts it with roughly $100,000 NVIDIA GB300 systems that offer substantially more bandwidth and memory.

In: OpenAI says research agents breached its safeguards
📝 Article ServeTheHome

OpenAI Jalapeno Custom AI ASIC at Hot Chips 2026

OpenAI presented Jalapeño as a full inference platform, pairing a 700W ASIC with host and rack design rather than optimizing a chip in isolation. On the public InferenceX benchmark, it reports 1.9× higher peak mixed-token throughput per kilowatt and 1.7× lower end-to-end latency than GB200 on GPT-OSS 120B; on DeepSeek R1, the matched-point claims are 1.7× and 3.6× respectively. The architecture keeps KV state local and varies active compute, memory, and network resources by inference phase, aiming to avoid the data movement and idle-power costs of specialized fleets. OpenAI says AI-assisted design cut the RTL-to-tapeout path to roughly nine months and plans deployment in its infrastructure by year-end.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article ServeTheHome

Google’s TPUv8s for Training and Inference at Hot Chips 2026

Google is splitting TPUv8 into separate 8t training and 8i inference chips, arguing that MoE traffic and agentic long contexts make a single compromise design inefficient. The inference-oriented 8i carries more HBM and SRAM per unit of compute and uses a BoardFly network with at most seven hops, versus 16 for the prior torus topology. The 8t training superpod is specified at 9,600 chips, 2PB of shared HBM, 121 EFLOPS of FP4 compute, and roughly twice TPUv7 Ironwood’s performance per watt. It also adopts a dedicated Virgo network intended to span 134,000 TPUs at 47 Pbit/s and uses optical switching to reshape slices and work around failed chips.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article ServeTheHome

SambaNova’s SN50 RDU for AI at Hot Chips 2026

SambaNova argues that agentic inference is chiefly a decode and bandwidth problem: it says decode consumes 97% of DeepSeek V3 runtime, while GPU model-bandwidth utilization falls as clusters grow. Its SN50 RDU uses a software-managed dataflow design with on-chip SRAM, claims five times SN40’s FLOPS, and scales beyond 256 chips through 800GbE scale-up and 400GbE scale-out networking. The company reports model bandwidth utilization of 45% at 256 SN50s and more than 350 TB/s aggregate model bandwidth at 512 RDUs. Its approach is explicitly heterogeneous: pair SN50s for decode with NVIDIA H200s for prefill and move work over RoCE.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article ServeTheHome

Microsoft’s Maia 200 AI Accelerator at Hot Chips 2026

Microsoft’s second-generation Maia 200 is a 3nm, 140-billion-transistor accelerator with six HBM3e stacks, 7 TB/s of HBM bandwidth, 10,000 TFLOPS FP4 peak, and a 750W TDP. Its Software Defined Local Access architecture makes dataflow compile-time explicit while keeping data local to tiles, reducing cross-talk and fabric pressure. Microsoft combines this with an all-Ethernet scale-up network that it says can link 6,000 chips across 128 racks, rather than using a separate scale-out fabric. The tradeoff is a hardware-specific programming model: kernels must be tuned through MCCL and the architecture’s explicit data movement to reach its intended efficiency.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article ServeTheHome

Cerebras Talks Going Rack-Scale with Their WSEs at Hot Chips 2026

Cerebras’s CS-4 is its first dedicated rack-scale system, combining three higher-clocked WSE-3 Turbo wafers in one scale-up domain. The company claims twice the token rate and 10× tokens per watt versus CS-3, driven by on-chip SRAM bandwidth quoted at 43,000 TB/s and direct wafer links with as little as two microseconds of latency. Its Nexus rack redesign brings power, liquid cooling, I/O, metering, and leak detection into pluggable “backpacks,” while using 50% fewer components than CS-3. Cerebras expects the platform to support mixed systems with AMD MI455X GPUs and to underpin CS-5 and CS-6, the latter adding stacked DRAM to trade some SRAM for more compute.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article ServeTheHome

NVIDIA’s Groq 3 LPU Accelerators for Heterogeneous AI Compute at Hot Chips 2026

NVIDIA described Groq-derived LPUs as a decode-focused complement to Vera Rubin GPUs, which remain responsible for prefill and attention in a disaggregated inference flow. A 256-LPU LPX rack is specified at 128GB of SRAM, 40 PB/s aggregate SRAM bandwidth, 315 PFLOPS FP8, and 11,000 decoded tokens per second on Gemma 4 31B; NVIDIA cites a third-party benchmark showing four times the next public competitor’s output rate. The deterministic LPU design puts instruction and network scheduling in software, which NVIDIA says also enables power smoothing and thermal-aware placement. The stated tradeoff is efficiency: increasing LPU offload can improve high-interactivity performance by up to 5×, but reduces total-throughput efficiency relative to GPU-only operation.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article r/LocalLLaMA

Apple introduces new Mac Studio with M5 Max and M5 Ultra - up to 512GB of unified memory

The extracted Reddit entry contains only the submitter attribution and link/comments markers, with no discussion or substantive details to summarize. The linked item’s title identifies Apple’s new Mac Studio hardware and a claimed 512GB unified-memory ceiling. No additional claims from the Reddit post are included here.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article Lobsters AI

AI At Home Part 2: Multi GPU Drifting

This practical account explains why local LLM generation is normally constrained by memory bandwidth rather than raw math: each next token requires reading model weights and the existing context. Mixture-of-experts models reduce activated weights per token but still require the full model to reside in fast memory because routing varies token by token. For a four-GPU home server with 32GB per GPU, straightforward layer parallelism fits a larger model but processes layers serially and is bounded roughly by one GPU’s bandwidth minus inter-GPU transfer overhead. The article sets up tensor parallelism as an alternative that divides each layer across GPUs, trading synchronization and interconnect behavior against the chance to use their bandwidth concurrently.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article Lobsters AI

Apple's new desktop computers are designed specifically for local AI development

Apple refreshed the Mac mini and Mac Studio around local inference, pairing a new M6 mini with a M5 Ultra Studio that can reach 512GB unified memory and 1.2TB/s memory bandwidth. Apple says macOS 26.2 enabled low-latency Thunderbolt 5 communication for distributed MLX inference, which has led users to chain machines to run models too large for one mainstream device. The M6 starts at $899 with 16GB, while an M5 Ultra Studio starts at $5,499; the 512GB configuration is expected in late October. The article’s key caveat is that this is primarily a specifications refresh and that local large-model workflows still need substantially more memory than an ordinary developer laptop provides.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article OpenAI News

The full stack behind abundant intelligence

The extracted item provides only a one-sentence description: OpenAI CFO Sarah Friar argues that improvements in chips, compute, models, and products compound to make useful intelligence cheaper and more scalable. It frames the company’s strategy as full-stack coordination rather than a model-only advance. No further evidence, figures, or argument detail is present in the supplied content.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article OpenAI News

Jalapeño’s first results show industry-leading speed and efficiency in AI inference

OpenAI says its first custom inference platform, Jalapeño, delivers 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T on the public InferenceX benchmark. For highly interactive workloads it reports 2.1–4.1× higher performance, while sustained measured power was at or below 550W despite a 700W rating. The core design claim is that keeping KV cache local and co-designing chip, memory, networking, and serving software handles both compute-heavy prefill and bandwidth-heavy decode without transferring state among specialized systems. OpenAI plans deployment by year-end while continuing to use NVIDIA and other external accelerators, and says later Jalapeño generations are already in development.

In: OpenAI’s Jalapeño chip challenges GPU inference
📝 Article r/LocalLLaMA

Apple M5 Server

This Reddit post is presented as a discussion of an Apple M5 server. No readable body was extracted from the post, so the supplied material contains no technical claims, benchmarks, or configuration details to assess. The title alone does not establish what hardware was announced, tested, or proposed.

In: AI builders put verification ahead of autonomy
📝 Article ServeTheHome

Intel Crescent Island 160GB to 480GB LPDDR5X AI GPU at Hot Chips 2026

Intel detailed Crescent Island, a 350W air-cooled PCIe data-center GPU aimed at agentic inference, with 160GB of LPDDR5X on the branded card and designs allowing up to 480GB for partners. The Xe3p part has 32 Xe cores and 256 XMX engines, supports precisions from FP4/MXFP4 through FP64, and emphasizes KV-cache capacity, prefill, speculative decoding, reliability features, and an open software stack including vLLM and SGLang. Intel’s thesis is that agent workloads shift the bottleneck toward capacity, latency, and system coordination, not merely raw compute; it claims a 160GB card can keep FP8 weights and KV cache together more often than a 96GB alternative. The report notes a crucial unanswered question: Intel did not disclose memory-bandwidth figures, so real tokens-per-watt competitiveness remains unproven.

In: AI builders put verification ahead of autonomy
📝 Article ServeTheHome

AMD Helios MI400 System Architecture at Hot Chips 2026

AMD presented Helios as a rack-scale AI system co-designed around a 96-core EPYC Venice CPU, MI455X accelerators, and Pensando Vulcano 800 NICs. A 72-GPU rack is specified at 31TB of HBM4 and 2.9 exaflops of claimed AI compute, with 1.7PB/s HBM bandwidth, 260TB/s scale-up bandwidth, and 43TB/s scale-out bandwidth. Its switched UALoE fabric gives GPUs shared load/store access across the pod, supports virtual-pod isolation and failure recovery, and uses open Ethernet/ESUN standards rather than a proprietary scale-up fabric. The platform also layers in confidential-computing features, programmable transport and congestion control, telemetry, and a redundant Fabric Manager control plane.

In: AI builders put verification ahead of autonomy
📺 Video YT GPU MODE

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives

The lecture explains why GPU collectives, especially all-reduce, have different bottlenecks in prefill and decode. Prefill collective sizes grow with total sequence length and hidden size, making them more bandwidth-sensitive; autoregressive decode produces smaller messages tied to hidden size and batch size. Long contexts enlarge KV cache and can force smaller batches, making those already-small decode collectives frequent and directly critical to each next token. The result is that even microsecond improvements compound across layers and generated tokens, motivating latency-focused collective design rather than bandwidth optimization alone.

In: AI builders put verification ahead of autonomy
📝 Article ServeTheHome

AMD MI400 GPU at Hot Chips 2026

AMD presented its MI400/MI455X architecture as the accelerator inside its forthcoming 72-GPU Helios rack, targeting frontier training, fine-tuning, and persistent inference. AMD quotes 2.9 exaflops, 31 TB of HBM4, and 1.7 PB/s of HBM bandwidth per rack; each MI455X has 432 GB of HBM4 at 23.3 TB/s and 256 work-group processors. Relative to MI355X, AMD highlights larger caches and memory, up to 40.26 PFLOPS MXFP4 peak compute, dedicated data movement, and ROCm measurements such as 3.8x higher FP8 MLA decode performance. The hardware case is paired with ROCm tools and agent integrations intended to reduce the practical cost of moving workloads from CUDA.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article ServeTheHome

NVIDIA Vera Rubin NVL72 Rack at Hot Chips 2026

NVIDIA framed Vera Rubin as an AI-factory redesign for agentic inference, where long contexts, tool calls, and multi-turn work make time to first token, interruptions, and tokens per watt as important as peak FLOPS. Its NVL72 design links 72 GPUs through sixth-generation NVLink, claiming 3.6 TB/s of all-to-all bandwidth per GPU, while a 100 MW factory configuration is rated at 2 ZFLOPS NVFP4 inference, 1.4 ZFLOPS training, 11 PB of HBM4, and 800 PB/s of memory bandwidth. Rubin adds adaptive 2:4 sparsity and a sparse-attention path that NVIDIA says can accelerate downstream SoftMax and BMM2 work about 2x without model changes in most cases. The rack design also pursues warm-water cooling, 800 VDC power, cable-free fanless compute trays, and easier serviceability as system-level contributors to usable throughput.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article ServeTheHome

Waymo Sensor Fusion Processor at Hot Chips 2026

Waymo detailed a purpose-built sensor-fusion ASIC for the latency-critical path from camera, lidar, and radar inputs to embeddings in its autonomous-driving stack. The N5 chip is a 208 mm², sub-75 W package with LPDDR5X, PCIe Gen5 x8, 25G Ethernet, a custom carTPU/ISP/codec/MIPI path, and third-party GPUs for heavier sensor math. Waymo cites 160 INT8 TOPS and 80 FP16 TFLOPS, but emphasizes achieved latency and co-design over raw throughput: ahead-of-time compilation creates model-wide mega-kernels and keeps working sets in SRAM. The processor is designed for low-batch, first-pixel-to-embedding latency and is already deployed in sixth-generation Waymo vehicles, according to the presentation.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article ServeTheHome

Intel Diamond Rapids the 2027 Intel Xeon at Hot Chips 2026

Intel’s 2027 Xeon 7 Diamond Rapids uses a modular design in which compute-building-block chiplets connect to Fabric Hubs that centralize memory and I/O. A top configuration combines four compute blocks for up to 256 cores, 1.28 GB of last-level cache, 1.6 TB/s of memory bandwidth, and 128 lanes configurable for PCIe Gen6, CXL 3, or UPI 3. The 18A-P platform uses Foveros 3D direct die-to-die bonding, puts coherence filtering on die rather than in DRAM, and includes QAT, DSA, and IAA acceleration complexes. Intel is also extending x86 with spill-and-fill instructions that expose 32 integer registers while retaining source-level software compatibility after recompilation.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article ServeTheHome

Arm’s AGI Data Center CPU at Hot Chips 2026

Arm introduced AGI as its first complete commercial server CPU rather than an IP design, a shift that puts the company directly in the server-chip market. The chip uses two N3P chiplets and up to 136 active Neoverse V3 cores, with 12 DDR5 channels, over 800 GB/s of aggregate memory throughput, 96 PCIe Gen6 lanes, CXL 3.0 support, and a 300 W TDP. A UCIe die-to-die link supplies 1 TB/s in each direction so the two chiplets can approach monolithic behavior in a NUMA configuration. Arm is positioning the design for agentic-AI servers, emphasizing energy efficiency, memory bandwidth, and a roadmap of future in-house data-center SoCs.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article ServeTheHome

Fujitsu’s Arm-based Monaka Data Center CPU at Hot Chips 2026

Fujitsu’s Monaka is a 2027 Armv9.3-A server CPU aimed at AI and data-center workloads, with 144 cores, 256-bit SVE2 execution, 12 DDR5 channels, and up to two sockets per node. Its stacked chiplet design puts core dies on TSMC N2P while SRAM and I/O remain on N5, allowing advanced compute technology without putting the entire chip on the newest process. Fujitsu’s main efficiency lever is ultra-low voltage—reported as about 30% below comparable designs—alongside power-saving vector techniques and a floating-point register cache for high-locality workloads such as GEMM. It plans both a 500 W high-performance SKU and a 350 W efficient SKU, while a future Monaka-X is expected to add 1.4 nm fabrication and NVLink Fusion support.

In: Hot Chips puts AI racks and server CPUs on display
📝 Article r/LocalLLaMA

Xiaomi AI Cube announced with 1.2TB/s memory bandwidth

No readable body was extracted for this Reddit post. The raw material supplies only the title, which says Xiaomi announced an AI Cube with 1.2 TB/s of memory bandwidth, without architectural, pricing, availability, or benchmark details. There is not enough extracted information to assess what workload the bandwidth figure represents or how the system compares with alternatives.

In: AI shifts the edge from models to systems
📝 Article Lobsters AI

AI Chip Architectures

This survey explains AI hardware through the memory wall: training and prompt prefill are compute-heavy matrix-matrix workloads, whereas autoregressive decode becomes bandwidth-heavy matrix-vector work that repeatedly reads model weights and the KV cache. It argues that batching, speculative decoding, and multi-token prediction recover utilization by converting more work back toward matrix-matrix operations, though long contexts can shift the bottleneck to KV-cache bandwidth. The piece maps the deployed landscape—GPUs, systolic accelerators such as TPUs and Trainium, Cerebras’s wafer-scale engine, and Groq’s LPU—as different answers to where data lives and how it moves to compute. Its GPU walkthrough shows NVIDIA steadily pushing matmul execution toward larger, more asynchronous, lower-precision Tensor Core operations while retaining CUDA’s general-purpose programming model.

In: AI shifts the edge from models to systems
📝 Article ServeTheHome

d-Matrix Raptor 3D-DRAM Accelerator for Generative Inference at Hot Chips 2026

d-Matrix presented Raptor as a 3D-DRAM inference accelerator aimed at the widening gap between model/KV-cache demands and conventional memory bandwidth. It contrasts SRAM’s extreme bandwidth but tiny capacity with HBM’s larger capacity but roughly 20 TB/s practical package ceiling, proposing compute stacked directly atop DRAM for about 0.3–0.4 pJ/bit vertical I/O—roughly a tenth of HBM’s chip-level energy. The company says a 72-card system with 32 GB per card can hold a frontier-class model such as Kimi K3 at one-million-token context, and claims approximately 1,000 tokens per second per user for a three-trillion-parameter-class model. Its design addresses non-aligned DRAM bank accesses with stream blocking, lowers I/O toggling through stream flipping, and uses ECC, refresh techniques, and bank chaining to preserve performance and yield under heat; the presentation claims about 20× HBM bandwidth density and 13.5× lower power per GB/s, though the article notes that real-world validation and drawbacks remain unclear.

In: AI hardware makers redesign memory for inference
📝 Article ServeTheHome

Samsung Evolving HBM Base Die at Hot Chips 2026

Samsung’s roadmap turns HBM’s base die from a mostly passive PHY and routing layer into a logic-rich component that can offload work from the accelerator. It says HBM5 can exceed 60 GB and 6 TB/s per stack, driving the base die toward 4 nm logic beginning with HBM4; custom HBM can use that logic for memory controllers, repair SRAM, sensors, self-test, external-memory interfaces, and eventually processing elements. The move creates thermal challenges as PHY power density rises, for which Samsung proposes a Heat Path Block that it says lowers peak temperature by more than 35% when PHY coverage is high. Its longer-term zHBM concept vertically integrates the XPU and memory stack, eliminating conventional 2.5D interfaces; Samsung estimates that four zHBM stacks alongside a 1,200 W GPU could save about 100 W while increasing bandwidth.

In: AI hardware makers redesign memory for inference
📝 Article ServeTheHome

Micron Evolving Memory Architectures for AI at Hot Chips 2026

Micron argues that AI accelerators are increasingly memory-bound because accelerator compute has been growing roughly 3× every two years while 2.5D-attached HBM bandwidth has risen at less than 2× over the same period. HBM raises bandwidth and capacity through denser 3D stacks and more I/O, but the cost is substantial: Micron says providing the same capacity with HBM3E consumes about three times the silicon of DDR5. The company emphasizes that memory can occupy more than eight times the surface area of a GPU die in an HBM4 package, putting advanced packaging, reliability, and thermal engineering at the center of accelerator design. Its proposed path pairs faster I/O, larger interposers, memory-optimized die-to-die links, and co-packaged optics with liquid cooling, hybrid bonding, and materials changes to manage heat and mechanical stress.

In: AI hardware makers redesign memory for inference
📝 Article ServeTheHome

Bosgame M5 AMD Ryzen AI Max+ 395 128GB AI Desktop Review

ServeTheHome reviews the Bosgame M5, a Ryzen AI Max+ 395 desktop with 128GB LPDDR5X memory, positioned at roughly $500–$1,100 below AMD’s comparable Ryzen AI Halo machine while retaining the same CPU and memory capacity. The hardware emphasizes desktop convenience: a compact angled chassis, vertical stand, front 10Gbps USB-A ports, SD reader, selectable quiet-to-high-power performance modes, HDMI, DisplayPort, USB4, and multiple additional USB ports. The reviewer’s major criticism is networking: it has only Realtek 2.5GbE when 10GbE would make it much more competitive, although a USB 10GbE adapter is possible at the cost of a port. Cooling vents on the top and bottom mean it should not be stacked under other equipment.

In: GitHub puts shared Copilot agents in Slack and Teams
📝 Article ServeTheHome

Kioxia CD9P 7.68TB E3.S NVMe SSD Review Fast Gen5 Storage

Kioxia’s CD9P-R is a 7.68TB, read-intensive PCIe Gen5 data-center SSD rated at 1 DWPD for five years; a 6.4TB mixed-use CD9P-V variant is rated for 3 DWPD. The review emphasizes its E3.S EDSFF form factor, whose smaller Gen5 x4 connector improves density, cooling, and signal integrity compared with U.2-style designs. It argues EDSFF is likely to become necessary with Gen6, where retaining SAS compatibility in older U.2 connectors becomes less compelling than higher interface speeds. The family ranges from 1.6TB to 30.72TB in E3.S and up to 61.44TB in U.2, with the tested class advertised at 14.8GB/s reads, 7GB/s writes, 2.6M read IOPS, and 450K write IOPS.

In: AI tools shift from copilots to organizational infrastructure
📝 Article ServeTheHome

Cerebras Intros Faster WSE-3 Turbo Processor and First Rack-Scale CS-4 System

Cerebras introduced the WSE-3 Turbo, a faster version of its 5 nm wafer-scale WSE-3 with 900,000 AI cores, 44 GB of SRAM, and 4 trillion transistors. It claims 250 sparse-FP16 PFLOPS—twice the prior WSE-3—along with doubled on-wafer SRAM bandwidth to 42.2 PB/s, mesh bandwidth to 53.5 PB/s, and external networking to 300 GB/s. The new CS-4 rack combines three Turbo wafers, claiming six times the performance of a single CS-3 system, while the modular Nexus platform is intended to support future hardware swaps. Power consumption has not been disclosed; the analysis infers that the new rack’s ability to deliver twice the power may put a Turbo wafer near 54 kW.

In: AI coding agents gain autonomy, while MCP goes stateless
📝 Article ServeTheHome

Qualcomm Modular Open-Sourced at ModCon 2026

Qualcomm has open-sourced the Modular AI platform it recently acquired, licensing its code under Apache 2.0 with LLVM exceptions. The ambition is a common software layer that can run inference from local devices such as Macs and Snapdragon systems through NVIDIA and AMD GPUs, Trainium, TPUs, and Qualcomm accelerators. AMD’s participation at ModCon is presented as a significant sign of cross-vendor support despite its own AI-hardware business. Qualcomm is betting that an open, portable stack can reduce the work needed to bring software to new hardware, challenging the advantage of NVIDIA’s broader CUDA ecosystem.

In: OpenAI pauses frontier training over Astra cyber risks
📝 Article Hugging Face Blog

Same Cluster, 33 Points More Utilization: What Changed Was the Order

Dharma AI reports that a constraint-aware GPU allocator raised utilization by as much as 33 percentage points and priority-weighted output by as much as 105% versus FIFO scheduling on identical hardware and workloads. The key is to treat real-time inference demand as a changing curve rather than reserve its peak capacity for an entire day, then fill troughs with contiguous batch work selected by priority across the planning horizon. In five contended scenarios, utilization rose from a 52–85% range to 72–88%, while priority-weighted value increased 24.6–105.1%, averaging 52%. The article also shows why utilization alone is insufficient: in one 64-GPU test both schedulers reached 44.9% utilization and completed 27 of 30 jobs, but the optimizer still produced 15.9% more priority-weighted value.

In: Stripe moves to buy OpenRouter for $7B
📝 Article ServeTheHome

Intel Xeon 658X Review: Granite Rapids For Workstations

Intel’s Xeon 600 workstation line brings Granite Rapids and Redwood Cove cores to a platform that had gone more than three years without new workstation silicon. The family scales to 86 cores, up 43% from the previous top workstation Xeon, producing Intel-claimed gains of 61% in multithreaded work but only 9% in single-threaded performance. It also expands platform capacity with up to 128 PCIe Gen5 lanes, CXL 2.0, DDR5-6400, and—on higher-end SKUs—MRDIMM support up to DDR5-8000. Buyers should watch the segmentation: chips with 16 or fewer cores get only 80 PCIe lanes and four memory channels, while an 18-core roughly $1,199 entry point is needed for the fuller I/O and memory configuration.

In: Stripe moves to buy OpenRouter for $7B
📝 Article ServeTheHome

ADT R27A-BK3 EDSFF E1.S and E3.S to PCIe Slot Review

ServeTheHome tested low-cost ADT adapters that let E1.S or E3.S EDSFF SSDs occupy a PCIe slot, with fan-equipped models costing about $60. The tested card combines a 4-pin PWM blower, a PCIe x4-to-EDSFF 1C converter, and a shared connector supporting either drive form factor; its substantial size accommodates an E3.S drive and cooling hardware. Buyers need to match the adapter to their drive and host requirements, particularly because both PCIe Gen5 and Gen4 versions exist and U.2 variants are also sold.

In: Nvidia’s $500 billion AI financing plan lacks commitments
📝 Article ServeTheHome

KYY X90D “Triple” Portable Monitor Review

KYY’s $250 X90D is a foldable laptop-attached setup with two 15.6-inch, 1920×1080, 60 Hz IPS panels, making three screens once the laptop display is included. The review finds its protected folded form and integrated handle useful for travel, but stresses that it is substantially heavier and bulkier than a conventional portable monitor and can tip the overall assembly if set up carelessly. Each display has independent controls, and the reviewer was able to drive both using a Dell P514260 through one cable, without the extra power adapter required by the related X90G Quad. MacBook users needed Silicon Motion software from the included USB drive to connect all three displays, so the experience is not entirely plug-and-play.

In: Flue 2 brings React hooks to agent harnesses
📝 Article ServeTheHome

160-bay NVMe SSD 4U Server Shown at FMS 2026 Supermicro ASG-4116S-NU160R

Supermicro showed a 4U, single-socket AMD EPYC system with 160 U.2 NVMe bays, plus four front E1.S bays, capable of roughly 19–20PB when populated with 122–128TB SSDs. Because 160 four-lane drives would otherwise require 640 PCIe lanes before networking, the design uses PCIe switches and serviceable slide-out SSD and CPU/networking trays. Three PCIe Gen5 x16 slots can take 400GbE NICs or DPUs, while the one EPYC 9005 CPU replaces the CPUs and DIMMs of about seven 24-bay 2U servers. The tradeoff is reduced per-drive peak sequential bandwidth, but the article estimates 130–137W per PB at the outlet using redundant 2.6kW supplies—less than the power needed just to run roughly 500 hard drives for comparable capacity.

In: Chinese labs take command of open-model frontier
📝 Article ServeTheHome

Microchip Switchtec 160-Lane PCIe Gen6 Switch Shown at FMS 2026 with XpressConnect PCIe 6 Retimer

Microchip showed its Switchtec PFX 160xG6, a 160-lane PCIe Gen6 switch, and an XpressConnect PCIe Gen6 retimer operating at FMS 2026. The switch can expose up to 20 ports, though AI systems commonly consume them with x16 GPUs and NICs, and it enters a market increasingly central to AI infrastructure rather than just storage servers. Retimers matter because Gen6 links have shorter reach as systems become physically larger; Microchip’s part can also retime CXL 3.1. The report places Microchip against Broadcom, Astera Labs, Marvell, and Phison, while noting Microchip appears more PCIe-focused than vendors emphasizing combined PCIe/CXL switching.

In: OpenAI and Google slash the cost of agentic AI
📝 Article ServeTheHome

Minisforum N5 Max Review with AMD Ryzen AI Max+ 395

Minisforum’s N5 Max combines AMD’s 16-core, 32-thread Ryzen AI Max+ 395 and 64GB LPDDR5X memory with a five-bay NAS chassis. It pairs accessible tool-less SATA bays with dual 10GbE, three USB4 v2 Type-C ports rated up to 80Gbps, HDMI, and an internal power supply. The storage-focused design uses a JMicron JMB585 SATA controller and provides per-bay and per-LAN status indicators. The review’s central tradeoff is whether flagship Strix Halo compute and NAS-oriented storage connectivity justify a less flexible, purpose-built enclosure.

In: GitHub Copilot rolls out cheaper vision coding model
📝 Article ServeTheHome

Panduit E36G18L PDU Review A Sweet Managed and Switched by Outlet PDU

ServeTheHome reviews the 70-inch, zero-U Panduit E36G18L PDU, built for a 30A 208V NEMA L6-30P circuit and divided into two magnetically protected, color-labeled breaker banks. It offers 36 outlets: 18 four-in-one sockets accepting C13, C15, C19, or C21 plugs, plus 18 two-in-one C13/C15 sockets. Each outlet has a status LED, and the unit works with locking cables. The review emphasizes that it is more than a power strip, with per-outlet monitoring, switching, management, and a temperature-monitoring capability.

In: OpenAI widens access to frontier cyber models
📝 Article ServeTheHome

WisdPi WP-UT9 USB 10GbE Adapter Review

The WisdPi WP-UT9 is a compact USB-C-to-10Gbase-T adapter built around Realtek’s RTL8159 controller, matching the chipset and expected behavior of the previously reviewed Xikestor SKN-U310GT. Its RJ45 port supports 100 Mbps through 10 GbE, including 2.5 and 5 GbE multi-gig links, while the USB-C connection supplies both data and power; the aluminum shell is intended to dissipate heat from the controller and PHY. The main physical differentiator is its included USB-C-to-USB-C cable, though users needing more length will likely need a replacement. Linux support is improving—the RTL8159 integration is queued for Linux 7.2’s r8152 driver—but earlier kernels still need Realtek’s out-of-tree driver or a distribution backport.

In: AI rollout resistance turns on job-security promises
📝 Article ServeTheHome

Delta’s GoCool-150 Goes Big To Enable 150kW Liquid-To-Air Cooling for ASRock Rack’s NVIDIA VR NVL72

Delta’s GoCool-150 is a liquid-to-air coolant distribution unit intended to let an air-cooled facility host a direct-liquid-cooled NVIDIA Vera Rubin NVL72 rack without installing a building-scale liquid-cooling loop. It can reject up to 150 kW of heat, circulates 225 liters of coolant per minute, and is designed to supply 45°C coolant in line with NVIDIA’s Vera Rubin cooling targets. The 1,200 kg, 2.3-meter unit relies on five hot-swappable pumps and 32 hot-swappable 200 mm fans that move 17,658 CFM; the fan array can reach 81 dBA. It also consumes 18 kW itself, illustrating the infrastructure overhead created as AI racks push toward 200 kW today and planned densities above 600 kW.

In: Claude Code makes auto mode the default