The Agentic Post
Breaking
Black October 2026: Every Major AI Security Incident, Explained  ·  ARTEX: The Open-Source AI Hacking Tool Behind October’s Biggest Attacks  ·  Chinese Hackers Used AI Agents to Hit 2,250 Microsoft 365 Accounts  ·  PHP2Shell Is Now Being Exploited, and Attacks Began Before the Patch  ·  OpenAI-Backed Thrive Buys ADT for $15.3 Billion to Rebuild It With AI  ·  The AI Data Center Backlash Hits Minnesota and Australia in One Week  ·  
Home/Hardware & Robotics/Chips & GPUs
A New Memory Tier for AI Bottlenecks

A New Memory Tier for AI Bottlenecks

Chips & GPUs

SK hynix and SanDisk published the first open specification for High Bandwidth Flash, a new memory tier between HBM and SSDs aimed at easing AI inference bottlenecks.

AP
Staff
August 5, 2026 · 2 min read

SK hynix and SanDisk released the first open technical specification for a new memory category on August 4, 2026, aimed squarely at a bottleneck that’s becoming as important to AI progress as raw chip supply: getting enough memory bandwidth and capacity to actually feed today’s models.

Quick facts

  • SK hynix and SanDisk published the first standard specification for High Bandwidth Flash (HBF) through the Open Compute Project on August 4, 2026, at the FMS 2026 conference.
  • HBF is a new memory layer positioned between High Bandwidth Memory (HBM) and SSDs, combining HBM-like speed with NAND-based capacity.
  • Initial specifications support up to 512GB per device with three bandwidth grades, roughly 0.4 to 3.0 terabytes per second, using the UCIe chiplet interconnect.
  • Google and Tenstorrent joined the HBF consortium during standardization, alongside founding members SK hynix and SanDisk.
  • The specification arrived just six months after the HBF technology workstream launched under OCP in February 2026.

Why a new memory tier, instead of just more HBM

Per SK hynix’s own announcement, the problem HBF targets is specific: as models scale into the trillions of parameters, HBM capacity on accelerator packages has become both a technical and financial bottleneck, HBM is fast but expensive and capacity-constrained. HBF is pitched as a middle tier: not as fast as HBM, but far higher capacity, and cheaper, letting AI infrastructure providers mix tiers intelligently rather than relying purely on the most expensive memory available for every workload.

Making it an open standard is the strategic bet

Publishing the spec openly through the Open Compute Project, rather than keeping it proprietary, is a deliberate move to build a broad ecosystem fast, the same logic behind open hardware standards generally: interoperability and wide adoption matter more early on than capturing the whole category for one company. Google and Tenstorrent’s participation as consortium members during the standardization process, not just as customers afterward, suggests real buy-in from companies that will actually build systems using it.

This is the same memory-wall problem showing up everywhere

This lands alongside a broader pattern in AI hardware coverage this year: power, not just chip supply, has become a real constraint on new capacity, and now memory architecture is getting the same scrutiny. Tiered memory, using HBM, HBF, and SSDs together rather than defaulting to the most expensive option everywhere, is emerging as a genuine cost and efficiency lever for the whole industry, not just a niche optimization.

Key takeaway

HBF won’t replace HBM, it’s meant to sit alongside it, but if it gains real adoption it could meaningfully lower the cost of running large AI inference workloads at scale, a genuine, if unglamorous, lever on the economics of the whole AI buildout.

Related

Edge AI Chips vs Data Center Chips
Chips & GPUs

Edge AI Chips vs Data Center Chips
The AI Chip Supply Chain, Explained
Chips & GPUs

The AI Chip Supply Chain, Explained
TPU vs GPU: What Actually Differs
Chips & GPUs

TPU vs GPU: What Actually Differs
Up Next
Microsoft Caps AI Token Spending

Microsoft Caps AI Token Spending

Big Tech

Microsoft introduced division-level AI token budget caps and switched its internal default model to a cheaper option, even as it publicly promotes maximum Copilot adoption.

AP
Staff
August 5, 2026 · 2 min read

Microsoft is telling its own engineers to rein in AI usage, capping token budgets by division and switching the internal default to a cheaper model, even as the company’s external pitch this year has been that every developer should be running Copilot.

Quick facts

  • Microsoft EVP Jay Parikh emailed staff introducing division-level “AI token budget targets,” effective as of July 2026, first reported by 404 Media.
  • His framing was direct: “Tokenmaxxing is not what we are optimizing for. I want all of us focused on maximizing outcomes.”
  • Microsoft made OpenAI’s GPT-5.6, described internally as cheaper to run, the default model for internal use.
  • Internal guidance says many engineers currently spend “in the range of hundreds of dollars a month to a few thousand dollars” on tokens.
  • Microsoft reportedly cancelled most Claude Code licenses inside its Experiences and Devices group in May, directing engineers toward GitHub Copilot CLI instead.

The real irony in who’s writing this memo

Per 404 Media’s original reporting, the striking part isn’t the policy itself, cost discipline on a new expense category is a normal thing for a large company to do, it’s that Microsoft is doing it while its entire external enterprise pitch is built around encouraging maximum Copilot adoption. Parikh’s own email tries to thread that needle directly, insisting the company isn’t backing off being “AI-first,” just applying the same spending discipline it applies to any other resource.

Microsoft isn’t alone in this

This fits a pattern reporting has tracked since June across AT&T, Meta, Uber, Walmart, and Amazon, all of which have introduced some form of capping or throttling employee AI spending after early, largely unrestricted adoption. At Meta specifically, engineers have reportedly been competing on an internal “Claudeonomics” leaderboard tracking token usage, a dynamic multiple executives across the industry have publicly criticized as rewarding spend over actual output.

Why this matters beyond one company’s internal memo

The core tension Microsoft is navigating internally is the same one its own enterprise customers face: the value an AI coding tool produces has to comfortably exceed its token cost, and that math doesn’t automatically hold just because usage is high. A major AI vendor publicly rationing its own employees’ use of the exact category of tool it sells is a genuinely useful real-world data point on where that value threshold actually sits.

Key takeaway

If your own organization has been measuring AI adoption by usage volume alone, Microsoft’s shift toward “impact per token” over raw token spend is a framing worth adopting before a similar budget conversation forces the issue internally.

Powered by CookieAdmin