LPDDR and Specialization in AI Infrastructure

The growing public market attention to memory companies such as Micron and SK hynix reflects memory’s role as a critical bottleneck in AI infrastructure. We have been examining memory as a bottleneck for some time, including through our earlier work on CPUs and memory. Tight HBM supply and rising prices have brought up a pressing question: where could alternative memory architectures offer better economics for AI inference?

LPDDR is one option worth examining. It offers a different approach to memory capacity and power efficiency while avoiding the specialised packaging and integration required by HBM-based designs. Its potential depends on how effectively the surrounding processor architecture and software meet a workload’s requirements.

To explore these trade-offs, we invited our portfolio company Positron, who designs its next-gen solution centered around LPDDR5X memory, to an expert sharing session on how inference workloads are shaping processor and memory design. This study builds on that discussion to examine when an alternative architecture can improve serving economics, and what investors should look for when assessing whether those improvements can become a durable commercial advantage.

Large language models place two major demands on working memory: model weights and the key-value cache, which retains information from previously processed tokens. Prefill processes the prompt and is often compute-intensive. Decode generates the response and frequently becomes constrained by memory bandwidth, particularly at smaller batch sizes.

Capacity determines how much model and context state a system can hold; bandwidth determines how quickly it can access that information. Increasing one does not automatically solve a shortage of the other.

Agentic workflows make this distinction more consequential. A coding agent may read a repository, modify files, run tests and revise its approach across repeated model calls. Matching cached prefixes can avoid repeated prompt processing. The value of retaining that context depends on its size, its likelihood of reuse and the speed at which it can be retrieved.

Different memory technologies address these requirements in different ways. The table below summarizes their roles. LPDDR is particularly useful for examining how a memory choice can influence the design of the surrounding inference system.

Memory or storage	Typical role in inference	Principal strength	Main trade-off
SRAM	Holds frequently reused information close to computation	Very fast access and high local bandwidth	Expensive in chip area, limiting capacity
HBM	Feeds accelerators with weights and active cached states	Exceptional bandwidth in a compact package	Specialised manufacturing and integration
LPDDR	Provides primary inference memory or additional context capacity	Efficient capacity with a different integration path	High aggregate bandwidth requires careful system design
Server DDR	Supports CPU inference, model staging and context storage	Large, expandable memory pools	Access from a separate accelerator can be limited by the connection
NAND-based SSDs	Stores model files, retrieval data and less-active context	Low storage cost per GB	Slower access requires loading and transfer scheduling

LPDDR, or Low-Power Double Data Rate memory, can serve two distinct roles in inference. As primary working memory, it holds model weights and active KV cache and must support token generation directly. As an additional memory tier, it retains context for reuse alongside HBM. These roles create different opportunities: reducing the hardware needed to accommodate a workload, or reducing the cost of discarding and recomputing context.

2.1 Primary Working Memory

Positron’s Asimov illustrates a memory-first approach to using LPDDR as primary working memory. The company starts from the premise that generative decode is often constrained by moving model weights and KV-cache data rather than by peak arithmetic throughput. It therefore designs around memory bandwidth and capacity first, with compute resources sized to match the memory system. This approach allows Asimov to realise more than 90% of its available memory bandwidth on transformer workloads, compared with less than 30% for GPUs in its comparison.

This principle shapes both the choice of memory and the chip’s internal dataflow. Asimov uses commodity LPDDR5X rather than HBM and is specified with 288 GB to 2.3 TB of memory and 2.76 TB/s of realisable bandwidth per chip. Positron also places weight memory alongside processing elements to reduce data movement and uses a reconfigurable systolic array for the different computational patterns found in feed-forward and attention operations. The objective is to keep the compute supplied with data while gaining substantially more memory capacity per chip and avoiding the cost, power and supply-chain constraints that Positron associates with HBM.

LPDDR is therefore part of a broader architectural choice rather than a direct component substitution. The compute architecture, memory system and dataflow are designed together around the bandwidth available from LPDDR. Positron targets decode and other workloads for which realised memory bandwidth and capacity matter more than maximising theoretical compute.¹

2.2 Additional Context Memory and Memory Tiering

LPDDR can also provide additional context capacity alongside HBM through memory tiering. In this arrangement, HBM supports active processing, while LPDDR retains a larger pool of context that can be brought back when needed. This allows the system to preserve more KV cache without keeping all of it in HBM.

Micron’s study of NVIDIA Grace-Hopper systems illustrates how LPDDR can serve as one of these memory tiers. The platform combines GPU-attached HBM with CPU-attached LPDDR5X. The additional LPDDR capacity allows the system to retain more KV cache for reuse, reducing the need to discard cached context and recompute it when a request returns. Using Llama3-70B, Micron’s experiments and projections indicate that expanding LPDDR capacity from 512 GB to 1.5 TB per CPU could reduce time to first token by up to 98% in its long-context scenario and support larger batches of concurrent requests.²

Morgan Stanley’s October 2026 channel checks suggest that lower HBM capacity per chip could shift part of the inference KV cache into host memory, supporting demand for RDIMM and SOCAMM modules.³ This forecast points to broader adoption of tiered memory architectures, although it does not establish whether DDR or LPDDR will be preferred in a particular system.

NVIDIA Dynamo shows how software can coordinate such a hierarchy. It can offload KV cache from GPU memory to CPU RAM, local SSDs or remote storage, allowing less-active context to be preserved and retrieved for reuse. When retrieval is faster and cheaper than recomputation, this approach can reduce the work required for subsequent model calls. Dynamo supports memory tiering across these locations; the host-memory technology depends on the platform.⁴

2.3 Current and Announced LPDDR Platforms

The platforms below show LPDDR appearing across different system architectures and stages of commercial development.

Company	Platform	What it illustrates
NVIDIA	Grace CPU	An established server platform using LPDDR5X on the CPU side.5
Qualcomm	Dragonfly AI200	A published inference platform specification with 768 GB of LPDDR5X per accelerator card.6
Intel	Crescent Island	An announced inference GPU with up to 480 GB of LPDDR5X in a 350 W air-cooled PCIe design.7
d-Matrix	Corsair	Combines LPDDR5 with SRAM-based in-memory compute on organic substrates; full production announced in June 2026.8
HyperAccel	Bertha LPU	An LPDDR5X-based inference architecture reported at the engineering-sample stage in September 2026.9
Positron	Asimov	An announced LPDDR5X accelerator targeting 288 GB–2.3 TB per chip, with availability planned for 2027.10
Huawei	Atlas 300I Duo	An inference card with 48 GB or 96 GB of LPDDR4X and 408 GB/s of total memory bandwidth.11
Cambricon	MLU370-S4/S8	Cloud AI accelerator cards with 24 GB or 48 GB of LPDDR5 and 307.2 GB/s of memory bandwidth.12
Sunrise	Qiwang S3	An announced LLM and agent inference GPU using LPDDR6, with mass production planned for end-2026.13

The range of designs suggests that adopting LPDDR is not, by itself, a source of differentiation. The commercial test is how effectively each system converts its memory capacity and bandwidth into useful serving performance. For investors, this shifts the focus from the memory component to the processor architecture, dataflow and software built around it.

3.1 Capacity and Device Count

The first economic question is how much hardware a workload needs simply to fit in memory. Both model weights and retained context contribute to that requirement. One trillion parameters at four bits each require approximately 500 GB for weights alone, before adding quantisation metadata, working memory and the KV cache.

Positron’s Titan illustrates how larger memory capacity can change this requirement. Its announced design combines four or eight Asimov chips, with up to 18.4 TB of accelerator memory per system.¹⁴ Keeping a larger workload within one server could reduce the hardware, networking and coordination needed to distribute it across servers. The saving depends on whether that configuration also meets the required throughput and response time.¹⁴

Mixture-of-experts models make this opportunity particularly relevant. DeepSeek-V3 has 671 billion total parameters but activates 37 billion per token.¹⁵ This creates a substantial requirement for resident capacity, while each token uses only a portion of the model. Large, affordable memory pools can accommodate that footprint, with batching and expert routing shaping the bandwidth required.

3.2 Context Retention and Recomputation

A larger memory pool can retain more KV state, reducing the need to discard and later recompute it. This is particularly relevant to agentic workflows that repeatedly return to shared instructions, code or documents.

The economic benefit depends on how frequently that context is reused and the cost of retrieving it. Retaining context that is rarely needed offers less value, while slow retrieval can erode the benefit of avoiding recomputation.

Software influences this trade-off. KV-cache compression can reduce both the memory footprint and the amount of data transferred.¹⁶ Affordable capacity and efficient cache management can therefore work together to reduce the resources needed to serve repeated interactions.

3.3 Power and Rack Constraints

Memory shares a finite power budget with computation. Its efficiency affects both operating expenditure and the amount of useful work a rack can support.

NVIDIA’s Grace CPU provides an established server example. NVIDIA reports up to 500 GB/s at approximately 16 W for its LPDDR5X memory subsystem.⁵ The configuration matters: its single-CPU specifications list up to 512 GB/s for 120/240 GB and 384 GB/s for 480 GB. These figures illustrate LPDDR’s potential in a specific CPU design; they do not establish a general power advantage over HBM.

Lower memory power can leave more of the system’s power budget available for processing and reduce cooling requirements. The commercial measure, however, is the useful throughput delivered within the available power budget.

Deployment options also matter. Intel’s announced Crescent Island design combines substantial LPDDR5X capacity with a 350 W air-cooled PCIe card.⁷ Such configurations may be useful where existing power and cooling infrastructure constrains expansion.

From a performance perspective, HBM is the natural choice for bandwidth-intensive AI accelerators. The commercial question emerges when growing demand presses against the capacity needed to manufacture and integrate it. LPDDR offers a different path for workloads that can operate with less peak bandwidth. Evaluating that path requires separating two issues: whether systems can be produced and deployed, and what they cost once available.

4.1 Production, Supply and Deployment

HBM-based accelerators depend on specialised memory production and advanced integration. Micron has reported that HBM3E requires approximately three times the wafer supply of DDR5 to produce the same number of bits at the same manufacturing node. This comparison is specifically with DDR5, rather than LPDDR; it nevertheless demonstrates the additional manufacturing resources required to deliver HBM’s capabilities.¹⁷

HBM also involves specialised stacking and integration with the processor. TSMC’s CoWoS platform, for example, connects processors and HBM through interposer technologies. LPDDR avoids this particular integration path.¹⁸ TrendForce reported in April 2026 that CoWoS shortages had persisted despite expansion, with pressure extending to equipment, substrates and packaging materials.¹⁹ In September, Micron reported agreements covering the vast majority of its calendar 2027 HBM bit supply.²⁰

These disclosures highlight the importance of securing capacity. They do not imply that finished HBM accelerators are unavailable, since established suppliers may already have secured the allocations they need. With that being said, we need to note that LPDDR still participates in the wider DRAM market. TrendForce has linked suppliers’ allocation of capacity toward server and HBM products to tighter LPDDR supply. Choosing LPDDR therefore changes the supply-chain exposure without eliminating it.²¹

Production capacity alone does not make a system deployable. A potential path is demonstrated by Micron’s SOCAMM2, which brings LPDDR5X into a modular, serviceable format designed for servers, expanding its usefulness beyond soldered-memory implementations.²²

The economic benefit emerges when a working system becomes operational sooner. Earlier deployment can bring forward revenue, accelerate customer launches and reduce the time committed capital waits to generate returns.

4.2 Price and Complete-System Economics

Memory pricing helps explain the incentive to explore alternative architectures. In a September 2025 interview, HyperAccel CEO Joo-Young Kim estimated that LPDDR5X cost one-third to one-fifth as much as HBM per GB.²³ The comparison is historical and vendor-specific but illustrates the potential size of the component-cost difference motivating LPDDR-based designs.

But this estimate should be treated as historical and directional rather than as a current procurement comparison. The relative price of HBM and LPDDR can vary substantially with the memory generation, order volume, contract duration, timing and the buyer’s access to allocated supply: estimates based on large-customer contract pricing can place HBM below LPDDR per GB, while small-volume quotations can place HBM several times above it.

Memory pricing explains part of the incentive to explore LPDDR. The commercial test is whether the resulting system delivers the required serving performance at a lower total cost. Positron’s TitanRack comparison illustrates this system-level test. For DeepSeek R1 0528 at FP4, with 8K input and 1K output tokens, the company models approximately 2.4 times as many total tokens per dollar of ownership cost as NVIDIA GB300 NVL72, at the same generation speed of 100 output tokens per second per user.¹⁴

This brings the price discussion back to architecture. A component-cost advantage becomes commercially valuable when the processor, memory and software turn it into better serving economics. That is the opportunity for specialization.

Our work on memory has reinforced a broader conclusion: inference is a system-design problem. Memory capacity and bandwidth matter, but their value depends on how they interact with compute, interconnects, software and the operating environment. The resulting system must also be manufactured, qualified and brought into operation quickly enough to turn invested capital into productive capacity.

As Darren of Positron pointed out during our expert session, inference is becoming increasingly segmented. Prefill and decode, dense and sparse models, and interactive and high-throughput applications place different demands on infrastructure. These differences create opportunities for companies that design around a clearly defined workload.

LPDDR illustrates this broader shift. Its potential cannot be realised through memory substitution alone. The processor architecture, memory system and software must work together to deliver better economics under the customer’s actual serving requirements.

With both established vendors and emerging companies exploring LPDDR-based designs, adopting the memory technology alone offers limited differentiation. The next challenge is translating technological advantages into customer adoption. As investors, we look for advantages that extend beyond the component choice: distinctive dataflow, effective serving software, deployment time, and the ability to support changing models while preserving the system’s economic benefit.

At 3C, this is the opportunity we see in specialization: companies that understand a valuable workload deeply enough to deliver a measurable advantage, make it practical to deploy and sustain it as customer requirements evolve. Our investment in Positron reflects our conviction in this approach.