In-Memory Computing Chips in China: When Edge AI Outgrows Traditional NPUs

In late 2025, a team at a Hangzhou robotics startup benchmarked their latest visual navigation model on a state-of-the-art NPU. The model was modest by cloud standards—eight million parameters, INT8 quantized, designed for real-time obstacle detection on a warehouse AGV. The NPU was equally modest, a 4-TOPS part from a leading domestic vendor, fabricated on a 7nm node. On paper, the match was clean. In practice, the NPU spent seventy percent of its energy budget and sixty percent of its cycle time moving weights and activations between DRAM and compute arrays. The actual matrix multiplications consumed less than twenty percent of the total power envelope. The team had not built an AI accelerator. They had built a very expensive memory interface with some arithmetic units attached.

This is not a vendor-specific problem. It is the memory wall, and it is where traditional NPU architectures hit their physical limit. The von Neumann bottleneck—data shuttling between memory and processor—has been the defining constraint of computing for seventy years. For cloud AI, it is manageable: large batches amortize memory access costs across thousands of operations. For edge AI, where batch sizes are one, latency budgets are milliseconds, and power budgets are milliwatts, the overhead is fatal. Every weight fetch from off-chip DRAM costs hundreds of picojoules. Every MAC operation costs fractions of a picojoule. The arithmetic is cheap. The logistics of getting data to the arithmetic is ruinously expensive.

Chinese semiconductor researchers and startups recognized this constraint earlier than most, driven by the specific demands of domestic edge AI markets: autonomous vehicles in complex urban environments, industrial inspection at production line speeds, battery-powered surveillance drones, and wearable health monitors that must run for days. These applications need inference throughput that traditional NPUs cannot deliver within their power and thermal envelopes. The response has been a pivot to in-memory computing (IMC)—architectures that perform computation directly within memory arrays, eliminating the data movement that dominates traditional designs.

The concept is not new. It has been explored in research since the 1990s. What changed in 2025-2026 is that Chinese foundries, fabless chip designers, and system integrators converged on practical implementations that could be manufactured at scale, integrated into real products, and programmed with familiar tooling.

In a traditional NPU, weights are stored in DRAM or SRAM, fetched into register files, and fed into multiplier arrays. The energy cost is dominated by the fetch: moving a 32-bit weight from DRAM to the MAC unit consumes roughly two hundred times the energy of the multiply itself. In an IMC architecture, weights are stored in a memory array—typically SRAM, resistive RAM (ReRAM), or phase-change memory (PCM)—and the multiply-accumulate operation is performed by reading the memory in a way that encodes the computation into the analog properties of the bitcell array.

For SRAM-based IMC, this usually means configuring the bitcell array as a crossbar where word-line voltages encode input activations and cell conductances encode weights. The bitline current is the analog sum of products, computed in a single read operation. For ReRAM-based IMC, the resistive state of each cell directly represents a weight value, and Ohm’s law performs the multiplication as current flows through the cell. For PCM, the crystalline state modulates conductance to similar effect.

The critical distinction is not the specific memory technology. It is that the computation happens where the data resides. There is no fetch. The energy per MAC drops from hundreds of picojoules to single-digit picojoules. The throughput per watt increases by one to two orders of magnitude. The latency collapses from memory-access-bound to analog-settling-bound, typically nanoseconds rather than microseconds.

Chinese IMC chip designers have focused on SRAM-based approaches for near-term deployment, leveraging existing foundry processes and design tools. ReRAM and PCM are in active development for higher-density, lower-leakage applications, but the maturity gap with SRAM remains significant. The pragmatic strategy has been to ship SRAM-IMC products now while incubating emerging memory technologies for next-generation scaling.

The domestic IMC ecosystem in China has crystallized around three distinct approaches, each reflecting different trade-offs between performance, programmability, and manufacturing risk.

Compute-in-Memory SRAM (CIM-SRAM) is the most mature path. Several Beijing and Shanghai startups have taped out chips that integrate standard SRAM macros with analog periphery for vector-matrix multiplication. The weights are stored in conventional 6T or 8T SRAM cells. The computation is performed by activating multiple word lines simultaneously and sensing the resulting bitline current with analog-to-digital converters. The precision is typically 4-bit to 8-bit for weights and activations, sufficient for most edge inference tasks and compatible with standard quantization workflows.

A representative product from this category is a 28nm chip that delivers 20 TOPS at 300 milliwatts for INT8 operations, with peak efficiency above 60 TOPS per watt. For comparison, a leading 7nm traditional NPU achieves roughly 10 TOPS per watt under similar workload conditions. The IMC chip is on an older, cheaper process node and delivers six times the energy efficiency. The performance per dollar advantage is compounded by the lower mask cost and higher yield of the mature node.

The programming model for these chips is deliberately familiar. They expose a tensor operation interface that maps onto the underlying analog array through software-managed tiling and scheduling. Frameworks like ONNX and TensorFlow Lite can target them through compiler backends that handle quantization, mapping, and calibration. The abstraction is not perfect—developers must be aware of precision limits, array size constraints, and analog noise characteristics—but it is close enough that model porting is measured in days rather than months.

Digital In-Memory Computing addresses the analog precision and noise concerns that limit pure analog IMC. In these architectures, the memory array is digital—standard SRAM or register files—but the access pattern is restructured to maximize data reuse and minimize movement. The computation is still digital, but the spatial proximity of memory and logic eliminates the off-chip and long on-chip wires that dominate energy in traditional designs.

A prominent example is the near-memory computing architecture developed by a Nanjing research group, where small digital MAC units are distributed throughout the SRAM array, each operating on a local tile of weights. The global movement is reduced to activation broadcasts and partial sum reductions. The energy efficiency gain is smaller than pure analog IMC—typically 5-10x over traditional NPUs rather than 50-100x—but the precision is full digital, the noise immunity is robust, and the design can be ported between foundry nodes with minimal analog customization.

This approach has found traction in applications where 8-bit quantization is insufficient: some medical imaging tasks, financial risk scoring, and scientific instrumentation. The digital IMC chip trades peak efficiency for generality and reliability, a trade-off that conservative industries prefer.

Emerging Memory IMC represents the long-term bet. Multiple Chinese research institutes and venture-backed startups are developing ReRAM and PCM arrays for vector-matrix multiplication, targeting the density and non-volatility advantages that these technologies promise. A ReRAM cell is smaller than an SRAM cell, does not leak power when idle, and can store multi-bit weights in a single device by exploiting the analog conductance range. The theoretical density advantage is 10-100x over SRAM.

The practical challenges are formidable. ReRAM devices suffer from cycle-to-cycle and device-to-device variability that complicates precise weight storage. Read noise is higher than SRAM. Endurance is limited to millions of cycles rather than the effectively infinite cycles of SRAM. Programming requires careful pulse shaping to achieve target conductance states. These are not insurmountable—analog IMC is inherently tolerant of some weight noise—but they require co-design between device physics, circuit architecture, and algorithm robustness that extends development timelines.

Chinese teams have made notable progress. A Tsinghua University group demonstrated a 1Mb ReRAM array in 2025 that achieved 1-bit binary neural network inference with acceptable accuracy degradation on standard benchmarks. A Shanghai startup taped out a 4-bit ReRAM IMC test chip in early 2026, targeting 2027 production for always-on voice wake-word applications where the non-volatility and zero-standby-power of ReRAM are decisive advantages.

The transition from traditional NPU to IMC is not a simple replacement. It is an architectural rethink that changes how edge AI systems are designed, programmed, and validated.

Traditional NPUs are programmable processors. They load models from external memory, execute layer by layer, and cache intermediate activations in on-chip SRAM. The programmer thinks in terms of operators, tensors, and memory hierarchies. The hardware abstracts the physical implementation.

IMC chips are not programmable in this sense. They are spatial accelerators where the physical layout of the memory array encodes the model structure. A convolutional layer maps onto a specific region of the array with specific tiling to match kernel dimensions and channel counts. A fully connected layer maps onto a different region with different aspect ratio. The mapping is static for inference—weights are loaded once and remain in place—but it is not transparent to the compiler. It is a physical design decision that constrains which models fit and how they are partitioned.

This changes the design workflow. In 2024, an edge AI engineer selected an NPU, quantized a model, and ran it. In 2026, the engineer co-designs the model with the hardware: selecting layer dimensions that tile efficiently onto the target array, adjusting channel counts to match array aspect ratios, and sometimes restructuring the network to exploit the analog compute primitive. The hardware is not a target. It is a constraint that shapes the model from the beginning.

The validation workflow changes equally. Traditional NPUs are deterministic digital machines. A given input produces a given output, every time, on every chip. IMC chips are analog systems with process variation, temperature sensitivity, and noise. Two chips from the same wafer may produce slightly different outputs for the same input. The difference is small—typically less than one percent in top-1 accuracy for well-designed systems—but it is real and must be characterized.

Chinese IMC vendors have developed statistical validation frameworks that treat the chip as a noisy channel rather than a deterministic function. Models are trained with noise injection to improve robustness. Chips are tested across process corners, voltage, and temperature to establish accuracy bounds. System designers specify acceptable accuracy ranges rather than exact values. The engineering culture shifts from digital correctness to statistical confidence, a shift that some traditional hardware engineers find uncomfortable but that analog designers have practiced for decades.

IMC manufacturing depends on foundry capabilities that are not universally available. Analog IMC requires precise control over memory cell characteristics, matching between cells in the same array, and specialized periphery circuits that are not standard IP. Emerging memory IMC requires even more specialized processes: ReRAM needs integrated selector devices and filament engineering, PCM needs phase-change material deposition and thermal management.

Global foundry leaders have IMC programs, but access for Chinese fabless designers is constrained by export control considerations. Domestic foundries—SMIC, Hua Hong, and emerging players—have accelerated IMC process development as a strategic priority. The logic is clear: IMC is a domain where China can establish competitive advantage without depending on the most advanced lithography nodes. A 40nm IMC chip can outperform a 7nm traditional NPU on relevant metrics. The node dependency is relaxed.

SMIC’s 28nm IMC platform, announced in late 2025, provides a calibrated SRAM bitcell with analog sensing periphery, supported by a design kit that includes noise models, variation characterization, and compiler integration. It is not as mature as TSMC’s equivalent offerings, but it is available without technology transfer restrictions. For Chinese IMC startups, this availability is decisive. It enables product development and revenue generation on a domestic supply chain, with a path to more advanced nodes as they become accessible.

The emerging memory foundry picture is more fragmented. ReRAM and PCM processes are still in development at domestic foundries, with limited capacity and yield learning. Chinese IMC designers pursuing these technologies have adopted a hybrid strategy: develop and validate on international foundry multi-project wafers, then transfer to domestic production when the process matures. The timeline risk is real, but the strategic commitment to domestic manufacturing is explicit.

Traditional NPUs inherit a software ecosystem: frameworks, compilers, profilers, debuggers, and optimization tools that have been refined over years of cloud and mobile deployment. IMC chips cannot use this ecosystem directly because their execution model is fundamentally different.

Chinese IMC vendors have invested heavily in toolchain development, recognizing that hardware without software is inert. The stack typically comprises three layers: a model representation layer that accepts standard formats (ONNX, TorchScript), a mapping layer that partitions and places the model onto the physical array, and a runtime layer that handles inference scheduling, calibration, and noise compensation.

The mapping layer is where most of the IMC-specific complexity lives. It must solve a spatial packing problem: given a model graph and a fixed-size memory array, how do you assign layers to regions, route activations between regions, and schedule partial computations to respect array bandwidth constraints? This is NP-hard in general, but heuristic solvers can find good solutions for typical network architectures. The Chinese toolchain vendors have developed proprietary mapping algorithms that exploit the regular structure of convolutional and transformer layers to achieve near-optimal utilization.

The calibration layer addresses analog non-ideality. Each IMC chip has unique noise and offset characteristics that must be measured and compensated. The toolchain automates this: it runs a calibration sequence during manufacturing test, stores the calibration parameters in on-chip non-volatile memory, and applies them during inference through digital pre-distortion and post-correction. The user sees a calibrated, deterministic interface. The complexity is hidden in the tooling.

The result is that a developer with standard deep learning experience can target a Chinese IMC chip without learning analog circuit design. The abstraction is not perfect—developers must understand precision limits, array capacity, and the performance implications of model structure—but it is close enough that porting a model from GPU to IMC is comparable in effort to porting between mobile NPUs.

Not every edge AI application benefits from IMC. The performance advantage is concentrated in workloads with specific characteristics: high weight reuse, moderate precision requirements, and severe power constraints. Chinese IMC deployment has focused on these niches rather than attempting to displace traditional NPUs across the board.

Always-on sensing is the clearest fit. Voice wake-word detection, visual motion detection, and biometric monitoring require continuous inference on battery-powered devices. The inference is lightweight—a few million operations per second—but it must run forever without draining the battery. A traditional NPU in always-on mode consumes milliwatts of standby power and tens of milliwatts of active power. An IMC implementation with SRAM arrays can operate at sub-milliwatt levels, with near-zero standby if non-volatile emerging memory is used. Chinese smartwatch and earbud manufacturers have adopted IMC for always-on health monitoring, extending battery life from hours to days.

High-frame-rate vision is another strong fit. Industrial inspection lines running at thousands of parts per minute generate image data faster than traditional NPUs can process within latency and power budgets. IMC chips can process each frame in microseconds rather than milliseconds, enabling real-time rejection of defective parts without buffering or line slowdown. Chinese automotive component suppliers have deployed IMC-based inspection at welding and assembly stations where cloud or even edge server inference is infeasible.

Multi-modal fusion in robotics and autonomous systems benefits from IMC’s ability to perform parallel inference across heterogeneous sensor streams. A warehouse robot fusing lidar point clouds, camera images, and tactile sensor arrays must run multiple models simultaneously. Traditional NPUs serialize these workloads or require multiple chips. A single IMC array can be spatially partitioned to run all models in parallel, with activations routed between partitions on-chip rather than through external memory. Chinese autonomous vehicle startups are evaluating IMC for sensor fusion in low-speed urban scenarios where the computational density of highway driving is relaxed but the sensor diversity is higher.

Neural network training at the edge, while still experimental, is a frontier application where IMC’s energy efficiency advantage is decisive. Federated learning and continual adaptation require on-device gradient computation, which is far more memory-intensive than inference. IMC arrays can perform forward and backward passes with the same spatial locality benefits, reducing the energy per gradient update by orders of magnitude. Chinese research groups have demonstrated proof-of-concept edge training on IMC hardware, with practical deployment expected in 2027-2028 for applications like personalized health models and adaptive industrial control.

IMC chips introduce failure modes that digital NPUs do not. Analog noise can cause intermittent errors that are difficult to reproduce. Process variation can create chip-to-chip accuracy differences that complicate fleet management. Temperature drift can shift calibration and degrade performance in thermally challenging environments.

Chinese IMC vendors have responded with a multi-layered reliability strategy. At the device level, they employ redundancy and error correction: multiple cells per weight, majority voting for critical decisions, and digital fallback paths for operations that exceed analog precision. At the chip level, they include temperature sensors and dynamic recalibration engines that adjust for thermal drift in real time. At the system level, they specify operating envelopes and accuracy bounds that system designers must respect, with explicit graceful degradation when boundaries are approached.

The most sophisticated implementations include runtime monitoring that tracks inference confidence and triggers digital fallback when analog uncertainty exceeds thresholds. A voice wake-word system might run on IMC under normal conditions but switch to a digital core if the confidence score drops below a threshold, sacrificing efficiency for reliability. The system designer specifies the trade-off. The hardware implements it.

This is not a solved problem. It is an active area of engineering development. Chinese IMC teams publish regularly on noise-aware training, variation-tolerant architectures, and self-calibrating circuits. The field is moving from “does it work in the lab” to “does it work in a million units across five years of field deployment.” The transition is where the technology proves itself or fails.

The long-term viability of Chinese IMC depends not on any single chip but on the ecosystem that grows around it. This includes foundry processes, design tools, application libraries, developer training, and system integration expertise. The domestic investment in this ecosystem is substantial and accelerating.

Foundry investment is the foundation. SMIC and Hua Hong have dedicated IMC process development tracks with multi-year roadmaps. Design tool vendors are integrating IMC-specific mapping and simulation into their standard flows. Universities are adding IMC courses to VLSI curricula. System companies are building products around IMC chips and feeding requirements back to the semiconductor vendors.

The feedback loop is critical. Early IMC chips were developed in isolation, with limited understanding of real application requirements. As system companies deploy and report back, chip designers learn which precision levels matter, which array sizes are useful, and which periphery features reduce system cost. The second-generation Chinese IMC chips reflect this feedback: larger arrays for transformer layers, lower-noise ADCs for high-dynamic-range signals, and better power gating for intermittent workloads.

The ecosystem also includes the standards and regulatory frameworks that enable deployment. Chinese industry standards bodies are developing test methodologies for IMC chip accuracy, power measurement, and reliability qualification. These standards are necessary for procurement in state-influenced sectors like rail, power, and defense, where component qualification is mandatory. The standards development is itself a signal of institutional commitment to IMC as a strategic technology category.

The title of this piece frames IMC as the technology that outgrew traditional NPUs. The framing is accurate but incomplete. Traditional NPUs did not stop improving. They continue to advance on finer process nodes, with better memory hierarchies, and more efficient dataflow architectures. But they are improving within a paradigm that has fundamental physical limits. IMC is a different paradigm.

The outgrowth is not about performance alone. It is about the match between technology and application requirements. Edge AI in China is not a single market. It is a collection of niches with diverging constraints: some need absolute lowest power, some need absolute lowest latency, some need programmability, some need reliability certification. Traditional NPUs serve the center of this distribution well. IMC serves the tails: the applications where memory movement is the binding constraint and where the von Neumann architecture is not merely inefficient but structurally incapable.

Chinese semiconductor strategy has recognized this bifurcation. It is not abandoning traditional NPU development. It is pursuing both paths in parallel, with IMC as the high-risk, high-reward bet on the next decade of edge AI. The bet is that as edge AI models grow and edge power budgets shrink, the memory wall will become the defining constraint for an increasing fraction of applications. When that happens, the chips that compute in memory will be the chips that define what edge AI can do.

The transition is already visible in product roadmaps. Major Chinese edge AI chip vendors have announced IMC as a core technology direction, with first-generation products shipping and second-generation products in design. The engineering talent is migrating: analog and memory designers who previously worked on commodity DRAM and SRAM are now applying their expertise to compute-in-memory arrays. The academic output is growing: Chinese institutions now publish a significant fraction of global IMC research, with citation rates that reflect practical relevance rather than theoretical novelty.

The question for 2027 and beyond is not whether IMC will compete with traditional NPUs. It already does in specific applications. The question is how broad that competition becomes, and whether the ecosystem maturity that Chinese vendors are building can support mass deployment across the full range of edge AI markets. If the reliability, programmability, and manufacturing challenges are solved, IMC could become the default architecture for power-constrained inference. If they are not, IMC will remain a niche technology for always-on and high-throughput applications while traditional NPUs continue to dominate the mainstream.

Either outcome is possible. What is certain is that Chinese IMC development is no longer an academic curiosity or a startup experiment. It is a strategic industrial effort with foundry investment, government support, and market pull. The chips that compute in memory are being built in volume, deployed in products, and validated in field conditions. The outgrowth of traditional NPUs by IMC is not a forecast. It is a process already underway. Google’s Billion-Dollar EU Reckoning | Maya