Compute is the lifeblood of AI. He who controls the spice controls the universe the compute will control the production of tokens and reap the benefits of AI. Without compute you do not have a seat at the table. The United States technology community is all in on compute and AI as the next platform and is now adding compute at a staggering pace.

There is competition, and it not only comes from companies but from countries, and the US government has placed a series of export controls to limit China’s rising compute. Today the US controls and is the undisputed leader in compute with more than 70% of the worlds deployed FLOPs. One way to stay ahead is to keep going full steam while hindering your competition. Limiting your competitor nation state from compute, which will limit them from intelligence, is the current policy to stay ahead in the AI race.

These moves have led to backlash, including with China cutting rare earth minerals and magnets off from the US. Secretary of Commerce Howard Lutnick says the resumption of Nvidia GPU sales to China were required to restart China’s shipments of their linchpin supply chain materials.

But constraints have also led to adaptation, and Chinese companies have adapted. High batch sizes and disaggregated serving are but two examples. Despite advances, in the case of DeepSeek, most of their tokens are still inferenced on western hardware. We wrote about this dynamic in our recent DeepSeek debrief. Training of DeepSeek’s next generation model was also delayed by the use of Huawei chips, as we also said in the debrief.

This is not a stable equilibrium. There are always moving pieces in the race for intelligence. Beijing plans for the long term and knows it must secure its own domestic compute destiny. There is an irony: in the 2010s China kicked out Google to enforce its Great Firewall and foster its domestic industry, this time the US government is withholding hardware technology so they cannot seize the lead in AI.

We believe that at China’s core, they want to control not only its internet and AI, but the hardware that supports it. From silicon to tokens, China seeks sovereignty over every layer of the stack, and given recent history will never want to be beholden to foreign powers. Enter Huawei.

China loves national champions, and in it’s characteristic capitalism, tends to funnel resources to a few national champions. Today’s champion is probably Nvidia’s greatest adversary: Huawei. We expect Huawei to be able to make millions of chips this year, and to be bottlenecked by HBM next year. Today we want to talk about Huawei, which can translate as “China’s achievement”.

All In with Huawei’s chips

Huawei is the key piece for China’s compute destiny. Huawei’s chip ecosystem is vertically integrated and capable network of tools, fabs, and design that allows it to express a full stack vision of hardware. This hardware is very impressive although less efficient then western hardware.

Our previous investigation into Huawei’s fab network is a great example of their broad reaching network.

Huawei wants to vertically integrate the entire manufacturing process. The goal is to own not just the manufacturing of the logic die, the brains of the chip, but also the memory and packaging. They have even created their own tool company, SiCarrier to copy foreign firms tools. Huawei has bought over $9B of tools to put in their own fabs as well as reverse engineer to replicate.

Their efforts are not to be underestimated. SiCarrier, for example, has recently raised $2.8B in funding. That money is going into building fabs dedicated for Huawei staffed by Huawei employees. Some reporting on these fabs, which we believe to be Huawei owned and operated, suggests that their combined production by next year could entirely exceed SMIC, the current leader and where production is currently outsourced. With Huawei ramping up their own efforts, allocation at SMIC can be freed up for other chips, including Cambricon. The Cambricon chip is itself popular among Chinese companies, especially ByteDance.

Huawei operating their own fabs will represent a material increase in Chinese production, but also in their ability to iterate, control, and improve on processes. Huawei and SMIC will directly work on increasing yields, refining R&D for the next node, and bolstering Chinese semiconductor manufacturing capability.

Currently, all high volume chip production is outsourced to SMIC, the leading Chinese pure-play foundry. This includes the Ascend series of accelerator chips along with the Kirin mobile processors. Yields are poor for SMIC’s 7 nm-class processes due to a combination of immaturity, export controls, and the inherent difficulty in yielding large die such as the Ascend. Thus a relatively low percentage of SMIC’s overall capacity is allocated to producing Ascend die, since smaller mobile processors just make better business sense at this point. But this can change quickly. Let’s discuss what is possible for Huawei….

Huawei Production Numbers and SMIC’s Ramp

Our data shows Huawei’s production is 507k Ascend units shipped in 2024, the majority of which are 910Bs, and 805k this year, 653k of those being 910C. The 910C is the more advanced version. This includes die made by TSMC and SMIC.

SMIC, hampered by export controls, struggled to get production off the ground. But Huawei got Ascend die fabricated at TSMC while SMIC ramped. This was a violation of export controls and Huawei ended up receiving more than 2.9M Ascend die, which can be used for both 910B and 910C. We detailed this here.

It is specifically this “Die Bank” of foreign chips from TSMC that gets them through 2024 and 2025. Without this Die Bank, Huawei’s Ascend production numbers would be much lower.

We expect that the TSMC die bank to run out within the next 9 months. SMIC, however, now has more than enough capacity to produce meaningful volumes of chips. We forecast that SMIC will no longer be the bottleneck for Ascend production, as they will have sufficiently ramped capacity by end of year.

The chart above shows a baseline scenario with a modest increase in SMIC capacity allocation for Ascend. It requires at most 20k wafers per month (wspm) of SMIC capacity to produce millions of Ascend die per month.

For reference, a conservative estimate of SMIC’s total advanced node capacity (7nm and below) is 45k wspm by end of 2025, increasing to 60k wspm in 2026 and 80k wspm in 2027. In addition, Huawei is building their own fabs, not all of which are export controlled, and collaborating with SMIC on process technology, so production could ramp higher for advanced process technologies.

If 100% of capacity was allocated to Ascend die, their production capability would be in the tens of millions per year. They are on track to be more than capable of supporting a large domestic demand for China-produced compute die.

Our forecasts above use a conservative estimate for both yield and its rate of improvement in the future. It’s very likely that SMIC can exceed these estimated as its 7nm nodes mature. Our estimates for yields are lower than TSMC, Intel, Samsung, ASE, Amkor, etc for front end wafers and packaging.

Indeed, yield is an important lever that can be pulled to increase production without giving up more allocation. With just small increases in yield beyond our forecast above, SMIC will be able to produce several million Ascend die at lower allocation amounts than they would otherwise. Every percentage counts and SMIC has their best engineers working on exactly this problem.

In other words, SMIC only needs small amounts, as low as single digit percentages, of allocation to produce more than a million die as soon as early next year. Production to a couple million is possible, more allocation is all that is needed. For the reasons we articulate above, we believe the reported number of 200k Ascend chips to be significantly off the mark.

While SMIC is expanding and is no longer the bottleneck and Ascend production, Huawei’s long-term ambitions include its own fab network buildout. We noted in our Fab Whack-A-Mole report:

Huawei is clearly taking full advantage to the tune of $7.3B of WFE expenditure in 2024, up 27% year-on-year. They’ve gone from effectively zero in 2022 to the 4^th largest WFE customer globally in two years.

Since we published that report in fall of 2024, the Huawei fab buildout has, if anything, accelerated. Their ecosystem of semiconductor-related shell companies is expanding. Massive new cleanroom buildouts are in progress. And improved domestic options and possible diversion of $30B+ of imports of wafer fabrication equipment means they are likely able to equip these fabs as well. The details are worthy of their own report, but suffice to say Huawei continues with a large, concentrated effort to own every vertical in the Ascend supply chain.

TSMC Access

Huawei currently makes many mobile chips, but there is zero reason to do so from a strategic geopolitical perspective. Oppo and Xiaomi currently fabricate mobile SoCs at TSMC. They are ramping up their own designs to decrease dependence on companies like MediaTek and Qualcomm.

Huawei can reduce their mobile SoC production while not consuming, in any meaningful volume, SMIC allocation. The continued access to TSMC for other Chinese entities decreases the pressure for SMIC to make mobile SoCs, meaning more can be allocated to AI chips.

Huawei and SMIC can ramp up AI production without having to worry about serving all the nation’s demand for mobile chips due to other companies’ access to TSMC.

Export Control Lag Benefits SMIC

Another part of SMIC’s expansion strategy is stockpiling considerable amounts of semiconductor tooling. The controls are released on an announced schedule and it’s usually easy to anticipate what will be included.

This is due to timing differences in controls that they can take advantage of. For example, the U.S. routinely exempts Japanese and Dutch companies from its equipment export controls. Ostensibly this is done because they are allied countries and have their own export control regimes and semiconductor industries.

The problem, however, is that when new export controls come out, Japan and the Netherlands do not immediately follow. In many instances, matching controls are delayed by 6 months or never. Chinese companies can rush order years’ worth of equipment to stockpile, while American vendors are shut out of the process.   Japanese WFE vendors are happy to sell into this gap with margins fattened by order expedites. Many key suppliers are seeing well above 40% revenue share from China:

In cases where controls do follow the US, Japan and the Netherlands do not have any matching controls on re-exporting. This means that it is possible for restricted tooling to reach China if it goes through a different country first.

In addition, the use of “advanced ICs” as the bar for restricting tools leaves room for issues. ASML’s NXT:1980 scanners are perfectly capable of 7nm-class logic, and likely beyond if economics (throughput and yield) can be ignored or subsidized. These are allowed into China and even to certain SMIC facilities.

Export controls, as noted, can be expanded in addition to very stringent enforcement of existing mechanisms. This includes tighter coordination with allies on timelines for matching controls and getting buy in on re-exporting.

We are broadly encouraged by the administration’s Action Plan. As an example, we are pleased to see controls on semiconductor subsystems being called out. This is a drum SemiAnalysis has been beating for a while and think it is the right direction. With that said, many subsystem firms that supply western players such as VAT Group in countries like Switzerland without controls will not be stopped from shipping critical chambers to China.

The Action Plan also included concrete suggestions around aligning protection measures globally, which we think if implemented, will greatly ameliorate the issue of alliance’s lags in export controls described above. The issue is that SMIC and CXMT are able to continue to expand production because the sanctions on them are flimsy. International co-operation, especially with partners in Korea, is key to get right and ensure that Chinese commercial bottlenecks are not eased. Korea manufactures a huge amount of memory and Samsung has historically supplied large amounts of memory into China, so tight alignment of goals and enforcement is critical.

The other part of the supply chain that is just as critical is memory. This is where we believe the key constraint lies.

HBM Is the Bottleneck

We believe HBM production is the bottleneck. China does too, which is why they have asked US officials to relax controls on HBM as part of the recent trade talks. What the ask omits is telling: it does not include more TSMC access or lithography tools. Beijing is specifically asking Washington to loosen HBM restrictions.

Much like how Huawei was able to stockpile TSMC logic wafer inventory, they were also able to stockpile HBM inventory. Samsung, due to their failures in entering the accelerator supply chain for Western chips, sold their product to Chinese customers that funneled this inventory to Huawei. This is why it is so critical to work with Korea on enforcement of memory controls moving forward.

Samsung alone has directly provided 11.4 million stacks of HBM to China, including a staggering 7M stacks in the 1-month gap between controls announcement and enforcement dates. When including other providers and methods of shipment, that is 13 million stacks of HBM.

Specifically, on December 2^nd 2024, the Bureau of Industry and Security (BIS) announced controls on anything more advanced than HBM2E. Full compliance was required on December 31^st 2024. Samsung exported as much as possible to China in that one quarter. This comprises the majority of China’s HBM. They were able to achieve this due to the US Government and media telecasting the restrictions for many months before they came out.

After the ruling, Samsung HBM still made it into China. We previously detailed how companies like CoAsia Electronics and Faraday supplied non-functional chips with HBM into China, though we believe due to our efforts in exposing this both privately in late January and publicly later that it has now stopped with revenue numbers returning to normal. This does not mean that HBM smuggling has stopped entirely, though. There could be other sources.

In sum, China has procured 13M HBM stacks which is sufficient for 1.6M Ascend 910C packages. Despite this, we expect that China will be bottlenecked by HBM by the end of the year as they run out of foreign HBM.

China can easily make more than 805k Huawei Ascends this year from TSMC and SMIC capacity, but they will not because they do not have enough HBM.

We expect SMIC’s production to be 1M 910Cs and almost half a million 910Bs this year, however, not all of them will be turned into ASICs as the HBM is not available. If some HBM enters through smuggling, then Huawei can produce more Ascend AI ASICs.

Without Foreign HBM, China has no Domestic AI Accelerator Industry

Without access to more foreign HBM, Huawei will not be able to fabricate even 1 million Huawei Ascend chips next year. They must rely entirely on domestic production which we will detail below. Nvidia and AMD have effectively 0 competition in China once these HBM banks run out.

The other option available to China is utilize slower GDDR and LPDDR memory, but this is not suitable for the leading language models with modern reinforcement learning techniques or for large scale inference deployments.

Domestic HBM Industry – CXMT

China’s main DRAM player, CXMT, has caught up with the west rapidly. This is due to a combination of extremely strong domestic engineering capabilities, poaching engineers from Samsung, SK Hynix, and Micron, as well as the leading tool vendors Applied Materials, Lam Research, and Tokyo Electron teaching them sub-processes.

CXMT is able to ship DDR5 memory, only a couple years behind SK Hynix, Micron, and Samsung, and is winding down profitable DDR4 production previously reserved for PCs and mobile. While they have not shipped much HBM, their roadmap is aggressive. By next year, their production capacity will rival that of Micron’s, though not fast enough to save Huawei Ascend’s production. In 2026, we expect them to be producing 257k WPM, which would be just under 15% of global DRAM production. In our estimates, this scales to 490k in 2030.

CXMT’s shift in focus and production ramp are driven in part by investments from the CCP. China’s “Big Fund III”, which started in May 2024, invested $2B into the company. CXMT is also expanding in Shanghai as well as Beijing, with HBM packaging subsidiaries being set up in the former.

Expecting, though ultimately averting, an entity listing by USG, CXMT stockpiled years of tools in 2024 and is likely still adding HBM-specific gear. CXMT is still not entity listed! While HBM was noted as a key focus in that salvo of controls, the previous administration failed to include China’s HBM champion. The Trump administration needs to solve this failure from the Biden administration immediately.

The stockpiling matters as advanced tools, like Hanmi’s TCB systems for HBM3, are restricted, but they are not that instrumental. Older systems can be run slower without impacting cost too much. More importantly, CXMT can still procure leading edge equipment for Through Silicon Via (TSV) formation – that is critical for making HBM -through Japanese suppliers. In addition, Chinese OSATs such as JCET and Tong Fu are also racing ahead with their R&D efforts and building capacity for the critical TSV and stacking processes to package front end HBM wafers from CXMT. This is unlike the Western memory incumbents who have vertically integrated these processes. This is a typical example of Chinese industrial development where multiple players are encouraged to develop domestic manufacturing capabilities: creating cutthroat competition that accelerates the speed of development. It is therefore critical to not just focus controls on CXMT, but the whole of China as critical process steps can always be outsourced.

Because front-end logic is currently not the binding constraint, Beijing’s main asks in the trade deal target HBM and the relaxation of controls relating to this set of tooling. Given the importance of HBM, it is critical to understand CXMT’s future production depending on various allocation scenarios.

CXMT Production Forecast

This can play out in several ways, depending on how much wafer capacity is given to HBM. China can easily produce tens of millions of stacks with less than 50% of wafer capacity. CXMT is at a bit over ~250,000 wafers a month of production capacity and is expected to reach 300,000 wafers by the end of the year.

Currently they have not built out the tooling required to convert standard DRAM production lines to HBM, but it is inevitable. Properly designed, targeted sanctions can slow this conversion massively.

Different scenarios lead to different amounts of Huawei Ascends produced. As noted, SMIC can produce the die needed to match the HBM.

To be clear, this could change. The rate of production for CXMT could increase if they continue stockpiling key tooling or significantly improve yields. The might of China’s manufacturing capability and capacity knows no bounds. This estimate is somewhat conservative, it is likely that CXMT is able to produce the significantly more capable HBM3e in 2026.

We believe CXMT will only be able to make ~2 million stacks of HBM next year, which is only sufficient for 250,000-300,000 Ascend 910C’s. Yields and capacity conversion will take some time to improve for CXMT to commit significant capacity.

If all leading edge logic die capacity could be paired with HBM, Huawei production of Ascend would grow from 805k this year to 1,175k in 2025. More importantly, production next year would grow from 300k to over 5 million Ascend 910C!

Our analysis indicates that export controls have been effective in constraining and limiting Chinese chip production capabilities. Assuming no smuggling, China will be able to make less Ascends next year, not more. CXMT is squeezed tight. Had controls not been present, the Ascend ramp would be fully realized, Chinese models would be served on Huawei Ascend at scale, and there would be enough compute capacity that advanced models like DeepSeek R2 and V4 would already be here. Not to mention, with more capacity, China would be better positioned to export their AI on their chips.

As such, it is absolutely critical to ensure the application, enforcement, and continued updating of export controls to prevent CXMT and related entities from ramping production. As mentioned, this includes not just CXMT but OSATs and subsidiaries they work with. Second, it is also important for the intelligence community to track and identify any instances of HBM smuggling, like the Faraday + CoAsia scheme which we disclosed privately in January and publicly reported on later.

By no means should HBM be allowed to be shipped into China. Production of AI chips over the next few years is heavily gated by CXMT’s ramp and foreign shipments of HBM.

There is very strong incentive to find ways to ship HBM into China. This is why enforcement of controls are so critical.

There is another pillar to China’s production strategy, separate from memory and logic. This pillar is about networking the chips together and where they are made.

Networking and Datacenter CPU

Chips do not exist in isolation. We have argued that the system matters more than the microarchitecture for years. Clusters are comprised of many tens of thousands of interconnected chips and how they are interconnected matters.

We detailed Huawei’s CloudMatrix 384, or CM384, system below.

We believe the networking equipment used, specifically the scale up switches, are being made at TSMC and not at SMIC through shell companies. We also think they are stockpiling this equipment.

We believe Huawei has also been able to manufacture their datacenter CPU at TSMC. This showcases that current controls aimed to limit Huawei’s access to TSMC are insufficient. The Ascend AI ASIC is 7nm, but because of poor screening, Huawei has been able to procure technology from the more advanced TSMC 5nm node.

By producing these chips at TSMC, this frees up capacity at SMIC, meaning more allocation can be given to the Ascend or Cambricon programs (HBM permitting). This, as mentioned earlier in the report, is a critical variable. If SMIC had to make the networking equipment, this would squeeze the wafer allocations towards Ascend significantly.

There is all the incentive in the world for China to offload as much production as possible to TSMC. They have used shell companies before and there is ample evidence they are still doing it. While some ideas around KYC and red flags have been implemented, we believe that existing frameworks are insufficient.

Nvidia and Huawei: Blackwell and the H20

The US government previously banned the Nvidia H20 chip from going into China, but recently indicated that Nvidia will be granted a license to be able to export the chip. As we have previously noted, the H20 is a capable chip with more memory than the banned the H100, though it has less much fewer FLOPs. Having more memory is helpful for inference performance.

Nvidia is poised to sell its existing inventory of the H20 and H20E (a variant with even more memory) at a minimum. This will amount to more than several hundreds of thousands of chips and billions of dollars of revenue.

There have also been reports of a more advanced version based on the Blackwell series. This is slated to have just as much memory as the H20 but will also contain significantly more FLOPs. Specifically, the B30A could have more than 10x the FLOPs than the H20, significantly above the export control threshold. The B30A could be positioned at half the price and have half the performance of the B300. The solution to this is rather simple: just buy 2x the chips for equivalent performance. Given how strong and weak scaling work in machine learning workloads, this is a valid path to remain competitve.

The B30A is argued as needed to be shipped to China due to decreased interest from China in the H20. A lot of this is due to government pressure of China’s major tech firms. While there is significant messaging that China has decreased interest in the H20, we do not believe this to be true.

Specifically, we do not believe this is reflective of China’s demand for compute or foreign chips. It is an incorrect policy being pushed from the top down which will be reversed as soon as Huawei runs out of HBM to produce more Ascend chips, or it could even be orchestrated brinkmanship to get approval to a more powerful chip.

This is on the tail of other news, like DeepSeek unable to get acceptable performance on Huawei’s chips. That is no coincidence. Export controls are working and the biggest blocker for more progress is compute.

China may be trying to psyop their way into getting the significantly more performant Blackwell chip. While the H20 and the H20E have much, much better software than the 910C, it is arguable that they are in the same league. However, it is unquestionable that the B30A would be in a league of its own. Given that the H20, H20E, and especially the B30A are good at inference, this also allows Chinese companies more compute to serve their models and applications.

It will allow them to export Chinese AI on hardware they own. This will increase the proliferation of Chinese AI and Chinese apps, inevitably taking market share away from American applications. The American AI stack is American AI on American chips, not just the chips.

The US needs to balance having China on the US AI stack and slowing the development of their own, while also limiting the quality and volumes that are shipped into China.

The H20 is allowed into China, but the progression of shipping more powerful chips into China should be closely guarded and watched considering China’s domestic capabilities.

The US should only move the quality of a chip shipped to China when it becomes clear China can ship something competitive to the H20E in significant volumes.

Regardless of the exact SKU, the export licenses have several implications discussed below.

Implications of Compute Diplomacy

First, it means that there will be more chips for top players like DeepSeek and Alibaba to use for Reinforcement Learning (RL). RL is a big driver of progress right now. The majority of RL compute is inference, which the H20 and especially the H20E, are good at. The B30A will have even better performance.

Players like DeepSeek had enough compute to keep improving – R1 received a major update with performance increases on a timeline that is like o1 -> o3. We firmly believe GPT-5, Claude 4, and Grok 4 are considerably ahead of R1 though, especially in agentic tasks.

However, problems in AI are solvable by two things: talent and compute. DeepSeek has always had the former, now they will have more of the latter. We expect DeepSeek’s rate of progress to materially increase as they receive substantial volumes of any chip that gets into China. DeepSeek has ambitions to release a multimodal model in V4, but scarce compute is slowing progress.

Alibaba (Qwen) and Moonshot (Kimi K2), have also yet to produce a large multimodal model, primarily focusing on text only due to compute limitations. They will release multimodal models this year, but they are still squarely behind OpenAI, Anthropic, and Google in many capabilties. If Blackwell GPU shipments come too early, that will accelerate China’s pace.

The second major implication of this is that China will now have more compute to serve and inference models to its population. Compute constraints affect user experience, with DeepSeek deliberately serving R1 at low speeds to users to preserve compute.

Poor user experience from the lack of compute has greatly limited China’s ability to deploy AI. As user experience improves, so will adoption, and the economic benefits from AI. It is easy see a future in which models are pre-trained via H800s, post-trained on H20s, then served to the population on Ascends and H20s. There is so much demand for AI powered products that should more capacity to serve these models exist, it would immediately be satisfied. One of the reasons Chinese models are open sourced is that it allows for other people to serve their models for them. With more capacity, models can be closed sourced and dependence on American providers cut off.

Claims that models trained on one chip must be inferenced on the same type are entirely false. Anthropic did different stages of research and training for Claude 4 on GPUs and TPUs. Additionally, Claude 4 inference is offered on Nvidia GPUs, Google TPUs, and Amazon Trainiums. This complexity is all while Anthropic is also the fastest growing AI company in terms of revenue with by far the most capable model for software engineering.

DeepSeek, Alibaba, Moonshot, etc primarily train their models on Nvidia chips and we do not think this will change anytime soon. If the Blackwell version ships, gains across all levels will be more pronounced.

China’s compute at a snapshot

With the H20 approved, this represents a material increase in the number of FLOPs and memory that China has access to. The 910C will be the first meaningful effort of indigenous production in China in terms of realized FLOPs and memory. We also expect some re-exportation into China by various bad actors, constituting meaningful volumes of H100 and low volumes of B200.

Should the Blackwell version ship, gains around FLOPs will be even more pronounced as the chip has significantly more FLOPs. China will end up having more FLOPs in that scenario, in addition to more memory.

This is not to mention that Chinese players, including major ones like ByteDance, can still access compute through renting in non-restricted countries. For example, ByteDance can still access top of the line Blackwell GPUs from providers like Oracle and Google. We have previously covered this relationship in detail. Malaysia, in no uncertain terms, has become a huge market for Nvidia. Export controls were also effective in ensuring that it remains that way, with Malaysia immediately distancing themselves from any effort around the Huawei Ascend.

Renting enables a path for Chinese dependence on Nvidia without Nvidia chips being in China, which is possible because the chips do not need to be in China. Bytedance’s Seed models were trained in the US on a US cloud.

Inspecting GPUs is much easier in Malaysia than in China. As such, it is an effective way to limit any unauthorized re-exporting by bad actors, as we believe methods like location tracking are technically intractable and easy to bypass. Note that renting will still allow China access to the capability of training advances models. If allowed to be used for serving the models, this allows China the chance to proliferate their AI through new and improved AI powered applications, taking market share away from American products. The critical difference, though, is that access can be cut off.

With that said, China will not allow their data to be shipped out of China in large volumes hence there is still significant need for domestic AI capabilities. Selling chips into China does not change China’s drive for total silicon self sovereignty and will only serve to help provide a buffer while local production meets demand.

It is worth remembering that China’s focus on silicon self sovereignty predates US export controls. The argument Blackwell needs to be sold into China is a false narrative as Huawei will soon run out of HBM. Selling chips into China only served to bail them out until domestic production ramps. The production ramp will not slow until self sufficiency is achieved. Shipments of Blackwell must be weighed with a close eye on production of domestic capabilties. If Huawei, Cambricon, and CXMT accelerate production faster than expected, then the US should raise the bar sooner.

Next, we dive into Nvidia’s future in China. This includes expected revenue from the H20 and newer China specific chips on the horizon. We also compare how much compute China has relative to the US how the is absolutely dominating China when it comes to both memory and FLOPs.

The first portion of this report will explain HBM, the manufacturing process, dynamics between vendors, KVCache offload, disaggregated prefill decode, and wide / high-rank EP. The rest of the report will dive deeply into the future of HBM. We will cover the revolutionary change coming to HBM4 with custom base dies for HBM, what various different accelerators are doing with custom HBM including OpenAI, Nvidia, and AMD, the shoreline area problem, memory controller offload, repeater PHYs, LPDDR + HBM combos, and various beachfront expansion techniques. We will also discuss SRAM tags, compute under memory, supply chain implications, and Samsung.

A Brief Overview of HBM

As AI models grow in complexity, AI systems require memory with higher capacity, lower latency, higher bandwidth, and improved energy efficiency. Different forms of memory have different tradeoffs. SRAM is extremely fast but low density. DDR DRAM is high density and cheap but lacks bandwidth. The most popular memory today is on-chip HBM which strikes the balance between capacity and bandwidth.

HBM combines vertically stacked DRAM chips with ultra-wide data paths and has the optimal balance of bandwidth, density, and energy consumption for AI workloads. HBM is much more expensive to produce and has a warranted price premium to DDR5, but demand remains strong for HBM. All leading AI accelerators deployed for GenAI training and inference use HBM. The common trend across accelerator roadmaps is to scale memory capacity and bandwidth per chip by adding more stacks, higher layer counts, with faster generations of HBM. Architectures that rely on other forms of memory offer sub-optimal performance, as we have demonstrated.

In this report, we will examine HBM’s present state, what’s happening in the supply chain, and the groundbreaking changes happening in the future. We’ll examine HBM’s critical role in AI accelerator architecture, the impact HBM is having on the DRAM market, and why it is upending the way memory market analysis is being performed. For subscribers, we will also address the major questions on Samsung’s future viability as a supplier, as well as highlight one technological change that may reverse the trend of increasing HBM capacity.

HBM Primer

First, a brief primer on HBM – what makes it special and challenging to manufacture. While HBM is commonly associated with multiple DRAM dies stacked in a 3DIC assembly, the other key feature is HBM’s much wider data bus, improving bandwidth even with mediocre signaling speeds. This significantly wider bus results in HBM being far superior in terms of bandwidth per package than any other form of memory.

The implication of having much more I/O is increased routing density and complexity. Each I/O requires an individual wire/trace, with additional wiring required for power and control. For a HBM3E stack, there are over a 1,000 wires between the adjacent XPU and the HBM. This level of routing density is not achievable on a PCB or package substrate; therefore, an interposer (silicon or organic) in a 2.5D package assembly like CoWoS is required.

To reduce latency and energy consumption for data transfer, HBM needs to be placed directly adjacent to the shoreline of the compute engine. This makes shoreline (the edge of a SOC) more valuable as HBM can only be limited to the 2 edges of the SOC with the other 2 edges reserved for I/O off the package. This limits the area HBM can be placed in and requires vertical stacking of memory die to provide enough capacity.

To enable the 3DIC form factor, each layer of the stack needs to have TSVs (excluding the top of stack) that can deliver power and signal to the layer above. The additional area required to fit these TSVs is what makes HBM die sizes larger than their DDR equivalent: SK Hynix D1z DDR4 has a bit density of 0.296 Gb/mm², 85% more dense than their HBM3 which is 0.16 Gb/mm². This TSV process is one of the key differences between standard DRAM, and tooling for this is the main bottleneck when it comes to converting regular DDR DRAM wafer capacity to HBM capacity.

The other difference is at the back-end, where the HBM needs to be stacked to 9 or 13 layers in total (8/12 DRAM layers on top of a logic base die at the bottom). Along with CoWoS, HBM has brought packaging technology to the mainstream. Niche packaging technologies like MR-MUF have now become common knowledge for industry participants.

Explosive Bit Demand

We can see the huge growth in HBM bit demand that has come alongside AI accelerator demand. Despite the rapid rise of custom ASICs, Nvidia will still command the lion’s share of HBM demand in 2027, driven by its aggressive roadmap, where Rubin Ultra alone pushes per GPU capacity to 1 TB. Broadcom follows as TPU and MTIA volumes surge, while incremental OpenAI and SoftBank projects add a smaller but noticeable lift. Amazon also emerges as one of the top HBM customers. For Amazon, it has a strategy of procuring HBM directly rather than through design partners, helping them to lower their cost. For a detailed, chip-by-chip bit forecast, refer to our Accelerator Model. It includes projections for revenue and bit demand by memory vendor, as well as wafer starts and TSV capacity, broken down by supplier. The model also tracks HBM pricing across different generations, and provides chip-by-chip HBM type, layers, stack counts, capacity and bandwidth.

Process Flow: Front End

When regular DDR DRAM capacity “converts” to HBM capacity – the main change is the addition of tools for forming the TSVs, and more bumping capacity as HBM wafers are bumped on both sides. Both of these steps are to enable the 3D stack, although they are omitted for wafers used for the top die which only need a single side of bumps and don’t require TSVs.

TSVs require etchers to create the vias, and deposition and plating tools to fill them. To reveal the TSVs, grinders, another etch step, and temporary bonders to attach carrier wafers used in this process are required. This is why HBM capacity is now quoted in terms of TSV capacity, as this is the main incremental set of processes that turn DDR wafers into HBM wafers.

For bumping, this is mainly deposition, plating and stripping. Also, optical inspection tools from Camtek and Onto are available to check that the bumps are not defective and are of the correct profile.

Process Flow: Packaging

The other part of this is back-end packaging which Hynix continues to push forward with MR-MUF which we wrote about extensively here. In short, MR-MUF offers higher productivity and better thermal performance. Hynix’s proprietary (co-developed with NAMICS) molded underfill material offers more thermal dissipation than the Non-Conductive Film that Micron and Samsung use. Hynix has been able to sidestep thermal compression bonding (TCB) because they have found other ways to manage warpage. One of the benefits of TCB is the use of force to stabilize the bonding matter.

The use of force on the other hand increases the risk of causing damages to bumps. With stress applied to bumps, SK can also add more dummy bumps which also helps with thermal dissipation.

The process is also far more productive. A batch mass reflow and single over-mold step are used for joint formation, compared to a complete TCB step for joint formation for each and every layer with TC-NCF.

Process Flow: Yields

HBM is a more technically sophisticated product than other DRAM form factors, especially given the high 3DIC stack. As such, packaging yields are not going to be comparable to what manufacturers are accustomed to compared to their more conventional products. However, yields at the front end are also challenging, and we think yield is more of a problem at the front end. As mentioned above, HBM is not demanding with regards to speed bins, so why is this the case?

The reason goes back to the 3DIC assembly and the TSVs. One of the challenges is the Power Distribution Network (PDN) with the TSVs needing to be able to deliver power up the stack. TSV layout and design is proprietary and one of the main areas of differentiation between the various manufacturers.

One of HBM’s key challenges is delivering power up the stack with power TSVs. Refresh operations especially draw a lot of power, and the design of the power distribution network is important. Hynix’s HBM3E has reduced peripheral area and introduced all-around power TSVs on the die instead of having two banks of power TSVs, thereby increasing the number of TSVs by almost 6x. As a result, SKH has achieved a much lower IR drop, up to 75% lower for VPP.

Similarly, Micron’s surprising leapfrog in HBM technology (Micron didn’t even offer standard HBM3) was due to its focus on TSVs and the power delivery network. The TSV network seems likely to be the point of differentiation that allows Micron to claim 30% lower power consumption, though that claim is yet to be verified.

The other thing is delivering the promised speeds within the power and thermal envelope. As with any 3DIC assembly, thermal dissipation is an issue, and DRAM especially doesn’t like heat. As data from hyperscalers have shown, HBM failures are the number one cause of GPU failures, which happen more frequently than other chips in the data center.

All manufacturers have absolute yields well below what they’re accustomed to compared to their conventional memory wafers, so it is a question of relative yields and the end economics. For SK and Micron, yield loss is more than made up for by high pricing, and hence, HBM is margin accretive. For Samsung, yields are even worse. Ironically, their low yields tighten up the total DRAM wafer supply, leading to higher pricing.

This leads us to layer counts. Higher layer counts are harder to achieve. Simplistically if the stack yield of a single layer is x%, each layer’s yield will accumulate to x% to the power of n bond steps (which is just total layers less 1). Take an 8-layer stack with 99% stack yield per layer; total yield will be 92%. For a 12-layer stack, this becomes 87%. Of course, this is overly simplified. Yield degrades at higher layer counts as non-critical stack defects can accumulate. For example, a small but acceptable amount of non-co-planarity at a few layers can result in an unacceptable amount of co-planarity at higher levels.

Process Flow: Bonding Tools, SK Hynix + Hanmi Drama

The bonding or die attach step is a key contributor to yield and thus requires sophisticated tools. With TSV pitches around 40 µm, the bonders must be capable of single-digit or even sub-micron alignment accuracy. Even pressure distribution is also crucial to avoid warpage that will compound over many layers. And throughput of course is important as it drives cost.

Hanmi made an early bet to focus on thermocompression (TC) bonders for HBM, a segment at that time ignored by market leaders Besi and ASMPT. This paid off in a near monopoly in current HBM processes. At SK Hynix their share was 100% until last fall, when Hynix placed a large order with competing tools from Hanwha. Allegedly, they paid Hanwha a higher price for the tools.

This set off a firestorm at Hanmi who, understandably, were upset to see a competitor winning with higher prices despite not being qualified on Hynix’s process for HBM supply to Nvidia, the largest and most important HBM customer.

The dispute reached a fever pitch in early April when Hanmi pulled its field service teams out of SK Hynix fabs. Without service, it would be months if not weeks before Hynix was unable to ship its marquee products. Longer-term it would threaten the entire accelerator supply chain as Micron and Samsung would not quickly be able to fill the capacity void. Hanwha’s tools had not yet been delivered and a batch of ASMPT bonders ordered last fall do not work for Hynix’s HBM3E 12-high. This left the fabs with little choice but to beg forgiveness from Hanmi.

Under enormous pressure, SK Hynix placed a small order with Hanmi in recent weeks. It appears this was more to placate Hanmi than a large volume order, but it was enough to restore field service to the tools. There may not be much more juice for Hanmi to squeeze from their monopoly power, as ASMPT, Besi, and others are rushing to improve their HBM-specific TC bonders.

China: CXMT & Huawei HBM

Export restrictions ban the transfer of all raw HBM stacks into China; however, chips with HBM can still be shipped as long as they don’t exceed the FLOPS regulations. Currently, banned HBM is still being reexported to China through a network involving CoAsia Electronics, Faraday and SPIL which allows end users in China to desolder and reclaim the HBM from GPU packages.

As HBM is one of the key ingredients for accelerators, and with export restrictions threatening to cut off HBM supply, China is naturally pouring resources into domestic development efforts. China has planned 200B USD in subsidies for home-grown semiconductors over the next five years. Expect a material portion will go towards HBM. DRAM national champion CXMT is aggressively expanding HBM capacity, with a large stockpile of tools to insulate from updated export controls (U.S. increased HBM controls in Dec. 2024, Korea more recently). HBM2 8-high will enter mass production in the first half of 2025, with TSV capacity matching Micron’s by the end of the year.

Huawei, which has never seen a high-tech market it doesn’t want to enter, has its own HBM affiliates with XMC (Wuhan Xinxin) producing the HBM wafers and SJSemi (Shenghe Jingwei Semiconductor) packaging them. Current capacity is at R&D scale, not high volume, but is planned to ramp in coming years. Both XMC and SJSemi are entity listed and thus restricted from buying equipment with U.S.-origin content (GlobalFoundries recently got a slap on the wrist despite selling more than $17 million in chips to SJSemi without a license).

HBM Stack Counts – Let’s Get High, to Hybrid Bond or Not?

More layers in the HBM stack means more memory capacity. With each generation, layer counts have gone higher. This stack height has been contained within a 720-micron height cube (the current JEDEC standard) up until now. To fit in more layers, each die (excluding the top die, which needs to be much thicker to withstand handling during subsequent packaging steps) has been made thinner, and the bump gap between the dies has also been made thinner to create more space to fit in more layers. Thinner dies are increasingly difficult to handle and, therefore, more vulnerable to warpage and breakage, which hurts yields.

The main benefit of Hybrid bonding (HB) for HBM is it is bump-less. By eliminating the bump gap this frees up room for more DRAM core layers to fit. This introduces a whole host of new challenges for yield and cost which may not be worth it, especially as HBM doesn’t need the level of interconnect density that hybrid bonding offers. If anything, manufacturers are exploring how much the pad density can be relaxed in hybrid bonding to ease bond accuracy requirements to make the technology more viable for HBM. HB also offers a one off power and heat benefit but the primary benefit is stack height.

HB adoption for HBM has always been a “next generation” technology and the goalposts continue to keep shifting. For D2W hybrid bonding, getting yields to be acceptable is extremely difficult and expensive for a 2 layer. Imagine scaling that problem for 16 layers and higher. The memory players are still very early in their hybrid bonding technology development. In TSMC’s experience, HB adoption has taken a long time to reach volume production and even when the performance benefits are clearer in advanced logic – adoption has taken a long time.

HBM3 and HBM3E are going up to 12-hi stacks and with bump-based interconnect, 12-hi has approached the limit within the current 720um cube thickness. The two solutions to go higher are either bump-less or making the stack taller/thicker. In a blow to hybrid bonding adoption, the latter has been chosen with JEDEC confirming a relaxation to 775 um stack height.

Height relaxation can go even further. 775um is the standard thickness of a silicon wafer. HBM needs to be the same height as the logic die it is co-packaged with. To have something higher than a 775 um stack, the logic wafer would also need to be thicker and the current equipment is not designed to accept thicker wafers. One potential solution could be to raise the molding of the interposer underneath the logic to raise it and ensure coplanarity with the neighboring HBM, though this lengthens the traces and implementing silicon bridges for interconnect will not be straightforward.

While initially there was more talk about HB implementation for HBM4, this has been shifted back to 4E. Recently, both Hynix and Micron have been far quieter on HB adoption, while Samsung is the loudest. This is typical for Samsung which often promotes the most aggressive technology implementations in attempts to catch up, only to expectedly fail on execution. This leads to them falling even further behind.

While 16 layer is solved by the higher stack height, to get to 20 layer and beyond, there is likely a further reduction of the bump gap and more wafer thinning, or we simply cap out at 16 layers. With higher stacks you can get more density but more stacks offer bandwidth and density.

Throughput Optimized: I/O is the Lifeblood of the AI Accelerator

The key defining feature of an AI accelerator is they are highly parallelized and optimized for throughput. Accelerators are designed to maximize the total number of operations an accelerator can perform per second by sacrificing the complexity of these operations. Most accelerators focus on multiplication and addition operations for General Matrix Multiplication (GEMMs), which dominate AI training and inference workloads. This compares to CPUs, which are also focused on how many instructions per second they can execute, but CPU cores are much “smarter,” which requires far more circuitry and area. Therefore they are designed to execute a broad range of more complicated tasks but at much lower throughput.

The implication is that AI accelerators need a lot of off-chip bandwidth for memory and the scale up and scale out fabrics. Bandwidth is needed to move the processed data off-chip whilst also feeding the accelerator unit with more data to process. Without sufficient bandwidth, the XPU’s compute elements will be left unutilized, defeating the purpose of having all this parallel computing power available. Let’s start with memory needs

Memory content increases

To deliver more performance – increasing memory capacity and bandwidth along with FLOPs has been paramount and easily observed in accelerator roadmaps. There are 3 dimensions where capacity and bandwidth are scaling:

1. Newer generations of HBM deliver higher bandwidth through faster signalling speeds, and denser core die
2. Adding the number of layers per stack increases capacity. We are on the cusp of 12-high HBM becoming the mainstream configuration
3. Adding more HBM stacks per package adds more bandwidth and capacity

Here we can see this in Nvidia’s roadmap. HBM capacity explodes from the A100’s 80 GB of HBM2E to a 1024 GB of HBM4E for Rubin Ultra. Memory bandwidth per chip also increases dramatically. From Ampere to Blackwell Ultra, the biggest absolute and relative increase in the bill of materials comes from the additional HBM content- benefitting the memory vendors (primarily SK Hynix).

This is related to the need for non-memory I/O as well. Scaling up more GPUs in a single memory-coherent domain delivers more aggregate memory capacity and bandwidth. This allows scaling inference of larger parameter models and support for much longer context lengths that are becoming prevalent in reasoning models and complex workloads.

Just as Parkinson’s Law observes that work expands to fill the time allotted, modern AI follows a “memory‑Parkinson” dynamic in which neural‑network architectures relentlessly grow to occupy whatever HBM becomes available. Each generational bump in HBM capacity and throughput, whether 80 GB at 3 TB/s on H100 or 192 GB at 8 TB/s on GB200, quickly encourages designers to increase parameter counts, context lengths, and KVCache footprints, nullifying the headroom that seemed ample only months earlier. Techniques once deployed to squeeze models into tight budgets (activation checkpointing, optimizer off‑loading, weight quantization) are relaxed as soon as new HBM space appears, until the memory wall is hit again and efficiency tricks must be rediscovered. In effect, the mere presence of larger, faster HBM does not yield sustained slack; instead it resets the baseline for “reasonable” model size, ensuring that capacity and bandwidth remain limiting factors despite silicon advances. Essentially, as AI chips get more HBM, developers immediately build larger models to fill it, so memory is always the next bottleneck. Let’s go through how HBM is used, and where the pressures are.

HBM Usage in Inference

In LLM inference, all the model weights reside permanently in the on-package HBM memory so the GPU can fetch them without delay. Alongside the weights, HBM also holds the KVcache. Each time the model is asked to produce the next token, the GPU first reads the weights from HBM and simultaneously retrieves the entire KV cache so it can compare the new token against the conversation history during the self‑attention phase. After computing, the GPU appends a fresh key and value for the newly generated token back into HBM, enlarging the cache. This is very demanding on bandwidth as every token decode step repeatedly reads both the static weights and the ever‑growing KV cache. If memory bandwidth is not able to move this data at terabytes per second the GPU spends more time waiting for memory than performing computation. This is what happens in reality, as bandwidth significantly overwhelms the compute intensity of token decode, making most LLM inference workloads memory bandwidth-bound rather than compute bound.

As models improve, they have increased in horizon lengths. What this means is that models are able to think, plan, and act for longer periods of time. This rate of increase has been exponential and has already manifested itself in superior products. Deep Research from OpenAI, for example, can think for tens of minutes at a time, while GPT-4 mustered mere tens of seconds.

As models can now think and reason over a long period of time, the pressure on memory capacity explodes as context lengths regularly exceed hundreds of thousands of tokens. Despite recent advances that have reduced the amount of KVCache generated per token, memory constraints still grow quickly. One way to deal with this has been to serve reasoning models at lower batch sizes which is harmful to economics.

The main driver of AI progress is Reinforcement Learning (RL) and a huge part of the RL paradigm is inference. As an example, often what is needed for RL is synthetic data satisfying strict requirements, which means conducting many GPU-hours worth of inference to generate data that is later filtered by another model. Another example of heavy inference loads is RL for hard-to-verify tasks like creative writing. Unlike code, which can be checked and verified easily, things like creative writing, legal work, and teaching cannot be verified trivially. The way to get around this, and thus to get a signal to reinforce and improve the model, is to have another model judging the answers. This LLM-as-a-judge is then given a rubric, currently hand written but soon to be automated by LLMs, which it uses to grade the answer.

KVCache offload

There are various algorithmic or setup improvements that are designed to reduce pressure on scarce HBM. One technique is offloading the KVCache to a cheaper and more available tier of memory such as conventional DDR or even storage.

Today, KVCache offloading is already commonly used . Nvidia has a framework for this called Dynamo Distributed KVCache Manager. Conceptually this is not so different from the multiple tiers of memory available in general purpose CPUs: extremely fast but low density L1/2/3 Cache and slower but high density DRAM. In an AI system, where the where the KVs are stored based are managed based on frequency of usage. A well-optimized system keeps all currently used KVs in HBM, infrequently used KV in DDR, and very rarely used KV in NVMe.

Just like how DRAM is not cannibalistic to L1/L2/L3 cache demand for CPU, HBM and DDR/SSD offload do not compete with each other directly. In fact, for most modern LLM workloads, the prefill speed (aka the rate at which KVCache is produced) is typically slower than the transfer rate to DDR or NVMe SSD, meaning KV rarely ‘lives in HBM’ in their entirety. They get produced and evicted or sent to the decode node to be used to produce the next token. Mostly it’s the system prompt that is used for every user that is kept in HBM as well as other hot KVs such as the active sequence window and some prefetch buffers.

As for whether DDR or NVMe is used, it depends on workload needs and size of workload. It also depends on how often the workload is cycled as frequently cycled KVs are not a good fit for the limited write/rewrite tolerance of NAND. Agentic use cases that use tool calling to pull in documents and data at very low latency and high cycle rates today further push caching from NVMe to DDR. These are architectural and user experience tradeoffs to consider, rather than direct substitutes to each other.

As use cases evolve, there may be different hardware setups used for different inference needs. For example, querying a fixed code base or document would benefit from accessing larger amounts of KVCache per user per GPU as the memory demands of those user behaviors is extremely high compared to normal chat.

HBM for Pre-Training

For traditional pre-training, everything the GPU needs for a forward‑and‑backward step passes across HBM. First, the model’s weights are stored in HBM so each layer can read them quickly while computing the forward pass on a batch of data. As every layer processes the batch, it writes intermediate activations into HBM so they are available later. Once the forward pass finishes and the loss is calculated, the backward pass begins: the GPU revisits those stored activations and weights, reading them from HBM to compute gradients. The resulting weight gradients, plus any auxiliary optimizer statistics (for example, momentum or variance terms in Adam), are also written to HBM. Finally, the optimizer reads those gradients and statistics from HBM to update the weights in place, ready for the next iteration. However, training operations require more compute relative to data transfer, meaning training is more often compute bound. But as mentioned above, RL is now key to improving model capability – so what was traditionally achieved with pre-training is becoming more like inference in the form of reinforcement learning.

The rest of this piece will discuss OpenAI’s ASIC project, shoreline area challenges, the revolutionary change coming to HBM4 with custom base dies for HBM, the advantages on PHYs, memory controller offload, repeater PHYs, LPDDR + HBM combos, and various beachfront expansion techniques. It will also discuss SRAM, compute under memory, supply chain implications, and Samsung’s struggles.

Long time readers will recall that SemiAnalysis covers more than just datacenters and AMD. Today we’re back to semiconductors with a tech-focused roundup of the best from this year’s VLSI conference, the premiere design and integration. That includes the latest in chips manufacturing: fab digital twins, the future of advanced logic transistors and interconnects, DRAM architectures beyond the 1x nm nodes, and more. We’ll discuss Intel’s 18A process and compare with TSMC, where backside power will be adopted (and where it won’t), and the likely winners in 4F² versus 3D DRAM.

Digital Twins: From Atoms to Fabs

Semiconductor design and fabrication is getting exponentially more complex, increasing development costs and lengthening design cycles. Digital twins allow for design exploration and optimization to be done in an accelerated virtual environment. With this, engineers can ensure that designs work before any silicon is run through the fab.

Digital twins span the entire scale of semiconductor design:

• Atomic-level: Simulate the quantum and Newtonian interactions between atoms in materials engineering of transistor contacts and gates
• Wafer-level: Optimize tool chambers and process recipes in virtual silicon for yield and performance
• Fab-level: Maximize fab productivity with orchestrated maintenance and management across the fleet

On atomistic simulations, Synopsys provided an overview of their QuantumATK suite, used in materials engineering in transistor contacts and gate oxide stack design, which are critical to device performance. Traditional Density Functional Theory (DFT) modelling of quantum effects between atoms is the most accurate but computationally expensive, while conventional force field simulation of Newtonian atomic interactions is quick but with limited accuracy. GPU accelerated DFT-NEGF (Non-Equilibrium Green’s Function) demonstrated a 9.3x speedup using only 4x A100 vs CPU, while Machine-Learned Force Field simulation using Moment Tensor Potentials demonstrated near-DFT accuracy with 17 min compute cost vs 12 days with traditional DFT.

These atomic models are critical in understanding the electrical interactions occurring at the interface between different material layers. In contact engineering, MLFF is used to generate the contact interface between crystalline silicon and amorphous silicides, simulating the depth of interdiffusion where the boundary undergoes silicidation. DFT-NEGF is then used to calculate contact resistance and current-voltage curves across the interface. For gate oxide design, the complex multi-layer work function metal stack is built using MLFF and simulated to check its structure and chemical composition. Dipole dopants can then be introduced and optimized with DFT, which also does electrostatic analysis to calculate key parameters such as effective work function, Schottky barrier height and equivalent oxide thickness. As we move forward into Gate All Around design schemes, these atomic simulations become even more important in choosing the right materials.

On wafer-level optimization with virtual silicon, Lam Research presented work on their Coventor SEMulator3D software. With transistor geometries increasing in complexity from planar to FinFET to GAA, the number of possible process recipe combinations only grows exponentially, which they’ve marketed as Lam’s Law. Virtual wafer fabrication is done with process simulation using trained models with optimized parameters, allowing engineers to widen process windows and improve yield while reducing the number of physical test wafer cycles required to validate changes. Lam also builds their deposition and etch tools as a digital twin, building virtual chambers with plasma flow simulation to help in recipe prediction while also optimizing chamber design for uniformity across the wafer.

These simulation tools have been used in process window studies to select backside contact schemes with the widest process window while investigating how each recipe affects the stress and strain across the nanosheet transistors. High Aspect Ratio etch schemes also use virtual environments to predict tool output etch profiles for a given input mask pattern. These etch profiles are compared to a target output profile and given a distance which is then minimized with further tests in the digital twin.

Going up to the fab-level, Lam also presented the work needed to achieve a ‘lights-out’ fab. That is a fab that requires no human intervention and as such one can turn the lights off. Tool fleets need to be orchestrated in a virtual twin at near real-time speeds to coordinate tool downtime and maximize fab productivity. The tools themselves need to be ‘self-aware’ with their predictive maintenance, using built-in metrology tools that detect tool alignment and process drift over its lifetime. For a lights-out fab, each tool should target at least 1 year of uninterrupted operation with no human intervention, with automatic recovery after failure and self-requesting maintenance.

Maintenance of tools would be carried out automatically with robotic parts delivery and installation of consumables and wear items, with tools being designed around robotic maintenance. While a conceptual target of 2035-2040 was given by Lam, the main barrier facing lights-out fabs is with data and connectivity across tools from different vendors and the standardization of maintenance processes.

TSMC DRAM in BEOL

With SRAM bit density no longer increasing with new process nodes, TSMC R&D sought to revive eDRAM to boost chip cache densites. Embedded DRAM was last seen in IBM’s z15 on GlobalFoundries 14nm. The main innovation here is that TSMC can fabricate the entire memory array within the BEOL metal layers, with the DRAM transistor and capacitor formation scheme being within the 400 degree C limit of BEOL flows. This frees up the front-side transistors and lower metal layers for functional logic blocks. With modern processor designs increasing the ratio of SRAM to logic area, being able to stack a DRAM-based last level cache on top of active logic would represent a breakthrough in scalability and design.

However, the demonstration shown is still early in the R&D phase, with the available advanced logic area below just used to house the DRAM peripheral logic (Wordline drivers and sense amplifiers) to boost memory density. The 4Mbit macro that was fabricated only had a bit density of 63.7 Mb/mm2, not even 2x that of modern high density 6T SRAM. For reference, Micron’s latest 1-gamma DRAM would be about 9x denser than this, but without the performance and accessibility to serve as on-die cache.

While TSMC did not give any hints on when this will be ready for productization, it does show huge potential for future generations of this technology, which will fundamentally change how chips are designed.

DRAM: 4F² and 3D

DRAM has two inflection points on its 5-year roadmap: 4F² and 3D. The current 6F², in use for more than a decade, will only scale until the 1d node. With 1c in high volume now, 1d should debut in the next 1-2 years. SK Hynix highlighted a few key challenges in scaling beyond 1d:

Cell contact areas, in particular the storage node contact where the storage capacitor connects to the control transistor below, shrink quadratically with cell critical dimension. The contacts must be large / well aligned enough to provide a good electrical connection between the transistor and capacitor, but not be so large or misaligned that they short out to any neighboring cells. This is the “cell contact open margin” in the chart above, shrinking with every node. At 1d the process and tooling reach their limits for a workable, high-yield process.

The resistance in the shrinking devices and interconnects also grows as they scale down. This is the “cell external resistance” referenced above. It’s a catch-all sum of all resistive elements between the memory cell and the sense amplifier. The bitline contact and local bitline (metal) wire itself are two main contributors. Both increase in resistance as they scale. This slows down cell operation and reduces the read margin of the cell, both undesirable. Operation speed is affected by charge transfer between the cell and bitline, which slows as the resistance of that path increases. Resistance also saps the voltage differential that the sense amplifier sees. Too little and the cell cannot reliably be read – the memory doesn’t work:

The Memory Wall: Past, Present, and Future of DRAM

4F² solves these issues and more. We won’t rehash the basics of the architecture – see our Memory Wall report linked above – but a few specifics are interesting:

The cell contact challenge in 6F² comes from congestion where the bitline and storage node contact are on the same level (storage node contacts are denoted as BC for buried contact in the image below).

From a side-on view, its easy to see how little margin there is between the bitline and contact:

Compare this to vertical channel transistors (VCT) in a 4F² layout; the buried bitline has its own real estate well out of the way of any other components. The current path is also much shorter, directly down from capacitor, through the vertical channel, directly to the bitline. In 6F² the path is down through the bottom of the “U” shaped channel and back up, longer and consequently with higher resistance.

Of course, there are challenges for implementing 4F² or it would’ve been adopted already. Both the buried bitline and vertical channel transistor are high aspect ratio, difficult for etch and deposition tools. Until just a few years ago deposition tools were not capable of filling a deep trench with the required metals for the bitline, likely Ru or Co. The cell layout, although it reduces some of the alignment challenges, is still denser and thus requires EUV patterning. Last, there was simply no reason to take the risk of changing architectures when 6F² was still scalable.

There are still a few wildcards in 4F² development that may determine which fab can achieve lowest cost per bit and good yields, and which tool vendors might benefit. The gate structure, crucial to the performance of the memory cell, might be dual or even gate all around. SK Hynix and others are still deciding.

There is also a choice between peripheral-under-cell and peripheral-on-cell. Conventionally the peripheral circuitry would’ve been adjacent to the memory cells on the wafer frontside, but for increased overall density it will be moved underneath the cell array. Peri-under-cell is similar to backside power for logic, requiring fusion bonding of a second wafer. The control transistors are built in an array on the frontside before a support wafer is bonded on, the wafer flipped, and the peri built up. Then everything is flipped once more to add the storage node contacts and capacitors themselves. The tool vendors who would see incremental benefit are similar to the BSPDN supply chain – CMP, fusion bonding, TSV etch.

Peri-on-cell is simply hybrid bonding of completed storage node array and peripheral wafers. While this offers some process latitude – the peri can be made without worrying about damage to the array and vice versa – it requires hybrid bonding at a pitch of well below 50nm. That’s an order of magnitude lower than the current leading edge. Still, Hynix at least is looking at it in R&D, and other applications will drive hybrid bonder development regardless.

Last, 3D DRAM is being developed in parallel. Current progress suggests a few nodes of 4F² are likely before 3D is ready. Chinese chipmakers are a potential disruptor here, as they have strong incentive to develop 3D because it is not dependent on advanced litho.

NVDRAM

Micron’s NVDRAM (NV for non-volatile) resurfaced 18 months after its first appearance at IEDM 2023. This is their ferroelectric (HZO) DRAM using a 4F² architecture, Ruthenium wordlines, and CMOS under array. If you were trying to make an expensive memory using all the latest tricks, this is probably how you’d do it.

The bitcells were scaled by an impressive 27% since the previous paper, to 41nm on a side, without a degradation in performance. That brings density to nearly 0.6 Gb/mm², well above any high volume commercial DRAM available today.

Theoretically NVDRAM has a small advantage over traditional DRAM since it does not waste power and time performing refresh cycles. Unfortunately, the electricity savings amount to roughly $1 per year. When a single DIMM is on the order of $300+, the lifetime energy savings are nowhere near enough to justify the higher price of this exotic product. At the very least, the work on Ru wordlines, 4F², vertical channel transistors, and CMOS under array are all applicable to upcoming DRAM nodes.

2D Materials

Replacing silicon is a high bar. Any replacement must not only yield better performing, denser transistors, it must also be practical to use. Silicon wafers are a commodity and can easily be doped in select areas to form transistor channels. 2D materials are not yet practical to work with on an industrial scale. We’ve written many different times that on-wafer growth is they key blocker. But if chipmakers or labs are solving this problem, they’re keeping it quiet. The papers we saw on other innovations – Intel improving contact formation, Samsung building CFETs with 2D channels – are impressive, but ultimately fall flat if the material can’t be grown in an economic way in the first place.

Forksheet

Gate-all-around is no longer the “next big thing” in logic, it’s ramping towards high volume. Forksheet and CFET have taken the mantle of exciting next-gen architectures. An evolution of GAA, forksheet entails putting the N- and P- halves of a CMOS closer together by adding a dielectric wall between them.

In a traditional architecture, the spacing between NMOS and PMOS devices is limited by parasitic capacitance and the threat of latch-up. Increasing parasitic capacitance means the chip runs slower and uses more power. Latch-up is an outright failure of the transistors giving the input voltage V_dd an uncontrolled path directly to ground. Some techniques already exist – most importantly shallow trench isolation – for mitigating these effects.

Forksheet is a new, theoretically better technique along these same lines. While the material between NMOS and PMOS has always been some sort of insulating dielectric, a forksheet requires an exquisite layer of ultra low-k material to enable tighter spacing. This introduces new integration challenges and extra cost into the manufacturing process.

It’s not trivial to develop a material that can be deposited in nm-thick, high-quality films but also withstand subsequent processing as the rest of the transistor is formed. Plasma-induced damage from etch or deposition is a particular problem. Most papers aren’t detailing their material solutions here but it’s a good bet that AMAT, traditionally the leader in ultra-low-k dielectrics, is playing a part.

Forksheet also nominally has worse gate control than gate-all-around. This is because the gate only wraps around three sides of the transistor channel, the fourth abuts the forksheet wall. It’s basically a finFET turned sideways. Increased density but worse electrostatic control vs. GAA isn’t necessarily a good tradeoff. There are a few workarounds: 1) slightly etch back the forksheet wall, leaving room for gate material to envelop the fourth side of the channel, sacrificing some scaling benefit 2) add additional nanosheets to improve electrostatics control, adding cost and integration complexity.

TSMC, IBM, and IMEC all talked extensively about forksheets. For IBM and IMEC this has limited commercial relevance. For TSMC, a willingness to discuss in detail might even be a negative signal for real adoption. No publicly announced node – through the 14 angstrom families at present – is using forksheet.

CFET Timeline

Even so, the potential successor to forksheet is already being discussed. CFET has been in vogue for a few years and we’ve covered the basics before:

Current work is towards industrialization. Lab demonstrations of a single device work well and look great on slides, but the cost is high and yield low. Even though they are popular at conferences, we think real adoption of CFETs in high volume is still a decade out. A presenter from Intel, in a talk about “beyond RibbonFET,” outright said “We will probably see gate all around for another decade.” As with copper interconnects and finFETs, the core technologies of logic tend to extend 1-2 nodes beyond what is expected.

China’s FlipFET Design

Despite various sanctions, China is not slowing down in semiconductor research and development. Out of all the academic papers presented, Peking University’s FlipFET design caught the most attention, showcasing a novel patterning scheme to achieve similar PPA to CFETs without the headaches of monolithic or sequential integration.

Essentially, the FlipFET concept starts with forming the fins or nanosheets for both top and bottom transistors, but only does the high temperature Source/Drain epitaxy for the top transistor before flipping the wafer and exposing the backside for processing. The contacts and BEOL metal layers are patterned before the wafer is flipped again to complete the lower temperature processes on both sides. This method produces a self-aligned transistor stack that does not need high-aspect ratio processing that monolithic CFETs have to overcome. Forming the gates from both sides also allows easier threshold voltage tuning differences between the top and bottom device.

However, the main drawback to FlipFET is cost, trading off easier integration of the active transistors at the expense of multiple backside process flows with a greater susceptibility to wafer warpage and overlay errors, lowering yield. So far, the lab has only fabricated frontside and backside transistors on separate wafers, leaving doubts on whether the fabrication of the other transistor would affect device performance of the first. Alignment of fine-pitch contacts and metals after wafer flipping is also a concern, but should not be any more challenging than other CFET options.

While the Chinese labs have already demonstrated FlipFETs in silicon, they’re not stopping there. Further innovation of FlipFET designs were presented and modelled such as FlipFET with self-aligned gate, FlipFETs using forksheets with an embedded power rail within the isolation wall, and even applying the FlipFET concept to monolithic CFETs with high aspect ratio Vias to achieve a 4-stack transistor design.

18A Process Details

The star paper was Intel’s 18A presentation. This is the first detailed public look at a real high volume backside power process.

Intel claims 30% SRAM scaling for 18A against an Intel 3 baseline. A large one-time benefit like this is expected when changing from finFET to GAA. The cell diagrams clearly show the shrink achieved by replacing 2 fins with a single stack of wide ribbons:

Comparing high density (HD) cell areas, 18A is on par with TSMC N5 and N3E at 0.0210 µm². N2 should also see at least some benefit from the finFET -> GAA transition, but most of the claimed 22% SRAM scaling (vs. N3E) likely comes from the periphery not the bitcells themselves. Overall 18A density is likely to be slightly less than N3P, and close to 30% less than N2.

18A Cost vs. Leading Nodes