Produced by Zhineng Technology

For the past two decades, software has profited from duplication, while hardware, doing the heavy lifting, has seen thinner margins.

AI's emergence now consumes software itself! Each additional user served by a large model requires more GPUs, memory, and power, introducing unit costs to software.

Not all hardware is rising in price; HBM, advanced packaging, and power are the bottlenecks for computing power.

Large-scale companies are now developing their own chips. Nvidia no longer just sells a single GPU but an entire computing system.

The boundaries between hardware and software are disappearing, with tech companies increasingly resembling system companies.

Bain's "Hardware Strikes Back" report this year makes it clear. From 2020 to 2026, hardware and semiconductor stocks have outperformed software with a 24% CAGR, while software has only seen 6%. Hardware valuations are rising, and software valuations are being compressed—this reversal is structural, not cyclical.

1) Software No Longer Profits from "Duplication"

For the past two decades, a nearly unquestioned rule in the tech industry has been that software is more valuable than hardware.

Microsoft, Adobe, Salesforce, and ServiceNow all share a common trait. Once a software product is developed, the cost of selling it to the millionth customer is not significantly higher. A single codebase can be duplicated countless times.

Thus, once software companies scale, their revenue can grow rapidly. While server, sales, and R&D investments also increase, they do not rise proportionally with sales volume.

Investors have favored software for this reason. Light assets, high margins, subscription revenue, and scale effects have formed the typical SaaS valuation model of the past decade.

Hardware is the opposite. Producing an additional chip requires an additional wafer. Selling an extra server means actually building another server. Factories, equipment, depreciation, inventory, yield rates, and supply chains are all unavoidable.

For a long time, the tech industry had a clear division of labor: hardware did the heavy lifting, while software captured more profit.

AI Imposes "Unit Costs" on Software for the First Time

When you write a document in Word, Microsoft doesn't fire up dozens of extra GPUs just because you typed 1,000 more words. When you ask a large model a question, real computation is required. The larger the model, the longer the context, the longer the inference, and the more users—the more GPUs, HBM, network bandwidth, and power are consumed.

AI software introduces a change never seen before: the more users engage, the greater the backend physical consumption.

Past software companies loved it when users went wild with their products because marginal costs were low. AI companies, however, must consider how many Tokens a user generates today, what model size they invoke (use), how much GPU time they consume, and how much each inference costs.

AI is making the software industry confront a problem long faced by manufacturing: unit costs. Revenue growth no longer automatically equals proportional profit growth.

This is why large model companies are relentlessly pursuing model distillation, quantization, KV Cache, speculative decoding, and MoE. On the surface, these are algorithmic issues, but they all boil down to one thing: can the same task be completed with fewer chips?

2) Hardware Now Profits from "Scarcity"

What Bain means by "hardware is back" is that profitability is returning to the components that bottleneck AI system performance.

Take HBM, for example. Large model computations require constant data transfer between processors and memory. As GPUs get faster, memory bandwidth starts to lag. HBM has transformed from a relatively niche storage product into a critical component in AI servers. Samsung, SK Hynix, and Micron have evolved from ordinary memory suppliers into key development partners for AI data centers. After HBM becomes deeply tied to underlying logic chips, customers can no longer switch suppliers as casually as they did with ordinary DRAM. SK Hynix and Micron's gross margins have surged to 75-85% for two consecutive quarters—numbers previously unthinkable in the memory industry.

Advanced packaging is no different. Previously, the most direct way to boost performance was to shrink transistors. But as advanced nodes become prohibitively expensive, the industry has started integrating more chips. GPUs, CPUs, HBM, and I/O chips are combined into a system via 2.5D, 3D, or Chiplet packaging. Packaging has evolved from a final assembly step in chip production into a critical determinant of computing performance. Bain directly refers to advanced packaging as the new battleground for competition.

Scarce hardware—components that limit AI computing scale—is rising in value. Not enough GPUs? GPU prices rise. Not enough HBM? HBM prices rise. Not enough CoWoS capacity? Packaging expands. Not enough data center networking? Optical modules upgrade. Not enough power? Discussions turn to nuclear energy, gas turbines, and energy storage. The larger AI scales, the more the tech industry bumps into physical-world limits.

1) At Scale, Custom Chips Make Sense

Developing your own chip used to be a luxury. Design costs run into the hundreds of millions of dollars, and tape-out failures require starting over. If annual demand is only a few hundred thousand units, the math simply doesn't work.

But Google, Amazon, Meta, and Microsoft operate at entirely different scales. A single company's AI server fleet might consist of hundreds of thousands or even millions of chips, with the same inference tasks running billions of times daily. At this scale, even a 10% improvement in chip efficiency can save hundreds of millions in electricity, GPU purchases, and data center investments.

Thus, developing custom chips now makes financial sense. Google built TPUs, Amazon created Trainium and Inferentia, and Meta and Microsoft have also entered chip design.

Custom Silicon is now the fastest-growing category of compute chips in data centers, with ASICs capturing market share from Nvidia's GPUs. A company buying only 10,000 chips annually would likely find custom design uneconomical; but one purchasing a million chips yearly, especially when suppliers enjoy high margins on each, could save vast sums by self-designing. With commercial GPU margins as high as 75%, this is precisely the incentive for in-house development.

The larger an AI company, the stronger its incentive to move into chip design!

2) Why Nvidia Is Worth Far More Than Ordinary Chip Companies

Understanding this point also explains why Nvidia is so valuable.

If Nvidia only sold GPUs, it would still be an excellent semiconductor company. But today's Nvidia actually sells an entire computing system. The GPU is just one component. CUDA, compilers, communication, NVLink, networking, server architecture, and development tools are all bundled together.

Customers buy the complete computing environment needed to train a large model—this is the biggest shift in the future hardware industry. Chip parameters will remain important, but comparing TOPS, TFLOPS, or process nodes alone will increasingly fail to explain a company's competitiveness.

The real questions become: How much does it cost to train a model? How much power is needed to run 1 billion Tokens? What utilization can 1,000 cards achieve? How much memory and communication does an Agent task require? Software and hardware can no longer be fully separated.

3) The Automotive Industry Precedes Tech: Boundaries Dissolve, System Companies Rise

Automotive companies are increasingly bypassing traditional Tier 1 suppliers and collaborating directly with chip firms, as OEMs seek deeper control over underlying hardware, software updates, and safety systems.

This trend is already evident in smart vehicles. In the past, automakers purchased an ECU, and Tier 1 suppliers integrated chips, software, and controllers, with OEMs only specifying functional requirements. Now, everything has changed. Intelligent driving requires OEMs to train their own models, cabins demand custom-defined operating systems, and central computing platforms must consider chips, OSes, middleware, and vehicle control. As a result, automakers now directly partner with chip and computing platform firms like Nvidia, Qualcomm, Horizon Robotics, and Huawei.

Tesla, BYD, NIO, Li Auto, and Xpeng have begun designing their own chips. When a company's software becomes increasingly dependent on specific computing tasks, the economic value of self-designed hardware emerges.

Note: The economic value of automotive SoC chips currently pales in comparison to servers!

Summary: Tech Companies Increasingly Resemble "System Companies"

The once-clear boundary between hardware and software is disappearing, and the tech industry has long favored horizontal specialization—some build CPUs, others operating systems, servers, databases, or applications.

AI is reshaping the industry toward vertical integration. Model companies now study chips, cloud firms design their own processors, automakers participate directly in SoC definition, chip companies provide complete software stacks, and data center firms even consider power plants.

The highly horizontal industrial structure of the PC era is unlikely to be fully replicated in the AI era. Instead, more pronounced vertical integration will emerge.

The most famous line of the past two decades was: "Software is eating the world."

In the future, AI will make software consume hardware—and software itself will begin to be redefined by hardware.

What is truly scarce in the AI era is computing power, memory bandwidth, advanced packaging, energy, manufacturing capacity, and supply chains.