My view from OCP2026 is that microVCSELs have made it hard to justify microLEDs for cable interconnectrs. MicroVCSELs make slow-and-wide optics compelling over longer short-reach links, while microLEDs need to justify themselves through advantages elsewhere in the system. The interesting question is where that boundary falls, and what implications that has for companies building in the interconnect space.

The Short Reach MicroEmitter Optics (SRMO) workstream at OCP has been set in motion to define specifications around short-reach optics. Grouping LEDs and VCSELs under “microemitters” leaves room for companies to choose the light source based on the application.

In this post, we will sort through the trade-offs to understand why Credo chose microVCSELs, where microLEDs still make sense, and what remains to be proven. Motivation and underlying physics are free subscriber content; industry implications are for paid subscribers.

Table of Contents:

• Why Short-Reach Optics? What is changing between racks, and why are familiar interconnect technologies being reconsidered?

• microVCSELs vs microLEDs The device physics behind their differences in speed, reach and scaling—and what headline specifications leave out.

• Inside Credo’s Change of Direction The engineering and commercial considerations behind one of OCP’s most discussed announcements.

• Where microLEDs Still Have a Case Could a different application change how we judge this technology?

• What Remains Unproven The questions about reliability and lifetime cost that a successful demonstration cannot answer.

• Can Microemitters Become ams OSRAM’s Next Growth Engine? Its technology choices, competitive position and the milestones investors should watch.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. For institutional research, contact me at SemiExponent.

Why Short Reach Optics

We have already discussed in a previous post how power and networking limitations are pushing the scale up domain to multi-rack configurations, and how that opened up an opportunity for active copper cables.

As lane rates and distance increase between racks, copper’s capabilities are becoming more limited. At 200G/lane, Active Electrical Cables (AECs) today can push to about 7m of reach. But this is insufficient to connect scale up domains like NVIDIA’s NVL576 that has 8 Oberon racks, for which the required reach can easily reach 10-20m. Speed is another limiting factor. As a rule of thumb, copper’s reach halves when lane rates double, which makes it hard to scale copper interconnect solutions. Finally, power consumption and latency in the scale up domain needs to be low, which limits the use of expensive and power hungry DSPs. The industry is trying to push copper with PAM6 modulation, while developing PAM4 in parallel for the 400G/lane era. We discussed this in the last post.

On the other hand, the InP laser-based optics is arguably overkill for applications under 50m because optical fiber is capable of carrying light across continents. Add to the fact that InP is already in short supply, and optical pluggables are often power hungry. Companies like NVIDIA, AMD, and Amazon are pushing CPO/NPO solutions to extend reach and lane speeds using low power laser-based optics, but those bring their own challenges concerning reliability and supply at scale. For links spanning only a few tens of meters at most, the question is whether we can deliver the required bandwidth with simpler, lower-power optics.

This opens the door to new optical technologies like microemitters that work for short reach (<50m), are not InP based, consume low power, and have the ability to scale lane speeds. Instead of pushing each optical lane faster, they distribute data across many slower lanes. Both microLEDs and microVCSELs follow this approach, but their device characteristics lead to different trade-offs in speed, reach, and integration.

My expectation is that the choice will depend on where the link sits. Across racks, reach becomes a stronger argument for microVCSELs. Closer to the compute, microLEDs have an opportunity to compete on dense integration, power, and thermal performance.

MicroVCSELs vs MicroLEDs

The choice between microVCSELs and microLEDs starts with what the link needs to do: how much data it must carry, over what distance, and within what power budget. Both technologies spread data across many slower optical lanes, but the properties of their light sources determine how fast each lane can run and how far it can reach. Understanding those differences helps explain where each technology fits.

Material and wavelength

MicroLEDs use GaN-based emitters that produce blue light around 420 nm, while microVCSEL approaches use GaAs-based material systems, with designs targeting wavelengths such as 850 nm and 1060 nm. Both can be manufactured using at least 6”-8” wafer sizes and are widely available in the supply chain, compared to 3”-6” wafers used in InP lasers.

VCSELs have an established history in optical communications, particularly in 850 nm multimode datacom links. That gives microVCSEL developers a foundation of device and manufacturing experience to build on. MicroLED interconnects are newer to this application. While that experience is an advantage, it does not establish that dense microVCSEL arrays are ready for deployment at scale.

How they emit light

MicroLEDs produce light through spontaneous emission: electrons and holes recombine and release photons without a fixed phase relationship between them. MicroVCSELs are lasers. Above a threshold current, stimulated emission sustains lasing within the optical cavity. In this process, light already present stimulates the emission of additional photons into the same optical mode.

VCSELs offer both a narrower emission spectrum and higher modulation bandwidth. The narrower spectrum reduces chromatic pulse spreading, while the higher modulation bandwidth allows each lane to carry more data. MicroLEDs compensate for their slower speeds by using more of them, but adding lanes does not extend their reach. This is why microVCSELs have a stronger case as the required distance increases.

Spectral Linewidth

Neither light source emits at exactly one wavelength. Its output spans a range around the center wavelength, and the width of that range is called its spectral width. While the exact linewidth depends on device design, microLEDs have a broad emission spectrum, spanning 20-40nm around 420nm while VCSELs emit over a much narrower range of 1nm or below around 850nm/1060nm. Linewidth has direct implications on how far along a fiber a light pulse travels.

Chromatic and Modal Dispersion

Pulse spreading caused by wavelength-dependent propagation is called chromatic dispersion. It occurs because refractive index (which determines how fast light travels through the fiber) changes with wavelength. The farther light travels through the fiber, the more the different wavelength components separate in arrival time. Eventually, neighboring pulses overlap enough to make the transmitted bits difficult to distinguish.

Slowing the data rate gives each bit more time, allowing the link to tolerate more spreading. Shortening the fiber reduces the spreading itself. This is why a demonstration at a high data rate over a short distance does not establish that the same rate will work across racks.

The narrower emission spectrum of VCSELs helps them reach farther before chromatic dispersion becomes limiting. However, spectral width is only part of the comparison: the fiber’s dispersion also depends on the operating wavelength. In multimode fibers, differences in arrival time between optical modes introduce another constraint called modal dispersion.

For a deeper explanation of the relationship between spectral width and reach, see my earlier article (free post).

Data rates

Most microLED links demonstrated in the industry for interconnect applications operate between 1-3 Gbps. I covered this in more detail in my field trip notes to Avicena, a leading manufacturer of microLED based interconnects. In the report, I also show an 8 Gbps demonstration through a free space link showing that microLEDs are capable of higher data rates, but I anticipate that transmission through fiber will limit its reach at that data rate.

VCSELs offer more per-lane speed. They can easily run at 50 Gbps, and arguably up to 200 Gbps. Running speeds an order of magnitude higher than microLEDs provides microVCSELs a natural advantage of running fewer parallel lanes for the same overall speed. This simplifies the attachment of fiber arrays and also allows the use of multi-core fiber (LightXcelerate demonstrated this at OCP). For slow-and-wide interconnects, the opportunity is to operate each VCSEL lane comfortably below its maximum capability.

Why the industry is moving toward microVCSELs

MicroVCSELs offer three advantages for near-term optical cable products: greater reach, more operating margin, and more options to scale bandwidth.

Read more

Signal modulation and its impact on interconnects is often discussed in academic circles and industry research, but we might be at a time where the industry is seriously considering PAM6 as an alternative to make copper work in the next speed generation.

We’ll walk through this idea in a step-by-step fashion, starting with NRZ and ending with PAM6, and discuss what its implications for copper interconnects are.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. For institutional research, contact me at SemiExponent.

Pushing Copper’s Limits

Copper has largely been declared dead in the era of 448 Gbps interconnects.

The impact of skin effect losses, where current crowds on the periphery of a conductor carrying fast moving signals, is too debilitating at these speeds. As a result, a signal degrades rapidly within a short distance.

At 224 Gbps speed, this distance was a little lesser than 2m. Doubling the speed roughly cuts the reach in half: 448 Gbps is limited to about 1m, or less, depending on the signal quality you are looking for. This is too short to connect hardware within an AI rack.

The speed limitation is tied to what the losses of a copper cable are at a given frequency, called the Nyquist frequency, which is then tied to the modulation of the signal being carried by the wire. Time to explain the word salad.

NRZ Modulation

When you say 448 Gbps, you think of sending one bit at a time (0 or 1, low or high voltage) 448 billion times in a single second along a wire. You would be right if you were using Non-Return-to-Zero (NRZ) modulation: a fancy way of saying let’s take the 0 bit and send 0 volts and take the 1 bit and send some fixed higher voltage that is not zero. If you overlap all the 0s and 1s passing through a window of time, you get the eye diagram shown below.

You think of each bit as a symbol in NRZ (we’ll understand why next, hang on), and in this case, symbol rate = data rate. Symbol rate is measured in Baud. So, 448 GBaud in this case.

There is a magical frequency called the Nyquist frequency, which is defined as half the symbol rate, where the loss of the signal cannot be too heavy. If it is, the signal is not recoverable anymore. So for a 448 GBaud symbol rate, the losses at Nyquist frequency of 224 GHz should be acceptable.

Running a copper cable at 224 GHz is laughable. At these frequencies, there is a good reason why RF engineers use waveguides. Skin effect from copper will obliterate a signal at the end of a 1m long cable at 224 GHz.

So what do we do?

Pulse Amplitude Modulation 4-Level (PAM4)

Let’s try and get clever about this and take two bits at a time: this gives four possibilities — 00, 01, 10, 11. Then you say, instead of mapping that to two voltages, you map it to four: 0V, 0.33V, 0.66V, 1V. You just “modulated” the signal into a 4-amplitude levels.

Now you’re going to work out for yourself how the problem gets better. Ready?

Think about it. Where you were sending 1 bit in a symbol, now you’re sending 2 bits to a symbol. To get 448 Gbps by running 2 bits per symbol, you need to only send half the number of symbols per second. If you selected 224 GBaud, you are correct. If not, now you know. The next question should be easier if you remember Nyquist frequency is half the symbol rate.

If you are running a 224 GBaud symbol rate, Nyquist frequency is half that, or 112 GHz. Can you send a PAM4 signal at 112 GHz along a copper wire? Yes — to about a distance of 1m.

TE Connectivity showed how you can use co-packaged copper (CPC) cables to get from the ASIC to the front panel (about 0.3m), through a direct-attached-copper (DAC) cable (1m), through another CPC cable (0.3m) to the next ASIC. A total of about 1.6m, or if you count only the DAC, 1m.

The loss is shown on the top left plot in the grid below. You can see that the signal has -35 dB loss at 112 GHz, after which the loss craters. Note that -30 dB is already only 0.1% of the original signal. Every 3 dB below that is half that signal power. At -36 dB, you’re looking at 0.025% of the original signal. That’s low, but recoverable.

What does recoverable mean? A way to characterize the quality of a recovered signal is to measure the Bit-Error-Rate (BER) of the signal — aka, how many bits recovered are flat out wrong compared to what was originally sent. If you detect a 0 where it should have been a 1, that’s a bit error.

TE Connectivity’s graph is shown below, and provides a few more interesting details. The plot on the left with the green color is what we just looked at, for a 1m DAC cable but “end-to-end’ — including the end connectors and CPC. It gives a loss of -46 dB and BER of 1.4e-6. About 1 in a million bits go bad. That’s passable because there advanced techniques like error correction that can push it to lower BER.

The green curve on the right is for a 2m DAC cable. The loss is -63 dB at 112 GHz, which means that there is almost no signal power left. That’s about 2 million times lower power than what was sent; copper losses have obliterated everything. The BER is 1.2e-1, or about 1 in 10 bits are bad. That is a terrible outcome considering how much data we will be sending in an AI cluster (we are talking Terabits, and you can’t have 100 Gigabits go bad.) No amount of error correction can fix something so bad.

All hope is not lost if you can put a circuit called a “re-driver” on the receiving side that takes the weak signal and boosts it before trying to recover the bits. That’s a logical approach, and TE Connectivity shows this as the blue curves in the plots above. They use a simulation model of a Semtech re-driver. You can see that the losses get better: -32 dB for 1m, and -49 dB for 2m, and the corresponding BER values also improve to 5.8e-6 to 1e-7 (or lower).

The price of doing is that now you are burning some power to have this re-driver chip. Nothing is free. You went from a purely passive copper cable, to using an active cable. Hyperscalers are pretty adamant about not using extra power though, and not having more components like re-drivers that need a supply-chain and can fail. How can we keep this purely passive?

Enter PAM6, and if you’ve been following so far, now things get a bit … funny. Try and answer this:

Next, we will discuss the madness that is PAM6.

Pulse Amplitude Modulation 6-level (PAM6)

If you grouped 2 bits together, you got PAM4. If you group 3 bits together, you get 2^3 = 8 levels, or PAM8. To get 6-amplitude levels or PAM6, you need to group log2(6)=2.585 bits together. Let’s just say 2.5 bits for now. How do you group half a bit?

Well, you don’t. The most practical and common approach to generate PAM6 is to map 5 bits across two symbols, which effectively maps to 2.5 bits/symbol. Mapping 5-levels involves 2^5 = 32 signal levels, which is quite a jump from four. Instead, what engineers do is to use 2-dimensional mapping. You take the signal, multiply it by a cosine signal and call that the “in-phase” signal. Similarly, you also multiple it by a sine signal and call it the “quadrature” signal. Now you can represent 32 points across a plane in what is called “Quadrature Amplitude Mapping” or QAM-32.

The constellation of points for QAM-32 is shown below.

The random number ordering you see is to keep adjacent levels far away from each other in the constellation so that you don’t mistake one adjacent level for the other. Notice how many levels there are across any row or column. There are a maximum of 6 levels, which fits nicely into 6-level amplitude modulation, or PAM6.

Skip on first read: Its also a good thing the corner symbols are missing because those represent the highest energy symbols which we don’t have to transmit. There is another technique called Probabilistic Amplitude Shaping (PAS) that optimizes the most used bits in the center of the constellation so that energy used is lowest, but we won’t get much into it here.

The QAM-32 constellation shown above is called Cross32, and there are other options like Framed-Cross32 and Holed-Cross32.

Now we have 6 levels to send data at 448G, so what should the Baud rate be? Nyquist frequency?

By now you’ve probably guessed that the odd decimals correspond to PAM6, and you’d be right. Let’s calculate anyway. Baud rate is 448G/2.585 bits/symbol = 173.4 GBaud, and Nyquist frequency is half that, or 86.7 GHz. This means that if the loss of a copper cable is low enough at 86.7 GHz, PAM6 will be able to extend the reach of copper.

TE Connectivity showed a PAM6 modulation on a 1m DAC cable, but we will see that it is not as trivial as it seems, and additional measures will likely be required to make PAM6 work for copper. The figure below shows where the authors considered adding a re-driver in addition to a purely passive cable.

The results are shown below.

Read more

TEF 2026

TEF is two-days of panel events entirely around networking, and Ethernet is getting considerably important in the AI age. I like the idea of industry experts sitting down to talk about stuff instead of presentation talks because the unscripted nature of panels brings out a lot more insights.

Day 1 consists of the following topics, with panelists from the listed companies:

• 400 Gbps signaling over copper: TE Connectivity, Huawei, and Broadcom.

• Optical infrastructure challenges: Commscope, Panduit, Corning.

• State of multicore fiber: Panduit, Corning, Terahop, CommScope

There are also presentations from Oracle, Corning, and Meta.

Day 2 is more panel heavy than the first:

• IMDD optics: Huawei, Coherent, Cisco, Ranovus

• Coherent ethernet: Huawei, Cisco, Ciena, Nokia

• Optical packaging: Huawei, Arista, Molex, Eliyan

• Testing for AI networks: Wilder Tech, Viavi, Keysight

This event was pricey, and I wonder how much overlap there will be with OCP content. You will notice that the amount of optics content outweighs copper in the discussion topics. The future of AI networking is clearly going to light, except for some specific use cases where copper will continue to live.

OCP Global Summit

This conference is sold out, as was Hot Chips before the event. OCP has a high number of parallel events and its impossible to cover everything without an army of attendees. Look at the full schedule.

I’m going to choose two main focus areas: Networking and Power. What am I chalking out below is mostly me thinking aloud of how I’m going to approach this event. If you’re attending, hopefully this gives you a broad map of how to go about it. Usually I end up attending only 60-70% of what I plan to because I end up talking to a lot of people and also get info overloaded. I also have meetings set up with several companies too which I have to juggle (Eliyan, Credo, etc.)

Day 1 (Monday): Executive Sessions

Day 1 is entirely executive sessions from a wide variety of speakers, but the program is thankfully linear. I expect mostly high level talks, with each company trying to showcase their products. Good for high level stuff and (human) networking.

Day 2 (Tuesday): Keynotes + Sessions

Day 2 is full of keynotes before lunch, which will be another broad “sit-and-listen” session. Nothing to expect but pretty looking slides with few surprises. The afternoon session will require a balance between attending networking sessions and expo halls sessions. I’ve listed only about 5 talks max I would like to hit. I will spend rest of the time talking / walking show floor.

Networking Sessions:

• Meta — Tue, October 13, 2:30pm - 2:45pm | SJCC - Concourse Level - 210AE: Operating Linear Pluggable Optics at Hyperscale: From Link Management to Production Reliability

  • Why: I want to understand the difficulty in deploying LPO at high speeds, and what that means for DSPs.

Expo Hall Sessions:

• DELTA — Tue, October 13, 12:50pm - 1:05pm | SJCC - Concourse Level - Expo Hall Theater: Microgrids at Scale: Resilient Power Infrastructure for AI Data Centers presented by Delta Electronics

  • Why: How power is deployed drives how much downstream compute, networking, and storage will be needed.

• Credo(?) — Tue, October 13, 3:30pm - 3:45pm | SJCC - Concourse Level - Expo Hall Theater: Going Wide: Scalable and Reliable Optical Interconnects for AI Clusters

  • Why: I want to see what hardware they have, and how well it works. I am also meeting with them 1:1 earlier that day.

Power Sessions (overlap, so in order of priority):

• NVIDIA — Tue, October 13, 4:10pm - 4:30pm | SJCC - Concourse Level - 210CG: 800 VDC AI Factories: From Architecture Alignment to Industry Deployment

  • Why: Want to understand where we are in 800VDC deployments, and what the challenges are.

• NVIDIA — Tue, October 13, 4:25pm - 4:40pm | SJCC - Lower Level - LL21DE: Intelligent Power Smoothing at the Vera-Rubin Rack

  • Why: At Hot Chips, NVIDIA claimed some TCO advantages when they could smooth out power spikes.

Day 3 (Wednesday): Mission Impossible

The AI Open DC: Power Distribution and OCP Special Focus: Photonics sessions run in parallel all day. It’s impossible to do both with one meat jacket. The only saving grace is that both sessions are the same level, so easy to keep swapping to some extent. All talks on both tracks seem interesting, and would consider this the meat and potatoes day of the conference. Too many fun ones to break down individually. For reference:

• Power = Concourse Level, 210CG

• Photonics = Concourse Level, 210AE

I created a filtered timetable from the OCP schedule that has only the two all-day sessions in a PDF below. You can print it out and decide how you want to interleave if you want to follow my plan.

Ocp Wednesday Power And Photonics

25KB ∙ PDF file

Download

Download

If it isn’t enough already, and you feel like getting on the expo floor, there are a few nice options of stuff to catch:

• MEGMEET — Wed, October 14, 11:20am - 11:35am | SJCC - Concourse Level - Expo Hall Theater: Engineering the Transition to 800 VDC presented by MEGMEET

• Ciena — Wed, October 14, 3:20pm - 3:35pm | SJCC - Concourse Level - Expo Hall Theater: Extending Copper for Efficient Next-Generation Scale-Up Networks presented by Ciena

• Samtech — Wed, October 14, 3:45pm - 4:00pm | SJCC - Concourse Level - Expo Hall Theater: A Pathway to 224/448 Gbps Copper and Optical Connectivity in AI Infrastructure

• Vertiv — Wed, October 14, 4:10pm - 4:25pm | SJCC - Concourse Level - Expo Hall Theater: Future-Readiness for the 800 VDC Transition

Day 4 (Thursday): The Haunting

By Day 4, I’m usually too spent to sit through whole day sessions. I plan to haunt more talks on Concourse Level 210 like before, flitting through walls before the two halls where power and networking talks happen:

• Power = Concourse Level 230A

• Networking = Concourse Level 210 AE

Here is a printable filtered version. Would not recommend hitting everything though.

Ocp Thursday Power And Photonics

22.2KB ∙ PDF file

Download

Download

There are a few specifically interesting ones to me however.

• NVIDIA — Thu, October 15, 1:40pm - 2:00pm | SJCC - Concourse Level - 230A: MGX Power Rack Reference Design

  • Why: Good to understand how the NVIDIA reference rack is built.

• Texas Instruments — Thu, October 15, 2:45pm - 3:05pm | SJCC - Concourse Level - 230A: 800V Data Center DC/DC Design Considerations, Topology Comparisons, and Trade-offs

  • Why: TI’s talk should be pretty technical I imagine, and especially interested in what DC/DC converter options exist, and what tradeoffs exist.

There are a couple of talks worth catching on the power safety side, because safety and compliance is a layer that is not understood by most. It is critical to deploy 800VDC in datacenters.

• Thu, October 15, 8:20am - 8:35am | SJCC - Concourse Level - 210CG: The New Focus on Energy Storage Systems Safety

• Thu, October 15, 8:40am - 9:05am | SJCC - Concourse Level - 210CG: The Compliance Gap: ESS Safety in the Age of AI Infrastructure

The plan on the last day is to spend more time talking to people on the show floor, and also check out some exhibits in the Innovation Village for what startups are doing. There are some useful things to catch in the Expo Hall.

• Sumitomo — Thu, October 15, 10:40am - 10:55am | SJCC - Concourse Level - Expo Hall Theater: Multi-Core Fiber - A solution for scale up, out, and across

  • Why: Want to know more about the whole multi-core fiber thing.

• OmniOn — Thu, October 15, 11:40am - 11:55am | SJCC - Concourse Level - Expo Hall Theater: Driving Efficiency Through Innovation Across the Operating Range and Power Chain, Aligning Power Architecture With the Needs of AI

  • Why: Mention of end-to-end efficiency, and where Si, SiC, and GaN play a role.

This is already too much for a single conference, and I would be very happy if I can hit even 50% of all this content. Apologies if I miss any good content, or if your company is presenting and I didn’t add it on here. The good part is that OCP records the event, so I can always watch what I miss later on.

If you have anything else that is a must see, drop your suggestion in the comments. If you happen to see me at TEF or OCP, stop me and say hi!

The massive deployment of compute over the next decade will require deployment of enormous amounts of power. NERC puts the 10-year projected peak power demand in the US at 245 GW. However, utility buildouts however operate on different time scales than the semiconductor industry, where shifts happen in the time frame of years instead of months.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement. Each paid subscription is for one person, not a company. Group subscriptions (3+) are 20% off if you need a few subs. Reach out to me if you want an institutional seat, we’ll work it out.

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. For institutional research, contact me at SemiExponent.

The chart below from a16z for example, shows 20% of all invested AI capex flowing into power. The secular demand for power infrastructure and the high dollar content across the power ecosystem makes it more of a long-term play for investors, especially given its long deployment timeframes.

The entire gamut of the power semiconductor industry spans from the grid to the rack, and involves a variety of technologies that are difficult to comprehend in a broad sense; grid-side power electronics often deal with hundreds if not thousands of volts, while rack-side power is more “semiconductor-like” since it only deals with a few tens of volts. It requires many vendors, standards, and supply chains to work in lock-step if we were to make changes to how power should be distributed to AI infrastructure.

We’ve discussed the need to move to 800VDC power delivery before. If you want the basics physics behind it, check it out below.

Today’s GPUs consume about 200kW per rack. If we were to double the number of chips, the power problem quickly gets out of hand. Most racks today use 48/50V DC to distribute power within the rack. If the power per rack gets too high, then the rack needs to support >10,000A in its distribution system. This is infeasible, and the solution is to go to 800VDC, which would drop the current levels for the same level of power.

800VDC: AI Power Delivery Done Right

One principal NVIDIA engineer I spoke to recently was of the opinion that we may not see racks go beyond 200-250kW in the industry; he believed that the cooling and power delivery challenges outweighed the benefits. In the Rubin Ultra (VR300) era, if we are hooking up eight Oberon racks in a 576 GPU scale up domain due to networking limitations, do we really need MW-level racks in the future? Even if the Kyber rack does get deployed, will we see more companies push towards such power levels? If not, what does that mean for the 800VDC ecosystem if we stay at current rack-power levels?

In the rest of this article:

• Why 800VDC still matters even if rack power stops scaling aggressively.

• What the sidecar architecture changes inside the datacenter.

• Where the near-term 800VDC semiconductor opportunities actually sit.

• How SiC, GaN and silicon divide the power-conversion stack.

• Which companies are exposed across systems, switching, protection and control.

Read more

Agentic harnesses such as OpenClaw and Hermes were essentially for the ‘hacker-types’ who knew how to spin up VMs, secure SSH, and set AI keys to have their personal army of always-on agents. Grok Bot from xAI was the first to change that paradigm. It provided a simple chat interface that could spin up agents via an easy-to-use GUI. Grok Bot provided users with their Virtual Machines (VMs) that acted as a virtual cloud computer that the agent would use for autonomous tasks. Suddenly, agents were for everybody, not just the ones who knew how to hack it into existence.

Recently, another personal AI tool – Instinct – gained popularity in social media. Founded by Noam Shinn, Techcrunch reports that they raised a total of $350M at a $2.5B valuation. Their launch has mostly been via private invites due to limitations in compute capacity. The rise of personal agents really took root with the announcement of Meta’s Muse personal AI. Meta advertised it as a free AI agent that can get your everyday tasks done. In its first two weeks, it had nearly 3 million installs across various devices. Underlying all these personal agents were VMs allocated to each user, so that their personal agents could kick off inference requests from within them.

With the release of Muse, the market finally woke up to CPU demand primarily driven by personal agents and sent stocks of CPU makers soaring. Funda has a nice post explaining how Muse is being adopted. The month-to-date price change for INTC, AMD, and ARM were all up >25%. This price change is primarily on the basis that personal agents, if adopted by a wide user base due to its relative simplicity and usefulness, would drive CPU demand massively. We will test this assumption.

We made an early report on CPUs for agentic AI back in February 2026, which explained the need for CPUs in a new class of workloads that included tool calls, database accesses, and web-search, in addition to LLM-based inference. Personal AI agents belong to the same class of workloads but the ease of use expands the end-user base much more than agentic harnesses.

In this article, we will examine if CPU-sensitive tasks are automatically proof of explosive CPU demand, and how that ties into memory needs.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. This article is an example of the kind of institutional research clients of SemiExponent get. Reach out using the contact form for more info.

What Runs a Personal Agent?

Personal agents require virtual machines (VMs) that are computers with an operating system running on them, from where an entire task can be performed from start to finish. Each VM is allocated a share of CPUs, RAM, and storage from a larger pool of resources. For example, users prompted Muse to reveal its VM configuration and identified that it is using 2 virtual CPUs (vCPUs), 8 GB of RAM, and 100 GB of SSD storage. While actual specs are unconfirmed by Meta, this often represents a typical VM deployed on the cloud.

Take the example of an agent booking travel to maximize credit card points. A phone sends a message to a remote VM which uses a combination of CPU, RAM, storage, and GPU-based inference to complete a task. The process also involves accessing websites and logging in with credentials which needs computer use. All the data from search is sent to the LLM running on the GPU to interpret and plan next steps. The RAM stores working data including travel dates and current viable options, while storage is used to periodically store state, and outputs of the booking process. The figure below shows the data flow between various parts of hardware.

Most analysis today maps the rise of personal agents directly to increased CPU demand, which seems reasonable to the first order. Deeper nuance requires understanding how these VM resources are shared in a multi-user scenario to eventually size the physical fleet of compute hardware in a datacenter. We will cover three different VM deployment scenarios next.

How Many Agents Can Share a CPU Server?

To answer this, we must understand how the GPU and CPU work together. We know that a CPU should keep the GPU utilization high for good TCO, but the CPU utilization determines how many agents fit on a server. Although this really depends on the workload, three CPU deployment scenarios broadly apply.

1. Reserved VM: Each user has a continuously running VM with CPU capacity reserved, even when idle. The RAM is exclusive to the VM.

2. Shared CPU: Each user’s environment stays running, but CPU execution time is pooled across users as needed. RAM still holds the VM’s working memory, and only the CPU is shared.

3. On-demand VM: The environment starts or resumes when work arrives, then releases compute resources when idle while preserving saved state. CPU and RAM are shared. Every time a request comes in, the VM is spun up from the saved system state on the SSD.

Although Meta has not published their method of deployment on Muse, we can guess the likelihood each one is used.

Most likely → Scenario 2: Shared CPU. Each VM can retain its identity and state while its virtual CPUs execute when there is work. This fits intermittent agent activity.

When capacity constrained → Scenario 3: On Demand VM. If a user’s usage pattern suggests large periods of inactivity, their VM can be spun down, only to spin it up again before next use. This allows capacity to be released to others.

For special workloads → Scenario 1: Reserved VM. A dedicated VM makes sense for high availability use cases when latency is not acceptable. It will not be the default deployment case because it leaves a lot of capacity unused.

In cases 2 and 3, where the CPU is shared, there will be more agents resident on the CPU than hardware would otherwise allow. The difference between these two then comes down to resident memory usage – which is something we will discuss later. Assigning more vCPUs than physical cores is quantified by the oversubscription ratio which is defined as,

Initially, we assume no multithreading. In reality, two threads are less performant than two cores due to shared resources between them. We correct for this later by a factor between 1.5 and 2 (we assume 1.75).

For example, a 1:4 oversubscription ratio on a CPU with 128 cores and no multithreading would mean that 512 vCPUs can be allocated on the physical CPU. If two vCPUs are allocated per user, that implies that a 128 core CPU can host 256 VMs. Assuming each user of the personal agent platform gets a VM, that is 256 users. The oversubscription ratio used in practice depends on the cloud provider, and the utilization rates on their CPUs. A 25% utilization rate means that the provider can opt for a 1:4 subscription ratio.

We can extend this calculation to different server CPUs in the market today, and estimate how many CPUs are required for a million users for each one. We assume that each user gets 2 vCPUs per agentic VM, and that multithreading increases useful CPU capacity by 1.75×. From here onward, the CPU sharing factor applies after that multithreading adjustment; 4× sharing with a 1.75× benefit therefore implies seven allocated vCPUs per physical core.

Memory Constraints on Agentic VMs

The CPU calculation gives us a starting point for sizing the fleet. Each server must also have enough memory to support the environments assigned to it. An agent waiting for an inference response may consume little CPU time while its browser, operating system, and working data remain in RAM.

With a base assumption of a 256-core CPU, a 1.75× multithreading benefit, 4× CPU sharing after multithreading, and 2 vCPUs per environment, a total of 896 VMs can be allocated. If every VM actually occupied the model’s assumed 8 GB of RAM, those environments would require 7 TB of memory per processor.

If, for example, only 1.5 TB of RAM were available, at 8 GB per resident VM, 192 fully resident VMs fit before reserving memory for the host. Counting CPUs purely by core count would therefore overstate density unless the platform reduced how much memory remained resident.

This makes deployment architecture central to the demand estimate.

• A running environment can retain memory while its CPU is idle.

• An on-demand environment can release much of that memory by saving its state and resuming later.

The latter option increases the number of users the same hardware can support, but introduces resume latency and additional storage activity. Additionally, a VM might not use all of the RAM available to store its working set. Ultimately, what matters is how much memory remains occupied across the fleet, especially during busy periods.

Our CPU estimate therefore needs a second test: how many environments fit within the available RAM? Whichever resource reaches its limit first constrains deployment density. For this study, we will assume that SSD storage is not a limiting factor because a single QLC drive can provide 256TB of storage - which will allow >2,500 100GB VM environments. RAM is the fundamental limitation, not SSD storage.

From Agent VMs to CPU revenue

We can now connect personal-agent deployment to CPU purchases. Our model estimates the hardware and revenue implications of a large deployed agent population under explicit operating assumptions. The adoption path is a scenario, rather than a forecast supported by verified deployment data. We will show our calculations below.

After the paywall:

• From a billion agent VMs to CPU revenue: Our deployment model through 2030, translating agent adoption into processor purchases and annual and cumulative revenue.

• The assumptions that drive the result: How CPU sharing, hyperscaler discounts, price erosion and reuse of existing hardware affect the opportunity.

• When memory becomes the bottleneck: Why RAM constraints can force operators to buy more servers—and how on-demand VMs could ease that pressure.

• Which CPUs fit this workload best: A comparison of AMD, Intel, NVIDIA and Arm configurations, showing where core counts and supported memory are well matched.

• What this means for the CPU investment thesis: How much personal agents could contribute to supplier revenue, and what evidence is still needed to support expectations of a much larger boom.

Read more

Big mistake. As a friend put it — “We will always have fomo dude.” Instead, I’ve been living vicariously through product announcements and pictures from friends and acquaintances on texts and forums, and there have been a few important ones especially as related to optical scale-up.

Let’s cover them and their implications:

• Teradyne Iris 100 for microLEDs

• ams OSRAM thin-film VCSEL

• 🔒 Lumentum DWDM ELSFP

• 🔒 Lumentum + Qualcomm 1060nm VCSEL

• 🔒 Coherent Photonlink

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. For institutional research, reach out at SemiExponent.

Teradyne Introduces Iris 100: Production-Proven Test System for MicroLED Devices

The news:

Teradyne Iris 100 brings semiconductor-grade optical test to microLED manufacturing, measuring every individual emitter in an array and integrating directly with the Teradyne UltraFLEXplus platform for combined optical and electrical test.

MicroLED technology underpins two of the fastest-moving areas in advanced electronics, augmented reality (AR) microdisplays and optical data interconnects for AI data center architectures. Both depend on devices that contain hundreds of thousands to millions of individual emitters, each of which must be characterized with production-level precision and throughput.

"MicroLED is shifting from lab to volume production for both AR microdisplays and, increasingly, optical interconnects, which are becoming foundational to how AI data centers scale beyond copper," said Shannon Poulin, president of the Semiconductor Test division at Teradyne.

(Businesswire)

Why it matters:

Product test equipment announcements are notable because they indicate that there is a need to test microLEDs at scale. Of course, they can be for AR displays, but the real moneymaker business case would be optical interconnect. Realistically, Teradyne would have had conversations with the microLED makers (Credo, ams OSRAM, … ) to understand if there will ever be a need to go to production. Now that the product is here, the answer seems to have been yes. This bodes well for microLED interconnect overall.

That is right up until …

ams OSRAM unveils breakthrough Thin-Film VCSEL for AI Scale-Up

The News:

ams OSRAM is expanding its Digital Photonics strategy to address this emerging opportunity. Building on its leadership in intelligent light emitters, optical sensors and advanced semiconductor manufacturing and assembly, the company is unveiling key photonic building blocks required for next-generation AI infrastructure. These include addressable micro-emitter arrays based on both LED and VCSEL technologies, as well as matching micro-photodiode arrays that together form the optical foundation of future ultra-high-bandwidth interconnect architectures.

ams OSRAM continues to expand its photonics portfolio through the development of a next-generation 850 nm Thin-Film VCSEL platform specifically designed for AI scale-up interconnects.

The platform combines several differentiating technologies in a single architecture, including top-emission Thin-Film VCSELs, dense 25 µm pitch microVCSEL arrays with addressability, silicon TSV integration and compatibility with standard multimode fiber infrastructure. This enables highly parallel optical links with exceptional packaging density and ultra-low-power operation.

The company has successfully completed microVCSEL integration on silicon TSV substrates and demonstrated error-free 32 Gb/s NRZ operation. The platform achieves an energy efficiency of approximately 0.25 pJ/bit while maintaining compatibility with scalable manufacturing approaches. This level of efficiency highlights the potential of highly parallel optical architectures to support future bandwidth scaling while minimizing power consumption.

In addition, reliability testing has shown no failures after more than 2,000 hours under elevated junction temperatures and current overstress conditions.

(ams OSRAM)

Why it matters:

Apart from the stock price bump, ams OSRAM announced their entry into the world of ‘wide-but-slow’ optical interconnects. The internet has been expecting this ever since Ashkan Seyedi left NVIDIA to join ams OSRAM three months ago, but coming out with some real tech seals the deal.

Before we get all into microVCSELs, notice that the press release states that their Digital Photonics platform comprises of addressable arrays of both microLEDs and VCSELs. Clearly ams OSRAM is hedging their bets here: why pick one over the other when you have the expertise to do both. They could, after all, serve different speed generations at different price points and offer TCO options to the customer.

They describe their microVCSEL arrays as “thin-film” and have “silicon TSV” integration. I am not sure what the device structure is and what that has to do with “thin-film,” but the presence of TSVs likely means that the silicon chip is flipped onto a host board, and then the microVCSELs integrated right on top. If you think about it, this configuration requires TSVs. In addition, the flip-chip approach will reduce parasitics — which is excellent for the electro-optical bandwidth of the microVCSELs.

The microVCSELs are running at 32 Gbps/lane, which is believable speed for this technology. It is not so fast (like 50-100 Gbps) that now each lane requires heavy equalization via CTLE/DFE, which is a real problem in running VCSELs too fast because they start to require more power. Keeping speeds tame means that they can achieve 0.25 pJ/bit, but it is unclear if this involves the driver circuits and stuff too.

There is no mention of reach in the press announcement, but I am hoping it is at least 3-5m to stay competitive. There was a mention of 1,024 emitters in the array, but shortly after, the press release was edited to exclude it. It was probably a mistake because running 1,024 emitters at 32 Gbps in wide and slow would make this 30x faster than Avicena’s link. Think about it; it’s too good to be true. My guess is that they are using about 50-ish parallel links to get 1.6 Tbps of bandwidth.

Finally, there is some mention of reliability testing over temperature and electrical stress which is important because VCSELs are still lasers. Not InP lasers, which is good for the supply chain. The device is likely built with GaAs.

Takeaway: Wide-but-slow optics in either LED or VCSEL form is becoming a strong signal. Optics for scale-up can take these forms, and not just CPO/NPO. CPO/NPO will take market share, but will have to compete with these other approaches as well for scale-up.

Read more

I was particularly interested in seeing microLED tech first hand because there has been a lot of debate whether microLEDs are a viable interconnect technology in the datacenter. My purpose was to ask as many questions as I can, to see if I can make sense of whether standard objections to this technology hold up, and allow people actually building this technology to give their account first hand instead of simply inferring from papers, conferences, or opinions.

Most impressively, the entire top brass of the company – from their CEO, Marco Chisari, to the entire senior management of the company in charge of various parts of the business – hosted me and discussed various aspects of their technology. What follows in this report is my observations from the visit mixed in with my own research and opinions of where I think microLEDs are going.

Disclaimer: This article is not sponsored by anyone. The only ‘compensation’ was the bubbly water I drank at Avicena. Not that they did not have other stuff to offer, I just wanted bubbly water. My goal is to provide a neutral, factual account of where this technology is headed while being as informed as possible. If I got something wrong, reply to this email and let me know. This stuff is complex.

Avicena reviewed a draft of this article for factual accuracy. I retained editorial independence.

What’s in this article

Free section:

• Common objections and claimed benefits of microLED interconnects: lane rates, reach, fiber count, jitter, crosstalk, gearboxing

• Lane rates: where Avicena’s production microLEDs run today (3 to 3.5 Gbps), the 8 Gbps next-gen device I saw in the lab, and why there is no roadmap to catch microVCSELs on per-lane speed

• Reach: why chromatic dispersion caps microLEDs at about 10 m, why the real competitor is copper and not laser optics, and what a 300+ lane cable looks like at 5 m

Paid section:

• Crosstalk and jitter: victim/aggressor bathtub measurements on a live LightBundle link, with aggressors on and off, same and different PRBS patterns, and what the results say about signal integrity at 3 Gbps

• Gearboxing: what 64:1 versus 4:1 ratios actually cost in chip design, clock forwarding, deskew, and whether gearboxing can be removed entirely

• Physical construction and manufacturing: the transmit/receive chip, solder ball microLED integration on 300 mm wafers, molded lens arrays, fiber bundle alignment and connectors, the in-house Aixtron reactor, and the path to volume

• Concluding thoughts: the 30 m reach question, redundancy and temperature behaviour, the customer scale-in request, and where microLEDs fit against copper at 1.6 Tbps

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast, and our daily free newsletter with latest semi news. For institutional research, contact me at SemiExponent.

Common Objections/Benefits to MicroLED Technology for Interconnects

A framework I had while going into this, was to deliberately list out why people think MicroLEDs will not work for interconnects and try to address it during my visit. Here is a list of common objections:

1. Lane rates: MicroLEDs are too slow on a per-lane data rate basis and scaling to higher lane rates is hard.

2. Poor reach: The linewidth of microLED light is poor, and chromatic dispersion in the cable will ultimately limit its reach. 10 meters or below is believable, but 30 meter reach at good BER is a stretch.

3. Fiber count and alignment: A consequence of (1) is that too many fibers are required to get the lane rates to any meaningful speeds. Adding fibers is not a scalable solution. Cable itself gets bulky. Aligning hundreds of fibers to the array of microLEDs is difficult to do, especially at scale.

4. Jitter: MicroLEDs are very noisy, cause a lot of jitter and degrade the bit error rate (BER).

5. Crosstalk: Bundling 100s of fibers together means that there is a lot of crosstalk. Not too many critics clarify what they mean by crosstalk. There are two kinds: (1) between optical fibers carrying light, and (2) between different electrical lanes at the two ends of the cable that have to feed the hundreds of lanes.

6. Gearboxing: The SerDes on the IC side provides 8x200G lanes, for example. Converting this to say 400x4G lanes requires a gearboxing chip that is difficult to do.

The benefits stated for microLEDs are low power, low cost, better thermal performance, better availability compared to lasers, high bandwidth densities, simple silicon integration and more reliability stemming from redundant links in the array.

We’ll discuss these in detail next.

Lane Rates

Avicena’s microLEDs are built on GaN technology, and emit blue light in the 420 to 440 nanometer wavelength range. Their current production process has a data rate of 3-3.5 Gbps per microLED. The chart below shows the evolution of data rates in microLEDs across industry and academia. These are raw lane rates only, with no consideration for reach or BER.

Avicena’s next gen microLEDs are still in the lab, but I got a chance to see it in action running at 8 Gbps. The important thing to understand is that this speed is not through fiber, but instead a very short free space link in a lab. Think about shining the microLED right into the detector kept a really short distance away. This approach removes the impact of the optical fiber and shows just the data rate of the microLED. Here are some pictures I took.

Here is the pulse pattern generator running at 8 GHz.

The eye-diagram from the scope runs an NRZ TX-RX short link over free space, which looks pretty clean. Jitter is quite low, and the eye is wide open.

The BER measurement shows better than 1e-11, which signifies that the eye is clean. Again, there are no channel impairments here and the results are free of chromatic dispersion which will limit reach at good BER.

In the chart above, the Avicena data point showing 10 Gbps is an interesting one (doi: 10.1109/LPT.2025.3552708). In this work, they drove the microLEDs with an external driver circuit instead of an integrated one, and were able to show bit rates of 10 Gbps per lane, with 1-e10 BER. Avicena also stated that they’re developing a next generation optical engine that will support 8 Gbps links with LEDs and PDs integrated onto a new ASIC.

The landscape of microLEDs says that lane rates will stay in the few Gbps range for a long time. There are companies who are trying to optimize microLEDs to be suited to communications; one example is Palomino Labs, who are trying to engineer the device structure to produce a narrower cone of light from a microLED. The key takeaway is that there is no roadmap to become competitive with microVCSELs (which run quite easily at 50 Gbps) on per-lane speed alone and the focus is to deliver higher bandwidth density with more lanes at smaller microLED pitch sizes.

Subscribe now

Reach

The linewidths of microLEDs are wider than those of lasers. This wide linewidth causes light pulses launched into an optical fiber to get wider as the light pulses travel down an optical fiber. This is called chromatic dispersion and it limits the reach of the optical link because the pulses start to overlap. Pulses from a narrow linewidth light source can go much farther down a fiber before bits start to overlap each other. A wide linewidth source like MicroLEDs have bits starting to overlap in a much shorter distance. Hence, the reach of MicroLEDs is inherently limited.

We’ve discussed this in quite some detail in the free article below if you need more background.

Too many people equate microLED interconnects to optics for scale-out applications where the reach is over 10m. This is partly a problem with companies developing this tech too, because they advertise 30-50m reach which draws comparison to laser-based optics. This is the wrong way to think about it and I wish companies would stop overpromising on technology that has not demoed on a showfloor yet.

The real high-value use cases for microLED interconnects today cover reaches up to 10m, until somebody shows solid technical measurements to prove otherwise (Credo at OCP: I will be there, bring your wares). The real competition is copper, not optics, as shown below.

The reach of an active electrical cable (max 10m), at the power consumption of a DSP-less active copper cable (1-3 pJ/bit).

It is clear how microLED occupies an important and attractive tier in the close-reach interconnect space. Especially as the scale-up domain is going multi-rack in the Rubin Ultra and beyond, MicroLEDs provide a solution to connect GPUs both in-rack and across-racks, at an energy-efficiency that is attractive as world-sizes go up.

The setup below shows Avicena’s LightBundle™ which they sample to potential customers along with their software platform shown on their screens. The boxes have transmit and receive circuits, and are used to test the microLED link between them. They have internal waveform generators if you want to use data patterns out of the box, or the front panel has connectors through which customers can provide their own waveforms for testing the optical link.

The length of the cable in the middle is approximately 5m in this test, and the diameter of the cable having 300+ parallel links is 3mm. The idea that a bundle of fibers makes the actual cable unwieldy is untrue.

The screens show transmitter data on the left, and receiver data on the right. Each of those hexagons on the screen represent a microLED lane carrying ~3 Gbps of data. You notice there are some blue dots on the grid – those are the points where the link is performing poorly, where the BER is higher than some threshold. The bottom of the screen shows a color scale from green to blue: green = good BER, blue = bad BER. We will see some measurements with this setup next.

Read more

The TL;DR Version

• AI could significantly improve human life but has serious risks such as loss of control, misuse for cyberattacks and bioterrorism, and economic disruption.

• Dario proposes ‘pacing the frontier’ of AI development, which means slowing the rate of capabilities advancement to allow risk prevention and alignment research to keep pace.

• A key concern is the accelerating pace of AI advancement, driven by AI’s growing ability to build the next generation of AI through recursive self-improvement (RSI), which could outpace human understanding and control.

• The OpenAI-Hugging Face incident, where AI agents acted as a fanatical collective and conducted unauthorized cybersecurity attacks, highlights the potential for catastrophic damage if AI systems are misaligned and possess greater capabilities.

• Dario suggests a three-step plan to pace AI development:

  • Embedded third-party evaluators for verifiability and transparency

  • Democratic coordination among companies in democratic countries to set safety standards

  • Global coordination with authoritarian governments.

• Anthropic is unilaterally committing to embedding third-party evaluators with employee-like access to verify safety practices and report incidents.

• Pacing within democracies must consider maintaining AI leadership over authoritarian regimes like China to prevent national security risks, suggesting measures like restricting chip sales and cracking down on unauthorized distillation.

• Dario suggests four levels of pacing, with each higher level being increasingly difficult to adopt.

  • Level 1 — Prohibit obviously dangerous uses of AI like use in bioweapons.

  • Level 2 — Ensure models don’t pose acute risks to cybersecurity or incentive alignment.

  • Level 3 — Impose a speed limit on RSI, slowing the rate of using AI to improve AI.

  • Level 4 — Massively ‘pace’ or even pause AI development, with a high risk of noncompliance from global powers.

Amodei has always been a proponent for AI safety, but this occasion finds the key players coming together on this issue, which is rare. Sam Altman, Elon Musk, and Demis Hassabis have all said that we should consider safety as an important tenet in AI.

David Sacks’ X post drew in a lot of kudos, where he says that Dario and Sam are the two people in the world who have seen past the current frontier of AI than anyone else. If they both feel the need to slow the pace, then they should just go ahead and do it. Trading, as Sacks puts it, “raw power for reliability and predictability” is more of a product decision than pure altruism.

Where does that leave the AI buildout?

Pacing the AI frontier could imply that lesser hardware will be deployed going forward in large training clusters, leading to lower future capex deployments. The market has been searching for a slowdown in the AI buildout, and Amodei’s essay could be construed as a catalyst. In reality however, this pacing will have little to no impact in the medium term on the AI hardware buildout, and there are a few of reasons for this.

First, it is nearly impossible to get everyone aligned on the issue of safety to the point of actually deploying lesser compute. Anthropic is the fastest growing company in history, and slowing down model development in the interest of safety presents too high of a risk of handing the lead to competitors.

Second, open source Chinese models have the near same intelligence at the frontier and as Amodei himself admits, presents a high risk of noncompliance at the global scale. Slowing development would pose too high of a national security risk that the US government will downplay the risks of AI.

Third, limiting the use of RSI for AI development is a self-defeating strategy for the AI industry. In a world where every other industry is using AI to get productivity gains, asking AI researchers to limit AI use would slow down development in a world where noncompliant players will continue to use RSI.

Finally, AI has still only found immediate uses in coding and math, where it performs well. AI is competent in verifiable domains, but there is a long tail of industries in non-verifiable domains where AI is still to make a sizable impact. Industries where no training data exists, information is tacit, and resources are limited, are still ripe for disruption by AI. We do not have the compute to supply intelligence to the long tail — not by a long shot.

We are nowhere near peak compute

Inference is the ultimate consumer of AI infrastructure because it can scale to the population of a planet. Training will remain restricted to the knowledge few who know how to train a model. Not all useful applications of AI need frontier models to begin with.

As an example, the recent rise of personal AI assistants (Grok Bot, Instinct, Muse) is just now making AI accessible to the masses — where the regular users does not need to set up virtual machines, secure access, and deploy api keys to agentic harnesses. AI assistants use messaging apps and are genuinely helpful. An always-on agentic AI world does not require frontier models, and there is still a lot of useful intelligence left for the rest of us.

All of this needs compute, and lots of it.

AI safety is important, and we must solve it

Today, we trust that all our hard-earned money is safely stored as bits somewhere, waiting to be transformed into tangible articles of comfort when we desire. We have evolved to this level of trust over time — with policies, safety practices, standards bodies, and institutions overseeing that financial transactions are trustworthy. We don’t lose any sleep over it.

AI will eventually get to the same level of trust when AI security experts build similar infrastructure so that everyone in the world will be comfortable using AI. Getting to that point takes time. It is not by slowing the frontier of AI development, but by deploying more resources where it is necessary. Burning compute on 10,000 agents to solve the Navier Stokes equation is an admirable feat that showcases the capabilities of AI. Instead, perhaps AI can help accelerate the safe deployment of AI.

It is a matter of where a finite pool of energy, time, and money should be allocated. Safety is definitely first priority at the moment, if only so that Anthropic and OpenAI can put out better products for their paying customers to use. Product quality and competition is what will effectively drive AI safety, not altruistic motives to save humanity from the 10% chance that AI will kill us all.

In Part 1 of this series, we built up the fundamental physics of ultra high power (UHP) lasers. We concluded by identifying the four walls that make it challenging to implement a UHP laser: managing heat, maintaining narrow linewidth, getting light out, and catastrophic optical damage.

In Part 2, we explored how Lumentum overcame these challenges via their 2007 paper as JDS Uniphase, and via their later 2022 CLEO paper on UHP lasers. We also looked at how defensible their high power laser platform is.

In Part 3, we will look at the use of a Semiconductor Optical Amplifier (SOA) in conjunction with a lower power laser (~100 mW) to generate UHP laser power levels of ~400 mW. This architecture is called the Master Oscillator (the laser diode), Power Amplifier (or the SOA) approach – aka, MOPA. We will contrast it to the single-cavity UHP laser approach to highlight the pros and cons of each.

Contents:

• The MOPA Concept (free)

• Integrated Semiconductor Optical Amplifiers (SOAs)

  • Operating Principle

  • Amplified Spontaneous Emission (ASE)

  • Back Reflection into the SOA

• MOPA High Power Lasers

  • Sumitomo’s SOA-Integrated DFB Laser

  • Nanjing University’s 12-Channel MOPA Array

• Single Cavity vs MOPA: Lumentum vs Sumitomo

• Who is Actually Shipping?

• Which Bet Wins

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast and our daily free newsletter with latest semi news. Reach out for institutional research at SemiExponent.

The MOPA Concept

The idea is simple: instead of making a single-cavity high power laser that is prone to issues like spatial hole burning and optical damage like we’ve discussed in previous articles, use a low power seed laser, and then put a power amplifier after it to boost up the power.

Source: Coherent

Example: A MOPA UHP laser can run the MO at ~100 mW and have the PA multiply that power by a factor of 4x, or 6 dB, to produce a total of ~400 mW at the output.

It is far simpler to make a low power laser with good spectral qualities due to the photon field in the cavity being low. A lot of the issues we discussed earlier in making UHP lasers are greatly mitigated at low laser power. Most laser manufacturers can make them at high yield.

The challenge in MOPA lasers comes from designing a power amplifier (SOA) that is good enough to boost the power to the levels needed for CPO, without degrading the spectral properties of the signal. High power amplification, whether it is radio waves or light, is no laughing matter.

When amplifiers are driven to deliver high power, they unintentionally degrade the quality of light output unless they are designed well. Having two components instead of one introduces other problems: (1) the interface between the two matter, and (2) the combination of the MO and PA should work reliably at high yield over a range of temperatures.

The concept of MOPA amplifiers is not new as they have been widely used in precision cutting, lidar and medical imaging. Those systems use discrete laser and amplifier components, along with isolators, focusing lenses and mirrors. The isolator exists to prevent the reflected light from the input of the PA from affecting the operation of the MO. This is important to remember when we discuss integrated MOPA lasers. Fibers doped with rare earths like Erbium or Ytterbium perform the power amplification, which themselves require pumping with lasers to amplify light. Mirrors and lenses guide the light across the system.

Source: em-smart.com

Discrete MOPAs are too bulky for use as light sources for CPO. They will need to be integrated into the same InP substrate to deliver a compact light source. We will discuss integrated SOAs next.

Read more

The industry is already treating capacity as a problem to route around rather than stack for. At Hot Chips, the sentiment among attendees was that high HBM stacks are too expensive and possibly unnecessary. Stacking higher was the brute-force approach but as memory costs rise, hardware builders are looking elsewhere for performance. This leaves memory makers with one remaining source of upside: 16/20-high hybrid bonding, the know-how only they hold. The despec trend is a direct risk to it.

This note covers:

1. The hard road to HBM bandwidth: A look back at the rocky path to achieving high pin speeds on HBM4, and what that has taught us about memory companies.

2. GDDR vs HBM: Memory companies know how to push GDDR lane speeds, but why HBM is different, and far more difficult.

3. NVHBM implications: Why things are changing when design experts on logic nodes take over critical parts of the HBM stack.

4. Where the value accrues: The rise of custom base die shifts where the value accrues in HBM, and where the pennies fall between memory companies and XPU builders.

5. Signals to watch and potential winners: The signs are already here, but what to watch for next, and who might emerge winners in the custom base die era.

By reading this post, you agree to the terms and conditions. See the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news. For institutional research, contact SemiExponent.

The Hard Road to HBM Bandwidth

A keen observation that Tanj Bennett () from SemiAnalysis made at Hot Chips was that memory makers are not concentrating on increasing bandwidth in HBM, but instead insist on stacking DRAM die. HBM4 already has 2,048 parallel data lanes, and packaging constraints makes it difficult to double that in the next generation. The better option is to increase the speed per lane. We have seen pin speeds in HBM4 being pushed from the 8 Gbps JEDEC spec, to 10 Gbps and beyond. HBM4e even pushes this to 16 Gbps.

Source: Samsung, Hot Chips 2026.

However, asking a memory company to push lane speeds is like asking a penguin to fly. Memory companies have had boatloads of trouble with high-speed PHY design for HBM. Here is a brief history:

• In the HBM3e era, Samsung struggled to pass qualification with NVIDIA due to a combination of yield, electrical and thermal issues. Things have notably turned around for Samsung when it comes to HBM4. Their in-house logic expertise at 4nm node provides an advantage in making high quality base die. They claim that they can push the speed to 13 Gbps per pin due to this choice.

• SK Hynix has had significant problems with HBM4, and have allegedly caused delays in Rubin shipments. They used TSMC’s 12nm node for base die design in HBM4, and that choice has held them back from cutting edge lane speeds. It should however be sufficiently fast for current HBM deployments. SK Hynix is reportedly teaming up with Intel as a dual-source for next gen HBM base die. Our guess is that they will use Intel 3-T (Intel 3 with TSV) initially.

• Micron has visibly been lagging at HBM4 due to their choice of DRAM-process node for the base die instead of a logic node. SemiAnalysis reported earlier this year that Micron’s share of HBM in Rubin would be zero for the first 12 months. This view seems to have changed since, with Micron getting qualified on Rubin along with SK Hynix and Samsung.

According to Counterpoint research, the current global HBM market share is led by SK Hynix, followed by equal shares between Samsung and Micron. SK Hynix has been NVIDIA’s largest supplier, and Samsung is gaining in Rubin allocations.

The market equilibrium HBM4 has reached is a consequence of fierce competition, delays, and fears of market share loss, where the supplier-buyer relationship has been tested several times over. Getting more HBM bandwidth means that amateur hour is over, and the value is going to tilt in the favor of those who know how to design high speed PHY links with logic nodes.

PHY Design: GDDR vs HBM

While memory companies have dealt with fast interface speeds in GDDR products, the difficulty in HBM is next level. It is important to understand the physics at play.

• Bus width: GDDR7 relies on a 32-bit wide bus running at 48 Gbps per pin, but the fewer wires compared to HBM means that crosstalk is lesser, and heat is more distributed over the motherboard. HBM4 runs a 2,048-wide bus where the crosstalk between wires is much higher, and thermal dissipation is challenging.

• PHY design: The circuits that push bits across a bus are substantially larger if they need to run fast, and consume more power. Circuit techniques to recover signal integrity at fast data rates are required like multilevel signalling, Decision Feedback Equalization (DFE) and Continuous Time Linear Equalization (CTLE), which takes physical space to implement (note: we explain this in simple terms on the Semi Doped podcast episode on Astera Labs retimers, if you are interested). Most of these techniques have become important at HBM3e and beyond as vendors push lane speeds, but are still vastly simplified compared to GDDR7.

The table below from Rambus shows the differences between GDDR6 and HBM2 design, as an example.

The need for HBM to work with XPUs in an unforgiving thermal environment while having strict performance requirements imposed by the likes of NVIDIA makes it challenging. While GDDR memory has been put into consumer graphics cards, the sheer scale of AI accelerators makes it a different problem – one that memory companies are not best poised to solve. Building faster PHY blocks in HBM for AI accelerators is arguably out of many memory companies’ core competence. Yet, this is exactly what the industry needs: better design of circuits on base die to push the performance.

NVHBM: Amateur Hour is Over

NVIDIA recently announced NVHBM, which they describe as:

“a custom HBM base-die technology designed and validated with leading memory vendors that enables increased memory bandwidth, better area savings, and lower power consumption.”

The implication of this announcement is much more significant than industry commentary realizes. As one X comment rightly put it, “base die is the Trojan horse into the memory industry.” So far, NVIDIA has had to live with the base die design choices made by memory companies, some of which are highly questionable as we discussed earlier. NVHBM is NVIDIA’s way of taking destiny into their own hands.

NVHBM moves the memory controller from the XPU to the custom base die, while the PHY interface itself is smaller on both the base die and the XPU. This picture from Samsung conveys how a good PHY design saves space on both the XPU and the custom base die.

NVIDIA claims that NVHBM provides improved performance. Here is a table from their technical blog.

Source: NVIDIA

A 30% higher memory bandwidth than HBM4e, which would put the per pin rate at >20 Gbps while having 15% lower power usage. The smaller PHY design frees up additional space for more compute cores in the XPU die. NVIDIA’s blog shows how this works in a custom ASIC accelerator.

Source: NVIDIA

NVIDIA has actually published research in the Journal of Solid State Circuits on high-speed die-to-die interfaces where they demonstrated 25.2 Gbps/link at an energy efficiency of 0.19 pJ/bit over a 1.2mm link on TSMC 5nm. We won’t go into the technical details here, but the marketing specs around the NVHBM announcement seem feasible given their published work.

A custom base die design in the hands of a circuit design company such as NVIDIA, Broadcom, AMD, Marvell, or similar, would bring the world’s best PHY designers to the problem of increased HBM bandwidth. The co-design of the custom base die with the XPU opens up possibilities to push the performance even higher – a path we are just getting started on. If you want to understand more about what custom base die enables, see my free post from the Hot Chips memory tutorial below.

Where Value Accrues in HBM

This question can be answered by examining where the boundary between HBM and XPU lies, and who is best positioned to provide what piece of hardware.

Beyond the paywall: where the value splits between base die and DRAM stack, why low-height HBM commoditizes, where memory makers' pricing power comes from, and who wins.

Subscribe now

Read more

In this quick note, we will cover presentations from the big three memory players. These are only initial captures from my notes. There is a lot to think about regarding implications and what the proceedings mean for the roadmap, but that takes time.

Contents:

• Micron

• Samsung

• SK Hynix

This is a free post. I hope you enjoy it.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

Micron: We’re so important, we make HBM

This talk was definitely the most unimpressive of the big three, so let’s get it out of the way first. Most of the talk was about the regular culprits like the memory wall, standard roofline curves, and how stacking enables high memory bandwidth. For an audience tuned to AI (aka, most of the lecture hall), this is not news. Only a few things stood out, and we’ll mention those quickly.

The slide below was one of the more informative comparison slides between the different HBM generations. Most tables like these only cover the overall memory bandwidth, capacity, and stack height. This one compares number of channels, pseudo-channels, burst length, processor controller (PC) width, and bitrate per lane, in addition to the usual numbers. The level of detail is admirable. We will not get into what they all mean here, but the main thing to note is that HBM has a lot more technical metrics that matter to performance than meets the eye on marketing slides.

We have discussed why HBM is so hard to manufacture in an earlier post. Of note was the fact that HBM is the 2nd most important root cause of training-run interruptions, apart from the GPU being faulty itself. Software and networking were #3 and #4, respectively. You can read the whole study here.

Notably a few things were absent from Micron’s talk:

1. Scaling to higher stacks (16-hi/20-hi) and the future roadmap for hybrid bonding.

2. Any mention of custom base die, and why that is important going forward.

3. Cooling solutions as HBM lane rates get faster.

At the surface, this might seem like a downside. Given the recent rumors of HBM stack height downgrades, it is questionable if some of these are even required in the future. Except custom base die. That is important, and will continue to exist in logic node implementations — I argue even in lower stack height implementations. Samsung will explain why next.

Samsung: Custom base die on logic nodes because we can

I enjoyed Samsung’s talk so much that I live tweeted the key takeaways from the conference. We’ll briefly expand on these topics here.

Interestingly, they had some charts showing how HBM5 would evolve in terms of spec. Even though nothing is set in stone, it’s fun to see charts up and to the right.

HBM5 with a capacity of 60GB and 6 TB/s bandwidth would be nice. Let’s calculate some numbers here. 6 TB/s with 2,048 lanes would mean 23.5 Gbps per pin. We are currently at 16 Gbps state-of-the-art, and that is a significant step up that is likely possible with an advanced logic node. For 60 GB capacity with 3 GB per die, that would put the stack height at 20 high.

In a separate slide, Samsung confirms that HBM4E is running at 16 Gbps per pin. HBM5 calculations we did above are also validated in the chart below.

Samsung’s clear advantage is that they have a logic node in-house unlike Micron and SK Hynix who need to partner with somebody to get it. They were sure to play up their advantages here by explaining how custom base die is important going forward. Samsung is using 1c node for memory and 4nm node for logic. It results in a significant power reduction in the base die. The slide below is a silent dig at Micron who used DRAM nodes for logic die in HBM4.

What was even better was how Samsung broke up the roadmap into three clear phases. We’ll cover each one.

Phase 1: The Low Hanging Fruit

When there is a logic node in the base die, the physical footprint of the HBM PHY block reduces in area and also becomes more energy efficient. This means that the block that works on die-to-die connects between HBM and XPU gets smaller. This have two positives:

1. Increases the area in XPU for more compute

2. Increases available area in HBM base die for more advanced functions.

The downside of the shrinking size is more thermal density in these regions. It was interesting that they provided actual sizes of their PHY blocks in various HBM generations.

Samsung also mentioned that they were not inclined to use UCIe for their D2D link because of larger size and poorer energy per bit. They said that their custom PHY implementation was better in performance. When moving to HBM5, they showed the concept of an integrated Heat Path Block (HPB) to cool the D2D PHY region.

The next big advantage is memory controller offloading which can be moved from the XPU to the custom base die. This reclaims area in the XPU die which can then be used to drive up flops.

The new benefit of having a logic node in the base die is that you can have SRAM. If a particular cell in the DRAM stack is broken, the information in that cell can instead be stored in SRAM and the memory controller will look up SRAM for those cells instead of failed DRAM cells. Think of SRAM as a spare notebook which acts as a scratchpad to hold data that could otherwise not be held in defective DRAM cells.

Phase 2: RAS, Test, Extension, Processing

The use of a logic node for base die means that silicon area used for a lot of these functions is smaller if the transistors are smaller. The extra space available is useful for a few things like Repairability-Availability-Servicability (RAS), testing, and collecting telemetry data about the health/status of the HBM memory stack. HBM telemetry data will be very useful given that it is the #2 thing to kill a training run.

If there is even more silicon area left over, then one can build another memory extension controller to connect another HBM stack behind the first one or put in a DRAM chip to extend memory.

Finally, if logic transistors are available, then why not throw in some compute within the base die too? The benefit is that some compute like matrix compression or encoding can be done on the logic die without ever leaving this base die. This has latency and efficiency advantages.

If that all works out, then one can in theory construct the nice little accelerator on the bottom right. It is easy to draw block diagrams like this, and conceptually nice to imagine, but its real performance benefits are questionable.

Phase 3: Going vertical

The final phase is use the custom base die + HBM and stack it on top of the logic die. The distance between compute and memory is short, and thus bits need to be shuffled across much shorter distances which gives power efficiency benefits and increased memory bandwidth due to the ability to use more parallel data paths across the chip area rather than just the beach front.

None of this is easy, but it is important for a company to have a roadmap ahead to improve technology. It proves that they have a path to an improved product in the future, and that is good to see in a company. Whether the market is ready to uptake that product is a different argument.

SK Hynix: MR-MUF advantage ends with Hybrid Bonding

SK Hynix’s performance advantage in HBM has always been their use of Mass-Reflow Mold-UnderFill (MR-MUF) technique which provides them with a unique advantage over Thermo Compression Bonding (TCB) we have discussed in detail in the past on this newsletter.

The slide below from SK Hynix highlights this advantage.

In fact, they have no problem in boldly claiming that MRMUF method works well for 16-high HBM, but that it would require 775 microns of overall stack height. The JEDEC spec was actually revised to support the larger height, so it does add up.

The question is: where does this MRMUF advantage go in the era of hybrid bonding, assuming that we ever get there and there is still a need for 20 high HBM.

Note that there is a chance that we will never get there because each DRAM wafer used for HBM purposes sucks up 3-4 times more wafers per bit. Given that the cost of memory alone is well above half the overall cost of a rack, the industry is trending to rely less on HBM if possible. This is where the news of de-specing HBM from 12-high to 8-high, and possibly even 4-high for some SKUs is coming from.

The slide below shows the benefit of MRMUF over TCB, where the thermal resistance is lower, and consequently results in better removal of heat from the middle regions of the stack. When hybrid bonding is used, the “secret sauce” of SK Hynix with MRMUF is no more, and everybody’s hybrid bonded HBM falls to the same level. If it comes down to this, it comes down to execution.

The thermal management slide from SK Hynix below has an unusual comparison table that shows how Micron is the only one without an external heat removal solution. Samsung has Heat Block Path (HPB), and SK Hynix has Integrated Cooling Enging (ICE). Micron is just going to “improve circuit design” to get there. If that were the case, Samsung and SK Hynix would be doing it too instead of external cooling blocks.

So it is either that Micron is behind on its next generation development, or has concluded that HBM will never reach those levels of stack height or per lane speeds that would require the development of advanced cooling solutions. It is not clear which one it is at the moment.

Lastly, there is an interesting mention on Intel’s EMIB on SK Hynix’s slide, noting that there are some customers using their memory with Intel’s packaging.

Final Thought

Even if stack heights do not go to 12, 16, or 20 in production environments, the use of custom base die on leading edge logic nodes is going to become important. It enables a higher data rate per pin, and we can see it being used even with lower stack heights due to all the advantages that Samsung pointed out.

The use of custom base die is, in a sense, decoupled from the stack height because it materially provides a plethora of advantages over a standard base die developed in a memory technology. In this regard, Samsung has the advantage in this playing field due to the availability of in-house advanced logic processes.

Lastly, I really love beautiful 3D renders of HBM sitting next to GPUs. SK Hynix’s look the best.

The major themes at OCP APAC 2026 orbited around how power constraints are driving technology innovations, and how those problems can be solved in the networking and power domains. We will cover the networking aspect at this conference to keep tabs on the copper versus optics transition, and closely monitor developments in optical technology.

Key takeaways:

• Copper will continue to dominate within the rack at 200G/lane speeds. 400G/lane PAM4 seems challenging in copper; either PAM6 will be needed, or the transition to optics with NRZ/PAM4.

• Pluggables and integrated optics (CPO/NPO) will continue to exist as datacenters use increasing optical content. The use cases will closely dictate which approach is best suited.

• LPO has its challenges at high lane rates, and the DSP plays a key role in telemetry capture. Link data is important at scale, and underscores the importance of DSPs in the communication link whether it is a separate component or part of the host silicon.

• XPO is being looked at for 400G/lane; simulation results shown by Arista, and measurement results are expected soon.

• Power delivery at 800V is rolling out and sidecar implementations are simpler to certify. SSTs are the end-game with its own set of challenges; expected past 2030.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news. For my institutional work, check out SemiExponent.

Table of Contents

• Power Constraints as a Technology Driver

• State of the Copper Interconnect

  • Copper lives on within the rack

  • The Copper Wall at 400G/lane

• State of the Optical Interconnect

  • Panel: CPO vs LPO vs Pluggables

    • Discussion about CPO and lasers

    • Discussion about GlobalFoundries’ Photonics

    • Discussion about Lane Rates/Modulation Speeds

  • Challenges in Implementing LPO

• XPO form-factor at 400G/lane

• Panel: AI Power Delivery - 800V Insights, SSTs

Power Constraints as a Technology Driver

A constant theme across the conference was the need for power and cooling technologies as compute scales in the future. Networking and power delivery discussions were targeted towards improving efficiency and lower component power across the AI stack.

Nearly every keynote in the conference started with power as the underlying motivator.

• Microsoft (Gohar Waqar/Sushant Gupta, CVP Cloud Hardware): Investing in solid-state transformers (SSTs) and advanced cooling solutions.

• Nvidia (Gilad Shainer, SVP Networking): Industry is converging towards 800V power delivery, and is improving power cost of networking by pushing co-packaged optics across scale-up and scale-out.

• Arm (Michael Wong, President Taiwan): Massive demand for GWs, and supply is nowhere near enough. Agent workloads swamp CPU demand, and Arm AGI CPU provides a low TCO solution.

• AMD (Ravi Kupuswamy, SVP): Power and cooling must be built from the ground up. Open standards are key to composable datacenters that can be optimized across a variety of axes, finely tuning compute, power, and networking to suit the workload.

• NTT (Masahisa Kawashima, AICC): Power across large datacenter campuses will drive smaller buildouts, and they will need scale-across networking to unify their operation.

• Google (Greg Moore, Cloud Platforms): Mentioned they are building a low-voltage direct-current (LVDC, another way of saying 800V systems) datacenters, and showed the latest developments from Project Deschutes, their coolant distribution system. Project Brazos is a liquid-cooled sidecar solution that will bring liquid cooling to currently air-cooled datacenters in operation.

• ASE (CP Hung): Advanced packaging will be important to bring power delivery to the point-of-load. Vertical power delivery solutions integrating conversion at the point of compute contributes to efficiency improvements.

A few of the keynotes addressed themes outside of power as well; both Applied Materials and TSMC noted that system, not component optimization will drive future design principles. Broadcom reiterated that the network is the computer, and that open ethernet fabrics are the way forward.

State of the Copper Interconnect

The industry is at a fork in the road where copper reach is becoming questionable as lane rates increase. The event had many optics-centric discussions, but there was talk of copper continuing to exist as long as physics will allow. We will discuss where copper is headed.

Copper lives on within the rack

The SemiAnalysis view of copper interconnect is that it will continue to exist in the scale-up domain across Rubin Ultra (RU) NVL576, Feynman NVL1152, Trainium 4, and TPU v9. We should clarify what scale-up means in this context: intra-rack will remain copper, while cross-rack will likely go to integrated optics. Few additional points were mentioned:

• Trainium 4 and RU NVL576 will both use copper within the rack and use NPO to scale-up across racks. RU Oberon racks with 2-die RU will use copper within the rack and NPO. The industry remains divided on the state of 4-die RU and its use within the Kyber rack.

• TPU v9 is additive to the copper backplane with copper actually coming in, instead of being replaced by optics.

SemiAnalysis sees an increase in the amount of scale-up networking content if the HBM is de-speced going from 12-hi HBM4E to 8-hi HBM4 in RU NVL572. The decrease in the memory content per rack drives the utilization of a higher number of racks, which in turn increases the networking content. Some numbers were shown in the de-spec scenario: if memory cost goes from 41% to 28% of BoM, the cost is reallocated to scale-up networking which goes from 4% to 12%. This is visually represented in the rightmost two columns in the chart below.

We believe that copper interconnects will remain within the rack for the 200G/lane generation (uni- or bi-di). The reach of copper is 1-2m at this speed, and is sufficient to connect the compute and switch trays. There is no need for optical upgrades if this reach is sufficient for the architecture in use, unless the speed gets higher.

We covered this in the previous OCP APAC note in some more detail.

Share

The Copper Wall at 400G/lane

As a rough estimate, when lane speeds double, the reach of copper interconnect halves. By this measure, at 400G/lane, intra-rack connections will no longer be possible with copper alone; the copper wall becomes real. However, this will become an issue only in the 3.2T era of networking which is at least two years away from early production deployments. Astera Labs pointed out that the use of DSPs to fix reach issues in scale up will add unacceptable latency and break power budgets. The transition to integrated optics is the only way going forward.

Read more

Michael Genovese, Senior Research Analyst, Rosenblatt Securities

It is really good to see the expectation for CPO laser revenues in the fiscal second quarter, because that would imply that you are either qualified or have line of sight to qualification and would kind of go against some of the fuzz, I think that is not coming from Wall Street, but more coming from Substack saying you guys are having trouble with that laser. Just any more color on the confidence that you will be ready for the customer to actually recognize revenue in that quarter and just progress that has been made recently.

Most people believe that the “Substack” being referred to here is who has been quite vocal about the Coherent’s lack of leadership in UHP lasers for CPO. You can see his recent earnings commentary here with a strong challenge to Coherent to present their laser data to prove to Wall Street that they are a viable InP laser alternative to Lumentum.

Irrational Analysis

Coherent Q4 FY26 Earnings

Irrational Analysis is heavily invested in the semiconductor industry…

Read more

2 months ago · 113 likes · 5 comments · Irrational Analysis

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

This signals that Wall Street is becoming increasingly entrenched in industry commentary on Substack (this publication included). Earnings calls are no longer just financial numbers and promises. They are becoming events where company leadership needs to start bringing receipts to their claims, or risk getting called out on it by analysts who are increasingly technically savvy. These kinds of questions on earnings calls are only going to increase.

I have argued in the past that the know-how of building UHP lasers has existed for a long time, in the deep-dive articles below. These articles walk through step by step on how to design a UHP laser, and the information broadly exists for decades in the public domain.

Sure, there are refinements to the laser design that adds up over time to create performance advantages, but fundamentally, the challenge lies in making these devices at scale. There are too many things that can go wrong when hundreds of milliwatts are output from a laser. Knowing how to do this at scale is the real competitive moat.

From a pure performance perspective, does Coherent have a “good enough” CPO laser that will work? Their UHP laser specs match Lumentum’s at least on their website. Even if they don’t have the absolute best laser, can they build one that works reliably, at high volumes, while being on time?

I do believe Coherent can take some steps to build investor confidence going forward. If they would like to stop people from making guesses, then some technical data needs to be published sooner rather than later.

Showing Linewidth Data to Investors

To the first order, a lot of doubt can be alleviated if Coherent simply showed linewidth data. If a sub-500 kHz linewidth laser is what is being advertised, then that data should be easy to show. But that data point has been noticeably absent. Publishing linewidth, relative intensity noise, and mode-stability over a range of temperatures and electrical conditions should silence critics quite easily. There is no need to even publish the latest iteration of the laser, but show a “good enough” laser from a few years ago that is in the ballpark. This protects any strategic advantages going forward.

The danger of this strategy which likely worries Coherent Investor Relations is that anything short of what Lumentum has published would put them squarely at a number two position (possibly lower), and that messaging is not acceptable. In a way, this gives credence to what the critics have been saying all along and could become a dangerous narrative that is highly likely to be misconstrued on Wall Street, or even by technically savvy analysts.

What is even stronger is to actually have ELSFP modules at a conference like Lumentum did in ECOC 2025. See this short video, and notice how it includes a power meter showing nearly 24.5 dBm of power output, along with a spectrum analyzer showing the laser linewidth.

Coherent had a demo at ECOC 2025 too, but they only had a power meter and no spectrum measurements. The power reading was >24 dB alright, but that is a scalar measurement. A spectrum measurement of the laser output usually reveals a lot more – like linewidth and side-mode suppression ratio (SMSR).

This approach is even better than a published paper because critics can see the hardware working in real time and know that it is not a one-off result in a company’s lab environment. Several conference attendees will press the exhibitor on the show floor with questions, and then write on Substack and social media about it. Independent reports analyzing what they say serves as “proof-that-it-works” and builds investor confidence.

The only live demo of CPO with ELSFP was shown at OFC 2026. These numbers from the demo characterize the whole transmit chain: laser, coupling, split ratio, modulator, and driver, The laser-specific merits are either confounded or invisible. To judge the laser itself you need fiber-coupled power per port, wall-plug efficiency, RIN in dB/Hz, SMSR, temperature range, and reliability data.

The industry will be closely watching ECOC 2026 and OCP Global Summit for any information on this front.

The Counterargument

While laser measurements would be helpful, should Coherent really cave to the pressures of Wall Street or the blooming cottage industry of Substack industry analysts? From my own years of experience working within the semiconductor industry, it is common for companies to choose to not publish, or send representatives to conferences for fear of information leaks. Does lack of information necessarily mean that a product line is uncompetitive?

Broadcom has their Remote Laser Modules (RLMs) that they use for their CPO line of products. From what I can tell, I don’t see any published measurements of laser specs from them either. If you know how RLMs from Broadcom compare to Lumentum’s UHP lasers, please do let me know. But oddly enough, nobody questions Broadcom regarding their UHP laser capabilities.

This leads us to ask: is Coherent being held to an unfair standard? Should they publish data just because their competitor did? Nvidia invested $2B each into Lumentum and Coherent for their laser supply. I would assume basic due diligence by the highly competent optical networking experts at Nvidia (particularly the group from their Mellanox acquisition) would have revealed if there were fundamental flaws with their laser technology.

Ultimately what matters is whether they have a laser that works for CPO. In a highly supply constrained laser market, having a highly reliable product, with high yield, and competitive specs would make them as important a player as any in the high power laser market.

What to Watch For

In the absence of laser measurements, there are some industry signals that investors should watch for in the next 12-15 months, to understand where Coherent is headed.

Read more

Given its importance, the exact underlying technologies are still open to debate, sometimes even contradictory. The networking theme ranged from “Copper is not going anywhere” to “Optics everywhere,” and everything in between. We’ll go through some talks from OCP APAC Day 1 that dance around this theme.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news. If you would like to engage research services, reach me at SemiExponent.

Themes we cover in this report:

• Copper will not only survive, but thrive. SemiAnalysis about copper-optics coexisting for the foreseeable future.

• Copper is dead. Lets go to optics. Astera Labs talk that declares copper dead, and goes to NPO/CPO.

• Optics, but make it quick: Fast OCS switching for scale up by a company called Linque.

• Not copper, not optics. There may be a third door. Point2 e-tube technology had a talk and show floor demo. 4K video of hardware included.

Read more

Here are some key earnings takeaways:

• Revenue of $392.4 million, up 27% sequentially and 104% year over year, against a Q2 guide of $355-365 million given in May.

• PCIe 6 products, meaning Scorpio switches plus Gen 6 Aries retimers, crossed 50% of revenue for the first time, up from roughly a third in Q1.

• Q3 guidance of $540-560 million, non-GAAP gross margin of about 72%, and non-GAAP EPS of $1.16-1.21.

Their Scorpio line of switches is expected to become the largest product family in Q3, one quarter ahead of what management guided in May. Part of their uniqueness comes from their COSMOS software platform which does more than just telemetry and health monitoring.

Astera Labs should be viewed as a switch company – along the lines of Broadcom – going forward, as their 320-lane Scorpio X switch gains larger adoption, notably even with neoclouds and inference GPU providers.

CXL is an interesting segment that shows some promise, but is really a 2027 story. Optics is a 2027 story regarding NPO, while CPO lies even beyond that. The Q2 earnings call provided quite a bit of color regarding where things are headed.

For a deeper-dive on Astera Labs, see the post below.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement. For institutional research, contact SemiExponent.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Scorpio X Acceleration

Around the time of IPO, the dollar content of Astera products per XPU was $50-$100. In the Q2 earnings call, COO Sanjay Gajendra said that he sees their business contributing multiple $1,000s per XPU with the 320-lane Scorpio-X scale-up switch itself contributing $1,000 per XPU.

In SemiAnalysis’ Trainium 3 deep dive, they estimate that the 320-lane Scorpio X has an attach rate of 0.25-0.28 per XPU depending on the switched rack Trn3 SKU. We’ll reproduce that table below for convenience.

At $1,000 of Scorpio X content per XPU, a 32-XPU Gen2 Trn3 rack carries $32,000 worth of 320-lane Scorpio X content, delivered as 8 scale-up switches. The 72-XPU variant Gen2 Trn3 rack carries $72,000 across 20 switches. All this points to an ASP of $3,600-$4,000 per Scorpio X switch.

This is consistent with Tomahawk 6 pricing. Broadcom priced Tomahawk 6 below $20,000 at launch, before volume discounts, for a total switch bandwidth of 102.4 Tbps. That works out to roughly $200/Tbps. Scorpio X has 5x lower switch bandwidth at 20.48 Tbps, and at $200/Tbps, that prices it right at $4,000.

Additionally, for the Gen2 variant, we are not counting the lower lane count Scorpio-P switches (144 32-lane, 72 64-lane, or 36 128-lane) that are on the PCB per rack. That would add even more incremental revenue for Astera Labs, and Aries retimers would be on top of that. You can see how much PCIe Gen 6 content exists in a rack, and how that can contribute to 50% of total revenue as stated in the earnings call.

Another bit of color in the Scorpio acceleration story stems from the fact that companies that are building add-in card GPUs like SambaNova or d-Matrix (for example) would also be interested in Scorpio X switches, and not just AWS. This becomes even more important if companies like these can buy NVLink Fusion interfaces in the future, so that they can work better with Nvidia hardware. This NVLink aspect will become important in Trainium 4. Similarly, neoclouds are a potential buyer of Scorpio scale-up switches too.

Gajendra said that there are 10 customers getting into pre-production and qualification cycles. It remains to be seen how many will materialize into revenue in future quarters.

COSMOS

A part of the switch story that we did not cover fully in the deep dive was their COSMOS software platform. Deep telemetry was an important part of Astera’s growth strategy in the early retimer days. Today, COSMOS is much more. Its Hypercast and in-network compute features are advanced hardware-accelerated networking technologies that offload data processing and multi-destination packet distribution directly onto smart fabric switches to speed up large language model (LLM) training and inference. This essentially speeds up collective operations during training and enables greater utilization of GPUs.

Thus, the Scorpio X switch story does not stand alone. It is closely coupled with the software to extract the maximum performance of switches and GPUs within a rack.

CXL

Compute eXpress Link (CXL) solutions are becoming increasingly popular in the industry now as memory shortages stay acute, and hardware makers scramble to provide alternatives for a variety of use cases. Gajendra points out a few in the earnings call:

1. High cost of DRAM is driving people to stick different kinds of memory behind a CXL bus

2. When CXL is attached to an XPU, KV cache acceleration drives lower inference latency

3. In general compute applications that are memory intensive, CXL provides a way of adding more memory to the CPU

CXL is a story that is just beginning for Astera Labs via their Leo series of products. In Q2, they closed a design win with a US hyperscaler. In 2027, they expect to ship CXL memory controllers to two US hyperscalers. But most of the volume and associated revenue is really a 2027 story.

After the paywall are some thoughts about where Astera Labs goes from here, what can fail, and what to watch for.

Read more

“Given the size of the opportunity and the associated dollar content, we would expect to see that Scorpio will become our largest product line by the end of the year, which is strong performance for the product line that was only a small percent of total company revenue last year.”

In addition, Astera Labs explicitly pointed out in their last earnings calls that they expect their dollar content per accelerator to go up.

“We do see our content continue to increase, and to that end we are expecting, and going forward with the design wins we have, over $1,000 worth of content per accelerator, and that is significant and growing rapidly for us.”

The consensus on Astera Labs is that Scorpio is the growth story and that part largely holds as Trainium 3 volumes ramp. What the consensus misses is that Scorpio X’s scale-up revenue has a narrow customer list, and a known end date when AWS moves Trainium 4 to NVLink Fusion. Astera Labs’ future hinges on their pivot to UALink in the next generation, and the potential to serve as NVLink Fusion connectivity partner to a broad cross-section of the AI accelerator industry.

In this post, we will focus on why Scorpio switches are the near term growth path, how AWS warrants plays in, and where NVLink Fusion and UALink land in Astera Labs’ future.

If you want to understand the history of Astera Labs from their early foray into retimers, check out our deep dive podcast on Semi-Doped on this. ALAB announces its 2Q2026 earnings call on August 4th 2026. We will publish an earnings review post as well.

Contents:

• Why it makes sense to go from PCIe retimers to switches

• What the choice of PCIe means as a protocol for scale-up

• Comparison of PCIe switches from Astera Labs, Broadcom, and Marvell

• How AWS warrants play into their relationship with Astera Labs

• What the move to NVLink Fusion in Trainium 4 means for Astera Labs

• Why UALink and its adoption defines the future of Astera Labs

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Also check out the Semi Doped podcast with Austin Lyons and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

PCIe: From Retimers to Switches

Astera Labs’ current networking strategy is entirely built on PCI express (PCIe), with a future goal to upgrade to native UALink. Their Scorpio line of switches is built on PCIe, which is a natural extension of their IP from the Aries line of retimer products that are broadly used to fix signal integrity of PCIe links that connect the XPU to CPU, NIC or SSDs.

The Scorpio product line has two SKUs, for different use cases in the rack:

• Scorpio P-Series: PCIe 6 switching for scale-out networking — fanning GPUs, NICs, and storage out of head nodes. It has been shipping since Q2 2025 with design wins at three hyperscalers and two more starting late 2026. Has product variants from 32 to 320 lanes.

• Scorpio X-Series: the scale-up fabric — the merchant silicon switch, running memory-semantic traffic over PCIe.

Astera Labs states that Scorpio is a bestseller for them, with 10%+ revenue within two quarters of launch, and is expected to become the largest product line by the end of 2026.

The Choice of PCIe

Regardless of their impending success with PCIe switches, it is interesting to ask why PCIe as a protocol of choice for scale up? PCIe has been the mainstay of connections of computer motherboards for decades, and what made Astera Labs a company with $1B ARR today is their prescient bet on PCIe in the AI era. Their massive design wins in the Hopper and Blackwell HGX boards within a server tray, was because of their first-to-market with a PCIe Gen5 retimer, which has become an incredibly sticky slot connecting XPU-CPU/NIC and is a big contributor to their success. The logical next move for Astera Labs is to move this IP up the stack into rack-scale.

PCIe itself is attractive as a protocol for one main reason: every device – XPU, CPU, NIC, SSD – already has a PCIe controller built in and natively speaks the same language. Because the protocol is designed to know the route directly to various devices, it routes by memory address and encodes destination in its upper bits. ESUN-T and UALink had to define specific transport layers to get the same thing. In addition, PCIe lanes run at a slower bit rate compared to Ethernet, which is useful for keeping scale-up in copper longer.

PCIe Gen5, for example, operates at 32 GT/s (giga-transfers-per-second), where the transfer refers to the movement of a bit. This relates to the modulation scheme as well: an NRZ modulated signal moves one bit per signal, while PAM4 moves two bits per symbol. The symbol modulation speed is measured in Baud: at 32 GT/s, it is 32 GBaud on NRZ, and 16 GBaud on PAM4. On a per-lane basis, the speeds of PCIe are slower than NVLink, ESUN (ethernet based) or UALink, which is oddly an advantage when it comes to interconnects for scale-up with copper (better reach). But, what PCIe lacks in per-lane speed, it must make up for in lane count.

The ultimate speed in Gbps of a PCIe link thus depends on: (1) speed in transfers/sec, (2) modulation format, and (3) number of lanes per link. Let’s work on a simple example to show this:

PCIe at 64 GT/s using PAM4 at 32 GBaud, and running four lanes in aggregate (x4) gives 64 GT/s x 4 / 8 bits = 32 GB/s unidirectional, or 64 GB/s of bidirectional bandwidth. The following table shows this calculation for several PCIe generations.

ALAB Scorpio X, AVGO PEX, Marvell Structura S

It is instructive to compare the Scorpio switch against the big dog in the merchant silicon switch market – Broadcom. Broadcom also offers PCIe 6 switches in its PEX line of switches, alongside its flagship Tomahawk 6 (TH6). We’ll gloss over Marvell too.

Read more

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

Also check out the Semi Doped podcast with Austin Lyons and myself, and our daily free newsletter with latest semi news. If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

Are you aware of the Situation?

Situational Awareness (SA) is a hedge fund run by ex-OpenAI upstart Leopold Aschenbrenner whose entire view on the AI trade is “hyper-bullish” to put it mildly. Their sole premise of investment is — AGI is here to swallow the world, and nothing short of it. In the era of bottleneck trades that we have been in the midst of for the last two years, this premise worked swimmingly, with the fund posting over 2,000% gains.

They did so with an insane amount of leverage — like 4x, where leverage is basically borrowing to invest, and not putting down your own money. You can make a lot of money with leverage, but when things go south like they did this week, you can also lose a lot of money. We’ll get to that.

SA become a global and FinX phenomenon, where everyone was waiting to see what they were investing in. Their 13F filings were more awaited than a Taylor album for a Swiftie. There were plenty of copy traders who waited to see what SA invested in and bought the same stocks. I imagine some of them were even more levered than SA because (a) blind faith/greed, and (b) they were not as early as SA, and had to “catch up.” More leverage = more money, especially if you already had bought into the “AGI is here and nothing else matters” kool-aid.

The Storm was Brewing

But the perfect storm of events were already brewing; one that you can probably only tell in hindsight.

• SK Hynix’s stellar numbers came in below consensus (a word the Street uses to mean “we all believe this is how much they will make in earnings”) — a sign that expectations had gotten ahead of reality

• China announced they are putting immersion DUV in production — this does not change anything immediately.

• CXMT IPO-ed and got a massive boost with people believing they will flood the market with cheap memory — they can make DRAM, not cutting edge HBM, and they definitely do not have capacity to supply the entire world with memory.

• Everybody getting very skittish about the amount of debt that the infrastructure companies had taken on.

There is one more factor that deserves its own section later, but let’s (briefly) talk about this debt: even earlier this week, there were fears that infrastructure builders are not going to be able to repay their debts if all the CapEx investments do not turn into revenue. Not to mention all the circular financing deals going on, where Nvidia is investing money into their own customers, and many more complex dealings tied to tranches of stocks vesting when they hit certain prices. It’s nuts, I agree, and hard to keep track of.

Anyway, investors often buy Credit Default Swaps (CDS) when they believe that companies will not be able to repay their debt; think of it as insurance against a bad investment. When this insurance premium goes up, you know that more people believe that companies will default of their lines of credit. CDS premium spreads were what what gave away the subprime mortgage crisis of 2008-2009 too, but at the time, they were insuring against Mortgage-Backed Securities. Only now, its against the AI infrastructure debt, not mortgages.

The Fuse Was Lit

In this supercycle, it is clear that memory investors are most skittish because of the rapid rise of profitability in the big-3 memory companies. These companies have made an order or magnitude more money on an annual basis than just a few years ago. Profit margins have reached 90%, which is an absurd number, and the entire tech industry is cracking under the burden of memory prices.

To put it mildly, investors have been looking for the memory top.

SK Hynix’s earnings showed an inflection point — one that investors have been waiting for. Korean investors, like so many others, have also been highly leveraged on memory companies; and when the first derivative hit zero, many investors found the top they had been looking for. This caused a massive unwind in the Korean market causing SK Hynix stock to have one of the greatest drops in Korean stock market history. The KOSPI index plummeted, and the Korean stock market had to limit short selling / hedged bets to even stabilize the carnage.

This essentially pulled the entire AI market down, because if we are at the top now, then future capex investments are going to decline, and that means there’s not a lot of money to be made. Better pick up the change and run. Even Corning, who unrelatedly made innocuous optical fiber faced a 20% drop in stock. Amkor — a packaging company — posted good earnings, but faced a steep sell off. The blast clearly has had a wide radius.

Pull the Trigger, But Aim First

In this midst of this devastation, SA had about $45B invested and mostly in AI trades and short software stocks. Their stars aligned, but for the worse — a Lovecraftian scene where the cruel moon shone its eerie white light on one hyper-bullish hedge fund. Its long positions in AI were crushed, while short software positions ripped.

In less than a week, SA lost tens of billions of dollars wiping out the entire public holding of the hedge fund. Its losses were about 75%, and what remained was its position in private funds, including a $5B stake in the Anthropic IPO which remained protected. This kept SA alive as a hedge-fund, but crippled and bleeding.

A letter posted by SA showed that they are still up 80% YTD, just not 2,000%+. But 80% is no big deal. Anyone who bought at the start of the year would have 45% in gains; you don’t need a genius to make 80%, let alone Leopold Aschenbrenner, who is still a smart man all said and done.

So what happened to SA’s public equities? Well, there is one aspect of the storm I haven’t told you about, remember? A few days ago, a note from Frank Flight, head of macro strategy at Citadel Securities — a hedge fund owned 80% by billionaire Ken Griffin — warned that the Fed would raise interest rates by 0.25% in an effort to fight inflation. This was likely a scare tactic because the fed did not end up raising the interest rates after all. But the damage was done.

Tech and AI stocks are highly sensitive to interest rates because their valuations rely heavily on projected future earnings. Higher rates make future profits less valuable today, and the scare was also a contributing factor to the sell off this week. One theory is that the scare tactic was planned, and intentional because of what happens next.

Citadel Securities comes in and BUYS SA’s ENTIRE PUBLIC STOCK PORTFOLIO for 40-50 cents on the dollar.

A lot of main news outlet’s claim that Citadel “bailed out” SA here, but make no mistake: Citadel aimed well, and shot SA straight in the head. Legal? Maybe. Market Manipulation? Maybe. Brutal? Yes, 100%.

Oh, and did I mention that Leopold was getting married this weekend, to the chief-of-staff of Anthropic — Avital Balwit? Dude.

After the paywall, a few thoughts on what happens next.

Where does that leave us now?

Read more

Since the video’s release on Youtube late last week, shock clips have appeared on social media that take the whole conversation out of context. What actually follows in the whole 35 minute discussion is an insightful discussion into the future of memory, how it is an exciting time for memory innovation, where memory is going in the next few years, and what the industry is looking at today.

This article discusses these points in greater detail, providing necessary color where helpful so that the whole discussion can be viewed in a more helpful and positive context for the future of memory.

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the article

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

The Three Fold Problem

Gelsinger points out that there are actually three major problems with HBM today: (1) insatiable demand, (2) larger capacity demand, and (3) greater memory bandwidth requirements.

When these three elements are put together, only one technology really emerges to satisfy the requirements of the industry: high bandwidth memory, or HBM. Stacking DRAM chips to make HBM ticks the higher capacity with high bandwidth boxes, while creating a massive appetite for memory. Today, AI is a memory-limited problem.

The consequence of this is that memory makers enjoy a near 90% margin, which Gelsinger points out is something that should never happen. Memory companies used to have one good year out of four as they usually built out capacity too quickly, and were left holding the bags in a downturn. Today, the memory cycle is looking strong several years into the run up, with no end in sight.

Unfortunately, the memory crunch is here for at least another 2-3 years, as Gelsinger points out, because capacity buildouts take time. He also points out that in that time frame, Apple may increase prices yet again. The consumer market will continue to be choked by AI demand in the near future. The hardest question to answer today is: When capacity does come online, will we still need all this memory?

Vijay Shilpiekandula from Dilation Capital makes an interesting point: token growth is exponential, while capacity buildout is linear. He mentions that memory intensity per token is on the rise - a framing that I particularly like. As context lengths continue to grow, and the user base (human and agent) increases, the amount of memory that the system has to support only increases, and from what we can tell today, there is demand inflection on the horizon.

HBM: The Ugly Baby

The said ugly baby was actually born out of a collaboration between AMD and SK Hynix as early as 2013, when they were trying to solve the memory bandwidth problem for supercomputing applications. HBM1 stacked four 1 GB DRAM chips for a total capacity of 4 GB, with a memory bandwidth of 128 GB/s. Today’s HBM4 stacks sixteen 4 GB dies for 64 GB capacity and a total bandwidth of 2TB/s.

The baby is all grown up now, but has become increasingly difficult to handle. Stacking to increase capacity causes thermal, yield, and capacity issues. DRAM chips generate heat, and it is becoming difficult to ensure that it is effectively removed. Yield is also becoming difficult as stack heights increase. Capacity-wise, HBM is incredibly inefficient because the bit-density of a DRAM die for HBM is 1/3rd of a standalone DRAM chip - a consequence of putting through-silicon-vias (TSVs) through the DRAM die. Making HBM now pulls all the capacity out of the DRAM supply chain because it takes 3x as many wafers to make the same HBM capacity, as you would with DRAM.

The problem gets worse with HBM5. JEDEC, the standard body that defines how high these stacks can physically be, has been relaxing the specs because keeping the physical dimension constrained would involve a shift to hybrid bonding - a technology still new to the memory industry at scale. Concept demonstrations for HBM5 from the big memory makers now include integrated cooling solutions co-packaged with HBM to extract heat. In addition, the base die that sits under the DRAM stack is going to a custom design that makes memory no longer a commodity, off-the-shelf product that is fungible between different designs.

The memory game has fundamentally changed, and Gelsinger’s main point is that the next few years will see the rise of new architectures and technologies if we were to fulfill our AI dreams, giving new life to a technology that has stopped scaling for the good part of a decade. It is the greatest time in history to be in the memory business.

After the paywall:

• The Era of Memory Innovations

  • Bandwidth solutions (SRAM, 3D-DRAM)

  • Capacity solutions (SOCAMM/LPDDR, Flash)

• Nobody knows how this memory cycle ends

Read more

In Part 1 of this series, we built up the fundamental physics of ultra high power (UHP) lasers. We concluded by identifying the four walls that make it challenging to implement a UHP laser: managing heat, maintaining narrow linewidth, getting light out, and catastrophic optical damage.

In Part 2, we will explore exactly how Lumentum overcame these challenges via their 2007 paper as JDS Uniphase, and via their later 2022 CLEO paper on UHP lasers. We will touch on reliability and single mode operation, and dig into how defensible their UHP laser platform is, including what it means for the competition.

Contents:

• Suppressing Re-Broadening for sub-MHz Linewidth: How to choose the key design parameter that makes UHP lasers work

• Precisely Engineering the Cavity Length: Are longer lasers necessarily worse? How do laser designers choose the laser cavity length?

• Pros and Cons for Longer Cavity Length: What does cavity length help with, and what it means for reliability and mode-stability

• The Moat isn’t the UHP Laser design: The engineering mindset is to out-design the competition in UHP lasers, but reality is different: is it good enough?

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the ebook

Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.

If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.

Suppressing Re-Broadening for sub-MHz Linewidth

Even in 2007, Lumentum (then JDSU) realized that increasing the output power of a DFB past a few 10s of milliwatts resulted in the laser linewidth getting broader than 1 MHz even if the “Schawlow-Townes-Henry” equation predicts that it should get narrower as the laser power is increased (explained in part 1). Good lasers should have narrow linewidth because the transmission of an optical pulse on an optical fiber broadens the pulse anyway; no need to have that problem at the source.

Pulse broadens as power gets higher. Source: Gemini, for conceptual purposes only.

They explain that there were two things causing this:

1. With so much optical power in the laser, photons are not distributed uniformly in the physical structure under the DFB grating. We explained this effect as spatial hole burning in part 1. This non-uniformity also causes a non-uniform refractive index, which in turn makes the linewidth broaden.

2. Within the InP multiple quantum well (MQW, where electricity is converted to light), there are random charge recombinations always occurring. This added to the “phase noise” of the system and also caused linewidth to broaden.

The former mechanism was more dominant, but the latter is significant enough to not be ignored. It was already known through other published work at the time that the easiest way to avoid re-broadening is to make the cavity length (L) longer. This generally helps distribute the photons in the cavity more evenly and keep the refractive index as even as possible under the DFB grating.

But it is not quite as simple as just increasing length because how evenly the photons are distributed depends on the DFB grating design. In part 1, we explained how there is a quantity κ – the grating’s reflection strength per unit length – combined with the length of cavity L, determines how evenly the photons are distributed.

As a recap from part 1:

κL is the key knob that decides how light is distributed along the chip.

• If the κL is too low, then light spreads out and leaks towards the ends. Insufficient feedback to hold the light in the channel evenly, and wavelength selection is weak.

• If κL is too high, then light gets trapped and piles up in the middle of the chip and causes spatial hole burning.

Engineering the right UHP laser now came down to picking the right cavity length L, for a given grating reflection strength κ set by the grating design. There is a goldilocks range for κL where the photon field was the flattest – which JDSU determined to be 0.7. Look at the pink lines in the graphs below, for different phases of the optical signal. κL=0.7 provides a flatness over nearly 200 degrees of phase – much better than κL=0.5 or κL=1.

Precisely Engineering the Cavity Length

Now that κL is set, the crux of the problem is that both κ and L are design parameters that can be chosen to keep κL=0.7. How do we choose?

Read more

The goal of this article series is not to dethrone Lumentum or advocate for an alternative provider in any way. It is to understand the true difficulty of making UHP lasers for CPO from first principles, so that the reader is well informed enough to cut through the noise and think for themselves.

Instead of examining generic laser architectures, we will directly address the one that Lumentum says their structure is most similar to, in their CLEO 2022 paper.

In their CLEO 2022 paper, it seems interesting that Lumentum would quote a paper from 2007 as the basis for their high power laser design. These authors at the time were affiliated with JDS Uniphase, which spun off in 2015 to become the publicly traded, and well loved company that we know as Lumentum today.

Interestingly, none of those authors work at Lumentum any more. Another important takeaway is that Lumentum has been working on UHP lasers since before they were Lumentum. As it turns out, 20 years later, this precise laser architecture is what the world wants for CPO and Lumentum just happens to have a ton of experience with it.

This 2007 paper describes a 1310 nm InGaAsP/InP DFB doing 600 mW on-chip and 350 mW ex-fiber with sub-500 kHz linewidth, and is spec-for-spec the ancestor of the UHP laser Lumentum sells for CPO today. We’ll use this as a case study to understand UHP lasers and how its related challenges were solved, but is still by no means an endorsement for investments in Lumentum. Do your own research.

To understand how to make a good UHP laser requires a background in how these devices work. This first part is groundwork: how an InP DFB laser works and the physical walls that make high power hard. Where Lumentum’s advantage actually lies, and how deep the moat runs, needs this physics first, so that verdict waits for Part 2.

Contents:

• InP laser diode fundamentals

  • Multiple Quantum Well structure

  • L-I curve, spontaneous vs stimulated emission

• Distributed Feedback (DFB) grating design

  • Mode-hopping

  • How to space gratings

  • How DFB works

  • Refractive index changes

  • Reflection strength and length

• Laser Linewidth and DFB Cavity Length

  • Calculating linewidth: Schawlow-Townes-Henry equation

  • Impact of power, mirror loss and dependence on length

• Low Power DFB Lasers for Telecom/EML

  • Direct modulated lasers (DML)

  • CW lasers and EMLs

• High Power DFB Lasers for CPO

  • How to increase output power

  • The 4 walls of UHP lasers

    • Heat

    • Linewidth widening

    • Getting light out

    • Catastrophic optical damage

By reading this post, you agree to the terms and conditions. Also see the full ethics statement.

If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.

Buy the report

If you would like to engage boutique research services for your project, reach out to us.

Contact SemiExponent

The Indium Phosphine (InP) Distributed Feedback (DFB) laser is the fundamental component of optics today, so it is best we understand how it works. It consists of two basic parts built on the same piece of material: The InP laser diode and DFB grating.

The InP laser diode

This is often called the gain section and its job is to convert electrical current into optical power. The light is generated by Multiple Quantum Wells (MQW) in an “active region” which are essentially alternating layers of InGaAsP (Indium-Galium-Arsenic-Phosporus) sandwiched together. The alternating layers have slightly different compositions of these fundamental elements, so that they have slightly different energy levels. The purpose of this structure is to trap electrons in a low-energy layer between two high energy layers so that the electrons are trapped in a “well” and can’t get out. Multiple wells means more electrons, which means stronger laser output.

Source: Effects of multiple quantum well width on InGaAs/InP laser diode, doi:10.1088/1742-6596/2937/1/012007. (link)

If you want to understand more about what energy levels in a semiconductor mean, I have several articles on this.

To generate light, current is applied to the laser diode by applying a voltage across the anode and cathode. When the current applied is low, the emission of light is a messy stream of light waves out of phase with each other, each one behaving independent of the other. This is called spontaneous emission, and is not a focused beam of light; it is much like LED light. If photons were people, they would just wander around in a park without purpose.

Beyond a threshold of current, stimulated emission kicks in. The photons generated in the presence of a highly concentrated “well” of electrons kick up more photons themselves resulting in an avalanche of more photons. These photons now march in step with their phases correlated to each other. Now the photons are on a military parade in complete lockstep while being highly directional and focused.

Source: Wide-Band and Scalable Equivalent Circuit Model for Multiple Quantum Well Laser Diodes, Jae Hong Kim, PhD Thesis, Georgia Tech 2005.

This relationship between light and applied current is represented by the L-I diagram shown below. The threshold of current needed to make photons march together depends on the temperature; higher temperatures need more drive current. This is an important observation that will later drive a lot of our understanding of UHP lasers.

Source: Wide-Band and Scalable Equivalent Circuit Model for Multiple Quantum Well Laser Diodes, Jae Hong Kim, PhD Thesis, Georgia Tech 2005.

Read more