Microsoft demonstrates how to run a 235 billion parameter AI model on three PCs linked by USB4
Microsoft has announced a developer preview of RDMA over USB4, a technology that allows the execution of an artificial intelligence model to be distributed across multiple Windows PCs. The proposal takes advantage of direct USB4 connections so that compatible software uses the memory and graphic processing capacity of the participating computers.
The announcement includes demos with platforms from AMD and Intel. The component code is now available, although the Windows build that will enable testing is still missing. Microsoft plans to distribute it soon through the Experimental channel, within the Future Platforms branch of the Windows Insider program.
Run models that don’t fit on a single computer
The primary use case is models whose size exceeds the available memory on a computer. Distribution across multiple computers allows parts of the same model to be stored and processed on different machines, while maintaining execution in a local installation.
This requires a program prepared to distribute the work. Connecting PCs does not automatically turn your memory into an extension that any Windows application can use. Microsoft has published a flame variant.cpp with changes to work through NetworkDirectthe Windows interface for RDMA communications.
The repository identifies specific features for distributed execution, computation optimizations, and tensor parallelism options. It also clarifies that these additions require compiling that variant: regular installers and versions of the original project do not automatically include the changes.
Lower latency over the same USB4 link
RDMA stands for Remote Direct Memory Access. It allows data to be transferred between memory regions of different systems, reducing the journey through the conventional network stack and avoiding additional copies of software. Windows already has this technology for specialized adapters; The novelty consists of bringing it to USB4 connections between compatible PCs.
Microsoft has measured round trip times of between 28 and 50 microseconds for small messages. Their comparison uses Ethernet over UDP over the same USB4 link and puts the latency of RDMA at about a quarter.
The published graph specifies 28 microseconds versus 128 for 64-byte messages, and 50 versus 194 for 16 KiB messages. The improvement depends on the size of the message and describes the communication between machines, without being equivalent to multiplying the speed of model generation by four.
The implementation includes 40 and 80 Gbps links. Microsoft notes that both achieve reduced latency, while 80 Gbps provides twice the nominal bandwidth. That difference can be useful in loads that transfer more data, such as processing inputs with long contexts.
Three AMD PCs for Qwen3 and up to four Intel for GPT-OSS
A demonstration uses three mini PCs with Ryzen AI Max+ 395, Radeon 8060S graphics and 128 GB of unified memory per computer. Together, they add up to 384 GB of physical memory, although part of that capacity must serve the operating system and the software itself. Connected via 40Gbps USB4, they run Qwen3-235B-A22B with four-bit quantization.
Microsoft communicates approximately 18.5 tokens per second in its lab. AMD clocks 21.98 tokens per generation second with its own settings.
Intel, for its part, has tested GPT-OSS-120B with Core Ultra Series 3 computers, Thunderbolt 4 connections and 64 GB of memory per computer. With a context of 1,000 tokens, you get 49.4 tokens per second using three PCs and 54.2 with four.
The table also shows a time to the first token of 1.1 seconds with three teams and 1.2 seconds with four in that test. Adding a machine increases available capacity, but these results show that not all measures improve at the same rate.

Each computer needs a direct connection to the others
Microsoft recommends a full mesh, with one cable between each pair of computers. For three PCs you need three cables and two USB4 ports per computer. With four, the calculation amounts to six cables and three ports per machine.
Connecting them in a chain greatly reduces performancewarns the company. They must also use the same supported build of Windows and matching versions of the test components. The NetworkDirect provider includes tools to test links and requires correctly configuring network interfaces.
The preliminary phase requires activating the test signature and disabling Secure Boot, so Microsoft assigns it to test computers. Additionally, the application must reestablish the connection if the connection is interrupted, since the transport does not automatically retry.
There is still no date for incorporating RDMA over USB4 into a stable version of Windows. The next phase will be to release the necessary Insider build so that developers can test the set.
