Silicon · Chapter 5 of 7
Compute waits on data
Per-core data engines proposed to end the queue that stalls arithmetic units
In the current shared-dispatch architecture, eight processing cores queue behind a single data-movement engine, leaving compute idle while data waits its turn. Replacing that bottleneck with an independent local DMA per core is the proposed fix, though results remain estimates awaiting testbench evidence.
- 8 cores, 1 engineShared-dispatch problem
- 470 ms → 85 msEstimate, not result
- 16-entry descriptor tablePer-core DMA design
- ~600 LUTs eachImplementation estimate
- ~4,800 LUTs totalEight-engine estimate
The animation runs the same workload twice, side by side, changing only how each core reaches memory. That single difference drives everything that follows.
The current arrangement puts eight processing cores behind a single data engine that fetches one descriptor at a time, so seven cores sit idle while one is fed.
The utilization column on the left never clears 12%, and most cores sit at almost nothing for the entire run, showing how much compute is wasted by the bottleneck.
On the right, the only change is that every core now owns its own load engine and pulls from banked static memory in parallel, with no other modification to the design.
All eight cores climb to around 59% utilization and hold there, reached by nothing more than opening the path to memory.
The frame that costs 470 milliseconds in the current arrangement lands in about 85 milliseconds in the parallel-engine simulation, a direct result of the memory path change alone.
Six competing processors are shown side by side, each representing a different answer to where data lives and what controls its movement.
The leftmost position shows DeepGrid's own SoC 1.1, running today on an FPGA, with 8 cores queuing behind a single DMA engine.
NVIDIA's Orin answers the scheduling question with 2048 CUDA cores and per streaming multiprocessor shared memory, at a cost of 60 watts.
Tesla's Dojo D1 gives each of its 354 cores private SRAM and no on-die DRAM, the most extreme position on the spectrum, though Tesla disbanded the project in 2025; Hailo sits nearby with a distributed memory fabric efficient only across a narrow band of workloads.
The roadmap points to per-core DMA against a shared L2 cache, one engine for every core with memory still shared, and that is where most of the industry already sits.
Technical detail
The shared-dispatch arrangement serialises data requests: all eight cores queue at one dispatch point, so adding more arithmetic units does not help when those units are waiting for data. The proposed local DMA design gives each core its own 16-entry descriptor table, a three-level address generator that walks tensor layouts without consulting a central dispatcher for every transfer, and a read FIFO that buffers arrivals between memory service and compute consumption. A frame-time reduction from 470 ms to 85 ms is cited as a proposal awaiting testbench evidence, the sources label it an estimate, not a measured result. The implementation cost is estimated at roughly 600 FPGA lookup tables per engine, or approximately 4,800 for eight engines before arbitration logic and firmware; these are engineering estimates, not finished implementation figures or production cost quotes, and the sources keep them distinct from any ASIC area or cost. Even with local dispatch, parallel requests still compete for finite shared memory-bank service capacity, and the sources note that differentiation would need to be demonstrated through workload results, compiler mapping and defensible implementation IP, not through the presence of a local DMA alone.
Portfolio deck, slide 36: Compute waitsPortfolio deck, slide 37: Shared engine, runningPortfolio deck, slide 38: Inside one local DMAPortfolio deck, slide 39: Placement is a choice