# Distributed Training **Entity class:** Distributed-computing concept **As of:** November 17, 2026 ## Definition Distributed training updates model parameters using work performed on multiple devices or sites. Local, tightly linked accelerator clusters synchronize frequently; [[wiki/DiLoCo|DiLoCo]] and [[wiki/Streaming DiLoCo|Streaming DiLoCo]] reduce communication demands between distant worker islands. [[wiki/OpenDiLoCo|OpenDiLoCo]] and [[wiki/INTELLECT-1|INTELLECT-1]] tested geographically dispersed training at substantial scale. The participant hardware and network requirements belong in every account of such a demonstration, because a geographically distributed data center is a different proposition from household volunteer training. ## Relationships - **ecosystem router:** [[wiki/Distributed Compute Ecosystem|Distributed Compute Ecosystem]]. - **worked example:** [[wiki/DiLoCo|DiLoCo]]. - **source analysis:** [[articles/Embedded Super-Spreader Events and Muse|Embedded Super-Spreader Events and Muse]]. ## Sources - [Distributed Training: primary project record or technical account](https://arxiv.org/abs/2311.08105).