Common Parallel StrategiesConsistent ViewConsistent TensorDistributed Training LauncherData Parallelism TrainingPipelining Parallelism