Pytorch distributed parallel processing

In the official documentation of version 1.9 of Pytorch, it is clearly stated that nn.DataParallel or multiprocessing is no longer recommended, but nn is recommended.parallel.DistributedDataParllel.Even if there is only one GPU core, nn.paralle.DistributeDataParalle is also recommended.The reason given in the official documentation is:

The difference between DistributedDataParallel and DataParallel is: DistributedDataParallel uses multiprocessing where a process is created for each GPU, while DataParalleluses multithreading. By using multiprocessing, each GPU has its dedicated process, this avoids the performance overhead caused by GIL of Python interpreter.

The general idea is that DistributedDataParallel is better because it allocates a fixed process to each GPU; and DataParallel is not recommended because it uses a multi-threaded method, which may incur performance overhead from the GIL or the Python interpreter.
Another Basic document mentions that for torch.multiprocessing or torch.nn.DataParallel, the user must displayCreate an independent copy of the main training script for each process.This is not convenient.