High Speed Data Write and Read

640 GB of High Bandwidth Memory spread across eight GPUs in every DGX A100 server !!!!!    PCIe, NVLink, NVswitch

NCCL  ( called as NICKEL )  for GPU to GPU communication