b78588d163
* CUTLASS 3.7 * clean up changelog --------- Co-authored-by: yuzhai <yuzhai@nvidia.com> Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
87 lines
2.6 KiB
Markdown
87 lines
2.6 KiB
Markdown
# Distributed GEMM
|
|
|
|
## Requirements
|
|
|
|
### Build
|
|
Make sure to set up CUTLASS with
|
|
support for [Programmatic Dependent Launch (PDL)](../../media/docs/dependent_kernel_launch.md),
|
|
that is with the `CUTLASS_ENABLE_GDC_FOR_SM90` flag.
|
|
|
|
```bash
|
|
cmake $PATH -DCUTLASS_NVCC_ARCHS="90a" -DCUTLASS_ENABLE_GDC_FOR_SM90=1
|
|
```
|
|
|
|
### Minimum software
|
|
|
|
Like all other CUTLASS examples, the NVIDIA driver, runtime, and CUDA Toolkit are required.
|
|
This example specifically requires CUDA Toolkit 12.6 or newer, due to some of the necessary
|
|
CUDA graph APIs.
|
|
|
|
### Hardware / driver settings
|
|
|
|
This example requires Hopper GPUs with NVLink network.
|
|
|
|
If you're not sure, first run the following command and make sure your GPU
|
|
compute capability is 9.0:
|
|
|
|
```bash
|
|
nvidia-smi --query-gpu=name,compute_cap --format=csv
|
|
```
|
|
|
|
Sample output:
|
|
|
|
```
|
|
name, compute_cap
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
NVIDIA H100 80GB HBM3, 9.0
|
|
```
|
|
|
|
|
|
Then you should make sure there is an NVLink network by checking the GPU network topology,
|
|
and making sure there's `NV*` links between every pair of GPUs:
|
|
|
|
```bash
|
|
nvidia-smi topo -m
|
|
```
|
|
|
|
Sample output:
|
|
|
|
```
|
|
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7
|
|
GPU0 X NV18 NV18 NV18 NV18 NV18 NV18 NV18
|
|
GPU1 NV18 X NV18 NV18 NV18 NV18 NV18 NV18
|
|
GPU2 NV18 NV18 X NV18 NV18 NV18 NV18 NV18
|
|
GPU3 NV18 NV18 NV18 X NV18 NV18 NV18 NV18
|
|
GPU4 NV18 NV18 NV18 NV18 X NV18 NV18 NV18
|
|
GPU5 NV18 NV18 NV18 NV18 NV18 X NV18 NV18
|
|
GPU6 NV18 NV18 NV18 NV18 NV18 NV18 X NV18
|
|
GPU7 NV18 NV18 NV18 NV18 NV18 NV18 NV18 X
|
|
```
|
|
|
|
Finally, check if the driver enables peer to peer access, which should usually be the case,
|
|
but it's good to check anyway:
|
|
|
|
```bash
|
|
nvidia-smi topo -p2p r
|
|
```
|
|
|
|
Sample output:
|
|
|
|
```
|
|
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7
|
|
GPU0 X OK OK OK OK OK OK OK
|
|
GPU1 OK X OK OK OK OK OK OK
|
|
GPU2 OK OK X OK OK OK OK OK
|
|
GPU3 OK OK OK X OK OK OK OK
|
|
GPU4 OK OK OK OK X OK OK OK
|
|
GPU5 OK OK OK OK OK X OK OK
|
|
GPU6 OK OK OK OK OK OK X OK
|
|
GPU7 OK OK OK OK OK OK OK X
|
|
```
|