CUTLASS 3.8 Release (#2059)

* CUTLASS 3.8 Release * update * Update README.md * Revert "Update README.md" This reverts commit b353e36fe83e0815f99b44e46c0c95494c44726b. * update * update --------- Co-authored-by: Haicheng Wu <57973641+hwu36@users.noreply.github.com> Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
2025-01-25 02:44:06 -05:00
parent 9eb01fa0b0
commit 389e493055
290 changed files with 91222 additions and 291 deletions
@@ -24,7 +24,8 @@ $ export CUDACXX=${CUDA_INSTALL_PATH}/bin/nvcc

 $ mkdir build && cd build

-$ cmake .. -DCUTLASS_NVCC_ARCHS=90a             # compiles for NVIDIA Hopper GPU architecture
+$ cmake .. -DCUTLASS_NVCC_ARCHS=90a            # compiles for NVIDIA Hopper GPU architecture
+$ cmake .. -DCUTLASS_NVCC_ARCHS=100a           # compiles for NVIDIA Blackwell SM100 GPU architecture
 ```

 If your goal is strictly to build only the CUTLASS Profiler and to minimize compilation time, we suggest
@@ -653,6 +654,105 @@ targeting NVIDIA Ampere, Turing, and Volta Tensor Core operations
 $ cmake .. -DCUTLASS_NVCC_ARCHS='70;75;80' -DCUTLASS_LIBRARY_KERNELS=tensorop*s*wgrad_optimized_f16
 ```

+## Instantiating a Blackwell SM100 GEMM kernel
+
+Blackwell SM100 kernels are instantiated very similarly to Hopper kernels. Let us start with an
+[FP8 GEMM without blockscaling](../../test/unit/gemm/device/sm100_gemm_f8_f8_f8_tensor_op_s32_batch_alpha_beta.cu)
+as an example.
+
+The kernel starts with setting up datatypes and cluster shapes. 
+```c++
+  using LayoutA = cutlass::layout::RowMajor;
+  using LayoutB = cutlass::layout::ColumnMajor;
+  using LayoutC = cutlass::layout::ColumnMajor;
+  using ElementA = cutlass::float_e4m3_t;
+  using ElementB = cutlass::float_e4m3_t;
+  using ElementC = cutlass::float_e4m3_t;
+  using ElementD = cutlass::float_e4m3_t;
+  using ElementAccumulator = float;
+  using ElementCompute = float;
+  using ElementBias = cutlass::half_t;
+  using ClusterTileShape = cute::Shape<_128,_64,Int<128 / sizeof(ElementA)>>;
+  using ClusterShape = Shape<_1,_1,_1>;
+  using AtomThrShape = decltype(shape_div(ClusterShape{}, Shape<_1,_1,_1>{}));
+  using OutputCtaShape = decltype(shape_div(ClusterTileShape{}, ClusterShape{})); 
+  using MmaTileShape = decltype(shape_div(ClusterTileShape{}, AtomThrShape{}));
+```
+
+The epilogue needs to be instantiated first as the mainloop collective builder takes the shared memory budget of epilogue in the template parameter list. The 3.x epilogue collective builder API has not changed
+for Blackwell, so the epilogue fusion is built in a same way as an SM90 epilogue.
+
+```c++
+  using EpilogueSchedule = cutlass::epilogue::TmaWarpSpecialized1Sm;
+
+  using FusionOperation = cutlass::epilogue::fusion::LinearCombination<
+    ElementD,
+    ElementCompute,
+    ElementC,
+    ElementBias
+  >;
+
+  using CollectiveEpilogue = typename cutlass::epilogue::collective::CollectiveBuilder<
+      cutlass::arch::Sm100, cutlass::arch::OpClassTensorOp,
+      OutputCtaShape, ClusterShape,
+      cutlass::epilogue::collective::EpilogueTileAuto,
+      ElementAccumulator, ElementCompute,
+      ElementC, LayoutC, 16 / sizeof(ElementC),
+      ElementD, LayoutC, 16 / sizeof(ElementD),
+      EpilogueSchedule,
+      FusionOperation
+    >::CollectiveOp;
+```
+
+One can refer to our Sm100 unit tests as examples of how to correctly
+choose mainloop schedules. All of our dispatch policies can be found in [dispatch_policy.hpp](../../include/cutlass/gemm/dispatch_policy.hpp)
+and more comprehensive Blackwell specific documentation for valid 
+dispatch policies can be in [blackwell_functionality.md](./blackwell_functionality.md).
+
+```c++
+  using MainloopSchedule = cutlass::gemm::KernelTmaWarpSpecialized1SmSm100;
+  using CollectiveMainloop = typename cutlass::gemm::collective::CollectiveBuilder<
+      cutlass::arch::Sm100, cutlass::arch::OpClassTensorOp,
+      ElementA, LayoutA, 16 / sizeof(ElementA),
+      ElementB, LayoutB, 16 / sizeof(ElementB),
+      ElementAccumulator,
+      MmaTileShape, ClusterShape,
+      cutlass::gemm::collective::StageCountAutoCarveout<static_cast<int>(sizeof(typename CollectiveEpilogue::SharedStorage))>,
+      MainloopSchedule
+    >::CollectiveOp;
+
+  using GemmKernel = cutlass::gemm::kernel::GemmUniversal<
+      Shape<int,int,int,int>,
+      CollectiveMainloop,
+      CollectiveEpilogue
+  >;
+```
+
+It is worth noting that the mainloop builder takes `MmaTileShape` while the epilogue builder takes `OutputCtaShape`.
+
+Instantiating a blockscaled GEMM kernel is slightly different. Referring to an [MXFP8 GEMM](./../../test/unit/gemm/device/sm100_gemm_mxf8_mxf8_mxf8_tensor_op_f32_auto.cu) sample unit test, it takes a different tensor operation class:
+ 
+```c++
+  using ElementA = cutlass::mx_float8_t<cutlass::float_e4m3_t>;
+  using ElementB = cutlass::mx_float8_t<cutlass::float_e4m3_t>;
+```
+
+are needed in the mainloop builder:
+
+```c++
+  using CollectiveMainloop = typename cutlass::gemm::collective::CollectiveBuilder<
+      cutlass::arch::Sm100, cutlass::arch::OpClassTensorOp,
+      ElementA, GmemLayoutA, 16,
+      ElementB, GmemLayoutB, 16,
+      ElementAccumulator,
+      MmaTileShape_MNK, ClusterShape_MNK,
+      cutlass::gemm::collective::StageCountAutoCarveout<static_cast<int>(sizeof(typename CollectiveEpilogue::SharedStorage))>,
+      cutlass::gemm::KernelScheduleAuto
+    >::CollectiveOp;
+```
+
+We encourage a user to refer to Sm100 unit tests and the generated profiler-based kernels as more comprehensive samples.
+
 # Copyright

 Copyright (c) 2017 - 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.