leavelet/cutlass
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
858 Commits 1 Branch 1 Tag
4ca61d0662bbc835c98dccfca022c8265edd9022
Commit Graph

157 Commits

Author SHA1 Message Date
Junkai-Wu
1b741cabaa v4.4.2 update. (#3104) 2026-03-17 00:58:19 -04:00
dePaul Miller
73c59c055c Support for Group GEMM in CUTLASS Profiler for Geforce and Spark (#3092) 
Co-authored-by: dePaul Miller <23461061+depaulmillz@users.noreply.github.com>
 2026-03-06 20:36:29 -05:00
Junkai-Wu
3bb6e28d3c v4.4.1 update (#3079) 2026-02-27 13:59:21 -05:00
Junkai-Wu
d4bbf728ca v4.4 tag release update. (#3032) 2026-02-13 23:27:58 -05:00
Junkai-Wu
6b3e607b85 v4.4 release update v2. (#2999) 2026-02-03 20:48:31 -05:00
Junkai-Wu
9fba3195f9 v4.4 update. (#2979) 2026-01-24 11:46:17 -05:00
Qi Yuhang
2fafefb7b9 [Bug Fix]Set NumSplitsM to 1 when TileShapeM < 128 in sm90 fp8 blockwise scaling CollectiveMma (#2965) 
* Fix NumSplitsM when TileShapeM < 128.

* Use cute::conditional_t to replace std::conditional_t.
 2026-01-23 15:56:52 +08:00
Junkai-Wu
0d2b201e8c v4.3.5 update. (#2934) 
* v4.3.5 update.

* Update copyright to 2026
 2026-01-08 15:02:56 -05:00
veritas-Qiu
61b560983a remove useless line (#2926) 
the parameter workspace is marked as unused like other kernels, but it is actually used after 3.3.0, so the code which mark it as unused could be removed.
 2026-01-06 23:54:08 -05:00
Qi Yuhang
ebf3165efb [Bug Fix]Bypass launch grids for SM120 Kernel with SM90 Mainloop & SM100 TileScheduler (#2865) 
* Delete unused #ifdef/#endif. Bypass sm120 case.

* Add todo.

* Fix pingpong.

* Revert "Add todo."

This reverts commit 246cb42091b1ed1b89c1eac2312eed6625a754fd.

* Refine name.

Refine name again.

* Apply suggestions from code review

Skip `is_last_tile` for all sm120 kernels.

Co-authored-by: Junkai-Wu <junkaiw@nvidia.com>

* Skip early stop for sm120 kernel.

* Fix typo.

---------

Co-authored-by: Junkai-Wu <junkaiw@nvidia.com>
 2025-12-18 08:51:38 +08:00
HydraQYH
95f8beb44c Revert "Remove unnecessary #ifdef #endif for general gemm." 
This reverts commit 17ffd56dfe.
 2025-12-09 11:52:23 +08:00
HydraQYH
17ffd56dfe Remove unnecessary #ifdef #endif for general gemm. 2025-12-06 10:20:35 +08:00
HydraQYH
ff7f2dcdfb Remove duplicated cutlass::arch::wait_on_dependent_grids(); 2025-12-06 10:20:35 +08:00
HydraQYH
929e1e0259 Remove unnecessary #ifdef / #endif for launch_dependent_grids. 2025-12-06 10:20:35 +08:00
HydraQYH
b6ad6db219 Delete unnecessary #ifdef / #endif. 2025-12-06 10:20:35 +08:00
HydraQYH
e1b2ec57e3 Hoist waits above the warp specialized region. 2025-12-06 10:20:35 +08:00
HydraQYH
1e5f95cbbe Support PDL in sm90_gemm_array_tma_warpspecialized_cooperative 2025-12-06 10:20:35 +08:00
HydraQYH
acf5990cc2 Refine position for wait_on_dependent_grids. 2025-12-06 10:20:35 +08:00
HydraQYH
91de7891a5 Support PDL in sm90_gemm_array_tma_warpspecialized_pingpong.hpp 2025-12-06 10:20:35 +08:00
Shreya Gaur
af8d5dfa54 bug fix for example 92 (#2830) 
Co-authored-by: Shreya Gaur <shgaur@dc2-container-xterm-012.prd.it.nvidia.com>
Co-authored-by: Shreya Gaur <shgaur@2u2g-spr-0015.ipp4a1.colossus.nvidia.com>
 2025-12-01 22:02:59 -05:00
Junkai-Wu
1de3a576cc v4.3.1 update. (#2817) 2025-11-27 09:49:30 -05:00
Shreya Gaur
2052fd3885 Blockscaled Ragged Contiguous Grouped Gemm for MoEs (#2790) 
* Adding blockscaled ragged contiguous grouped gemm for MoEs

* cleaning up the example

* introduction to example improved

---------

Co-authored-by: Shreya Gaur <shgaur@dc2-container-xterm-012.prd.it.nvidia.com>
 2025-11-26 20:16:49 -05:00
Junkai-Wu
8cd5bef43a v4.3 tag release update. (#2789) 2025-11-20 20:49:44 -05:00
Ali Hassani
d1ef0e87f2 DistGEMM bug fixes (#2713) 
* Blackwell DistGEMM bug fixes

1. If using preferred cluster, there needs to be a branch so that
   the universal GEMM wrapper finds the correct base params.
2. Workspace sizes can change depending on problem shape in Blackwell,
   and DistGEMM was previously using the per-device shape to evaluate
   workspace size instead of the per-gemm shape.
3. Flattened size used to initialize host tensors can overflow (in
   Hopper example as well)
4. Preferred and fallback cluster args need to be set explicitly,
   otherwise if someone modifies the example to use preferred cluster,
   it will just fail.

* Fix example runtimes

* Set default fallback cluster shapes to the static ones
 2025-11-06 13:31:24 -05:00
ANIKET SHIVAM
020c700e97 support for K=0 for sm100 GG (#2746) 2025-11-04 11:25:39 -05:00
Qi Yuhang
b2ca083d2b Fixed compilation error when using StreamK scheduler + PDL. (#2686) 2025-10-21 23:11:14 -04:00
Junkai-Wu
b1d6e2c9b3 v4.3 update. (#2709) 
* v4.3 update.

* Update the cute_dsl_api changelog's doc link

* Update version to 4.3.0

* Update the example link

* Update doc to encourage user to install DSL from requirements.txt

---------

Co-authored-by: Larry Wu <larwu@nvidia.com>
 2025-10-21 14:26:30 -04:00
Lain
e6e2cc29f5 fix (#2684) 2025-10-15 14:46:38 -04:00
103yiran
fd0312ddf6 Remove duplicate function calls (#1584) 2025-09-21 23:16:59 -04:00
Junkai-Wu
6a35b4d22f v4.2 tag release. (#2638) 2025-09-15 12:21:53 -04:00
Javier
496654bf2c Fix sm100 gemm wrong static constexpr that breaks compilation on Windows (#2167) 
* Fix a sm100 gemm wrong defined static constexpr that breaks compilation on Windows

* Fix a sm100 gemm wrong defined static constexpr that breaks compilation on Windows

* More Windows fixes

Signed-off-by: Javier <25750030+SystemPanic@users.noreply.github.com>

* Revert "More Windows fixes"

This reverts commit 2e8cfc13827fbc7e3f2475bbc83694c2e1514107.

---------

Signed-off-by: Javier <25750030+SystemPanic@users.noreply.github.com>
 2025-08-28 22:13:00 -04:00
Junkai-Wu
a49a78ffef v4.2 release. (#2587) 
* Fix default cluster callback values to 1 to avoid profiler failure when these values are not set in command line.

* v4.2 release.
 2025-08-22 18:11:24 -04:00
starwang1024
9e6ab77d27 Fix a copy error in the SM70 main loop when loading data from smem to rmem (#2540) 2025-08-10 22:42:01 -04:00
Junkai-Wu
fd6cfe1ed0 v4.1 release update v2. (#2481) 2025-07-21 22:03:55 -04:00
Junkai-Wu
a1aaf2300a v4.1 release 2025-07-03 08:07:53 -04:00
Junkai-Wu
8bdbfca682 v4.0 update. (#2371) 2025-06-06 02:39:20 -04:00
Kihiro Bando
f115c3f854 Release v4.0.0 (#2294) 2025-05-13 15:55:29 -04:00
Jiazhen Han
89f6bf2739 Fix group scale gemm when K==128 (#2275) 
Co-authored-by: Jiazhen Han <jiazhenh@nvidia.com>
 2025-05-02 15:41:18 -04:00
Qi Yuhang
e5b810bed1 Use cudaMemcpyAsync in gemm grouped with kRequiresPrecomputation schedule. (#2256) 
Co-authored-by: Yuhang Qi <qiyuhang@bytedance.com>
 2025-04-30 15:28:05 -04:00
Lain
2b78c2fe31 cherry-pick feature/hopper-blockwise-generalization-optimization (#2270) 2025-04-29 16:47:22 -04:00
Haicheng Wu
697126019e fix blackwell grouped groupwise hang (#2267) 2025-04-29 11:54:20 -04:00
Yujia Zhai
331a1f5b3f cutlass 3.9 update (#2255) 
* cutlass 3.9 update

* rebase

* fixes out of shared memory for blockwise Blackwell

* doc format

* fix issue 2253

* disable host ref by default

* fix sm120 smem capacity

---------

Co-authored-by: yuzhai <yuzhai@nvidia.com>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2025-04-24 15:42:40 -04:00
Tri Dao
ade6376fa0 [SM90] Change register allocation for TileN=208 to avoid spills (#2219) 
With the usual register allocation (producer 40, consumer 232) compiling Gemm
with tile shape 256 x 208 (cooperative) or 128 x 208 (pingpong) show lots of
register spilling (e.g. ~3000 bytes spill). For this case we can change
the register allocation to producer 24, consumer 240, which avoids spills.
 2025-04-21 00:02:30 -04:00
Yujia Zhai
79fc51f4b8 v3.9 update (#2213) 
Co-authored-by: yuzhai <yuzhai@nvidia.com>
 2025-04-03 02:10:16 -04:00
Yujia Zhai
6f4921858b v3.9 update (#2203) 
* v3.9 update

* voidD

---------

Co-authored-by: yuzhai <yuzhai@nvidia.com>
 2025-04-02 15:11:18 -04:00
Yujia Zhai
62750a2b75 v3.9 (#2185) 
* v3.8 update x

* fix blackwell gg

* doc change

* doc change

* doc change

---------

Co-authored-by: yuzhai <yuzhai@nvidia.com>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
Co-authored-by: Haicheng Wu <57973641+hwu36@users.noreply.github.com>
 2025-03-21 01:52:23 -04:00
Jack Kosaian
6c6b78550e Fix SM90 beta=1 hang and stream-K launch errors (#2172) 
* Fix stream-K occupancy calculation

* Fix beta=1 hang
 2025-03-13 14:07:37 -04:00
dePaul Miller
06e560d98a Blockwise/Groupwise kernel improvement and programatic dependent launch enablement (#2161) 
Co-authored-by: dePaul Miller <23461061+depaulmillz@users.noreply.github.com>
 2025-03-10 14:36:11 -04:00
Lucas Wilkinson
df18f5e4f5 Improvements for: Groupwise scaling along M for FP8 gemm (#2095) 
* fix blockwise fp8 kernels

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* wip, < 128 not working

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* fix < 128

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* reduce diff

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* review comments

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* support partial n blocks

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

* fix build errors

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

---------

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-02-27 22:39:29 -05:00
dePaul Miller
ca4fdbea70 Blockwise and Groupwise GEMM for Blackwell and Improvements for Hopper (#2139) 
- Blockwise and Groupwise GEMM improvements for Hopper.
- Blockwise and Groupwise GEMM for Blackwell.
- Blockwise Grouped GEMM for Hopper.
- Static ScalePromotionInterval for Hopper FP8 GEMMs.

Co-authored-by: dePaul Miller <23461061+depaulmillz@users.noreply.github.com>
 2025-02-26 12:44:58 -05:00