chenchenghao/DeepGEMM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
207 Commits 1 Branch 0 Tags
8ad348fb1185ae873f2180263fa87b681d87eeba
Commit Graph

207 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
LuminolT
8ad348fb11 feat(megamoe): add fp4 group16 to group32 requant path 
Add a utility and synthetic benchmark path for converting FP4 group16 tensors into group32 tensors that the existing SM100 block32 MegaMoE kernels can consume.

Document that this is a requantization path rather than a lossless metadata rewrite, so GLM-5.2 accuracy validation is still required before production use.

Tested: PYTHONPYCACHEPREFIX=/private/tmp/deepgemm_pycache python3 -m py_compile deep_gemm/utils/math.py tests/test_layout.py tests/test_mega_moe.py

Tested: git diff --check

Not-tested: CUDA build, SM100/B300 runtime, and GLM-5.2 accuracy validation are not available locally.
 2026-07-08 18:34:33 +08:00
LuminolT
79fcfd6abf feat(megamoe): add nvfp4 group16 capability gate 
Allow SM100 FP4 scale layout transforms to accept group16 and thread weight granularity through the MegaMoE Python wrapper, API checks, and synthetic benchmark entrypoint.

Keep fused SM100 MegaMoE compute behind an explicit group16 capability gate until the SFB/TMEM/MMA scale path is updated and validated.

Tested: PYTHONPYCACHEPREFIX=/private/tmp/deepgemm_pycache python3 -m py_compile deep_gemm/mega/__init__.py tests/test_mega_moe.py tests/generators.py

Tested: git diff --check

Not-tested: CUDA build and SM100/B300 runtime validation are not available locally.
 2026-07-08 18:29:09 +08:00
Xinyi Liu
0ff91f1285 docs: record sm90 megamoe phase6 dev log 2026-06-18 17:42:09 +08:00
Xinyi Liu
453fc7b046 feat: implement sm90 megamoe phase6 combine 2026-06-18 17:40:49 +08:00
Xinyi Liu
a0b3bb0017 docs: record sm90 megamoe indentation fix log 2026-06-18 15:54:59 +08:00
Xinyi Liu
71f012a356 style: fix sm90 megamoe dispatch indentation 2026-06-18 15:54:00 +08:00
Xinyi Liu
6b33b41c4a docs: record sm90 megamoe phase5 dev log 2026-06-18 15:18:28 +08:00
Xinyi Liu
9bd0519605 feat: implement sm90 megamoe phase5 l2 scatter 2026-06-18 15:17:20 +08:00
Xinyi Liu
fc8218750c docs: record sm90 megamoe phase4 dev log 2026-06-18 01:11:09 +08:00
Xinyi Liu
f3553f976c feat: implement sm90 megamoe phase4 l1 epilogue 2026-06-18 01:09:45 +08:00
Xinyi Liu
2bb1756787 docs: record sm90 megamoe phase3 dev log 2026-06-18 00:48:12 +08:00
Xinyi Liu
842124b1c5 feat: implement sm90 megamoe phase3 l1 wgmma 2026-06-18 00:46:41 +08:00
Xinyi Liu
ed6067c7a5 docs: record sm90 megamoe phase2 dev log 2026-06-18 00:01:30 +08:00
Xinyi Liu
540e5aeadc feat: implement sm90 megamoe phase2 dispatch-only 2026-06-18 00:00:36 +08:00
Xinyi Liu
74eb59dfaa docs: record sm90 megamoe phase1 dev log 2026-06-17 23:56:31 +08:00
Xinyi Liu
1dafd0f173 feat: add sm90 megamoe phase1 interfaces 2026-06-17 23:54:49 +08:00
Xinyi Liu
062cb160cf Phase 0: SM90 MegaMoE design doc, reference baseline, nsys script 
- MEGAMOE_SM90_DESIGN.md: complete design document with finalized decisions
  (fused single kernel, cooperative + single-WG, dynamic BLOCK_M, etc.)
- tests/test_mega_moe_sm90.py: PyTorch FP32/BF16 reference implementation
  for dispatch → L1 GEMM → SwiGLU → L2 GEMM → combine pipeline
- scripts/run_nsys_mega_moe_sm90.sh: nsys profiling wrapper script
- megamoe-research-reports/: research analysis of PR304/323/347/352/357/360
 2026-06-16 18:01:12 +08:00
Chenggang Zhao
714dd1a4a9 Update test_mega_moe.py 2026-05-11 19:20:18 +08:00
Zhean Xu
891d57b4db Add various optimizations and Mega MoE benchmarks (#316) 
* Merge with private repo

* Add Mega MoE Benchmark

* Minor fix

* Update

---------

Co-authored-by: Chenggang Zhao <chenggangz@deepseek.com>
 2026-04-24 18:41:37 +08:00
Chenggang Zhao
7f2a703ed5 [Public release 26/04] Introducing Mega MoE, FP4 Indexer and other features/fixes (#304) 
* Merge with private repo

* Update README

* Update README

* Update README

* Add PyTorch requirements

* Fix sync scopes for MQA logits (#256)

* Update README
 2026-04-17 09:45:14 +08:00
Ray Wang
d30fc36c8f Fix sync issue of TMEM alloc/dealloc (#292) 2026-03-22 16:41:28 +08:00
Xin Qiu
35c4bc8771 fix: k_grouped_fp8_gemm_nt_contiguous crashes with n = 768 on H100 (#238) 2026-02-25 10:13:54 +08:00
Ray Wang
477618cd51 Fix a sync issue in SM100 MQA logits (#285) 2026-02-03 17:29:49 +08:00
Zhean Xu
0f5f266202 Multiple updates and refactorings (#280) 2026-01-16 17:06:52 +08:00
Zhean Xu
3ccf40c53a Merge pull request #270 from yurekami/fix/sm90-archspec-bug 
fix: use SM90ArchSpec instead of SM100ArchSpec in sm90_bf16_k_grouped_gemm
 2026-01-06 09:56:33 +08:00
yurekami
6be0eb31d9 fix: use SM90ArchSpec instead of SM100ArchSpec in sm90_bf16_k_grouped_gemm 
The function sm90_bf16_k_grouped_gemm was incorrectly using SM100ArchSpec
to calculate TMA descriptor block sizes. Since this file is the SM90
implementation, it should consistently use SM90ArchSpec like the other
functions in this file (sm90_bf16_gemm, sm90_m_grouped_bf16_gemm_contiguous,
etc.).

This fixes a copy-paste error that could cause incorrect block size
calculations on SM90 (Hopper) GPUs.

Fixes #242

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-01-01 05:06:36 +09:00
Chenggang Zhao
9b680f4284 Update install.sh 2025-12-05 17:06:48 +08:00
AJ WISE
659a279bbd Better error handling, code consistency, compile-time safety (#234) 2025-12-05 16:49:52 +08:00
Ray Wang
38f8ef73a4 Multiple updates and refactorings (#231) 2025-11-21 17:49:47 +08:00
Zhean Xu
bb4424aad4 Fix sum_k * shape_m overflow 2025-11-19 11:51:36 +08:00
Chenggang Zhao
8da33d6bd9 Clean up 2025-11-19 11:00:55 +08:00
Guoteng
f63d7f24d6 fix: prevent int32 overflow in k-grouped GEMM size calculations (#226) 2025-11-19 10:52:08 +08:00
Ray Wang
ec5e9ed0b8 Fix SM90 MQA logits (#229) 2025-11-19 10:50:36 +08:00
Ray Wang
2f9d87877e Use larger MMA shape (#227) 2025-11-14 11:38:15 +08:00
oliver könig
c9f8b34dcd Merge pull request #220 from ko3n1g/ko3n1g/chore/revert-name-change 
Ko3n1g/chore/revert name change
 2025-10-15 16:30:23 +02:00
oliver könig
237041a257 revert 
Signed-off-by: oliver könig <okoenig@nvidia.com>
 2025-10-15 14:29:57 +00:00
oliver könig
f82018273d chore: Revert name change 
Signed-off-by: oliver könig <okoenig@nvidia.com>
 2025-10-15 14:29:16 +00:00
oliver könig
737e420fad chore: Rename project to ds-deem-gemm 
Signed-off-by: oliver könig <okoenig@nvidia.com>
 2025-10-15 12:44:21 +00:00
oliver könig
2b8a8e24f8 Update publish.yml 2025-10-15 13:00:51 +02:00
oliver könig
9528451969 Ko3n1g/chore/rename to deepgemm (#217) 
* py3.8

Signed-off-by: oliver könig <okoenig@nvidia.com>

* chore: Rename from `deep_geem` to `deepgemm`

Signed-off-by: oliver könig <okoenig@nvidia.com>

---------

Signed-off-by: oliver könig <okoenig@nvidia.com>
 2025-10-15 18:13:42 +08:00
oliver könig
93b3c28fa8 ci: Fixes for pre-built wheels (#214) 
* build: Allow NGC builds

Signed-off-by: oliver könig <okoenig@nvidia.com>

* reduce grid

Signed-off-by: oliver könig <okoenig@nvidia.com>

* update grid

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* upgrade cuda action

Signed-off-by: oliver könig <okoenig@nvidia.com>

* remove test

Signed-off-by: oliver könig <okoenig@nvidia.com>

* py3.8

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* exclude

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* torch-version

Signed-off-by: oliver könig <okoenig@nvidia.com>

* py3.8/torch2.1/cuda12.3

Signed-off-by: oliver könig <okoenig@nvidia.com>

* Update publish.yml

* fix grid

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* cuda11.8

Signed-off-by: oliver könig <okoenig@nvidia.com>

* no hopper for 118

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

---------

Signed-off-by: oliver könig <okoenig@nvidia.com>
 2025-10-14 13:05:47 +08:00
Chenggang Zhao
f8f41145da Use CUDA runtime API to get device prop instead of ATen 2025-10-11 09:16:31 +08:00
oliver könig
9f196058ae chore: Build and store bdist wheels (#181) 
* build: Minor tweeks for wheel build

Signed-off-by: oliver könig <okoenig@nvidia.com>

* ci: Workflows for wheel build

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* fix

Signed-off-by: oliver könig <okoenig@nvidia.com>

* build: Add CachedWheel

Signed-off-by: oliver könig <okoenig@nvidia.com>

* add version to init

Signed-off-by: oliver könig <okoenig@nvidia.com>

* revert

Signed-off-by: oliver könig <okoenig@nvidia.com>

* revert

Signed-off-by: oliver könig <okoenig@nvidia.com>

* revert

Signed-off-by: oliver könig <okoenig@nvidia.com>

* v2

Signed-off-by: oliver könig <okoenig@nvidia.com>

* update

Signed-off-by: oliver könig <okoenig@nvidia.com>

* test

Signed-off-by: oliver könig <okoenig@nvidia.com>

* from packaging.version import parse

Signed-off-by: oliver könig <okoenig@nvidia.com>

* local version

Signed-off-by: oliver könig <okoenig@nvidia.com>

* remove file

Signed-off-by: oliver könig <okoenig@nvidia.com>

* revert

Signed-off-by: oliver könig <okoenig@nvidia.com>

* Updates and lint

* revert missing cudaextension args

Signed-off-by: oliver könig <okoenig@nvidia.com>

* Add timeout

* fix DG settings

Signed-off-by: oliver könig <okoenig@nvidia.com>

* DG_USE_LOCAL_VERSION

Signed-off-by: oliver könig <okoenig@nvidia.com>

* Update version

* Detect local changes

* Minor fix

* Revert CUTLASS

* Unify options

---------

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Chenggang Zhao <chenggangz@deepseek.com>
 2025-10-10 18:23:40 +08:00
Jun Jiang
6e74faad5c Upgrade to CUTLASS 4.2.1 (#203) 2025-10-09 09:09:22 +08:00
PGFLMG
239112cb4c Fix syntax errors and correct the conditional statements (#206) 2025-10-01 20:31:43 +08:00
Chenggang Zhao
c1bf4cae4b Fix version 2025-10-01 20:31:27 +08:00
Chenggang Zhao
07b82fb8cd Fix old CUDA compatibility 2025-10-01 20:29:15 +08:00
Chenggang Zhao
594953acce Update version number 2025-09-29 17:12:21 +08:00
Chenggang Zhao
0ed3b949d0 Update README 2025-09-29 17:10:12 +08:00
Simon Mo
59f2c07cf2 Add SM100 kernels (#201) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-09-29 17:07:28 +08:00