leavelet/cutlass
1
0
Fork 0
Code Issues Pull Requests Actions 24 Packages Projects Releases Wiki Activity
455 Commits 1 Branch 1 Tag
bef1fbcbe61fcebfb563ddcfe2fdd77f5f5206ee
Commit Graph
8 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
dan_the_3rddanthe3rd <danthe3rd>Haicheng Wu
 		146d314057 Update fMHA kernels (#992) 
* Update fMHA kernels

Upstream recent changes to fMHA that we did in xFormers.
Previous version in CUTLASS: facebookresearch/xformers@b6be33a
Updating to: facebookresearch/xformers@55a4798

* minor changes

* make var work

---------

Co-authored-by: danthe3rd <danthe3rd>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2023-07-12 22:30:46 -04:00
dan_the_3rddanthe3rd <danthe3rd>Haicheng Wu
 		9b8166e3f0 fMHA: Add backward pass (#844) 
* fMHA: Add backward pass

* Better checks for strides/alignments

* Remove fb-internal URL

* torch.Tensor.untyped_storage requires pytorch 2.0+

* minor changes

* make test

---------

Co-authored-by: danthe3rd <danthe3rd>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2023-04-06 20:44:58 -04:00
dan_the_3rdanddanthe3rd <danthe3rd> f396cdd15c ex24[gemm_grouped]: Allow to change layout/dtype (#841) 
* ex24[gemm_grouped]: Allow to change layout/dtype

* Address suggestion from @jackkosaian

---------

Co-authored-by: danthe3rd <danthe3rd>
 2023-03-01 07:13:51 -05:00
dan_the_3rddanthe3rd <danthe3rd>Haicheng Wu
 		f303889ed9 fMHA: Sync FW with xFormers (#828) 
* fMHA: Add support for bias+dropout in FW

* Remove 'getMaximumSharedMemoryPerBlockKb'

* fix comments

---------

Co-authored-by: danthe3rd <danthe3rd>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2023-02-22 23:25:31 -05:00
dan_the_3rddanthe3rd <danthe3rd>Haicheng Wu
 		2e10404d26 xFormer updates to fMHA FW (#773) 
* xFormer updates to fMHA FW

* Convert format to BMHK for '41_fused_multi_head_attention_fixed_seqlen'

* Add missing files

* Remove xFormers specific code

* Update fused_multihead_attention_fixed_seqlen.cu

* rebase and solve conflicts

* remove white space

---------

Co-authored-by: danthe3rd <danthe3rd>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2023-02-08 23:00:10 -05:00
dan_the_3rddanthe3rd <danthe3rd>Haicheng Wu
 		1b4e24470a Example 43 - DualGemm (#670) 
* Ex50 wip

* IS_PROFILING mode

* MultiStage2 - but is slower

* Add SwiGLU

* Support SplitKSerial reduction
Support not storing D0/D1
Cleanup code

* Option to disable bias

* Renumber example

* Fix build

* Remove references to pb_size_0 / pb_size_1

* Add support for bf16 inputs with float accum

* small changes

Co-authored-by: danthe3rd <danthe3rd>
Co-authored-by: Haicheng Wu <haichengw@nvidia.com>
 2022-10-26 14:04:42 -04:00
dan_the_3rdanddanthe3rd <danthe3rd> 4db6a6140e ex42: Fused MHA imported from xFormers (#662) 
* ex42: Fused MHA imported from xFormers

* Remove std:: references

* Support K>128 in the example

* Support causal option

* Support different head size for V, and different seqlength for KV

* Update FLOPS counter

* Remove bit_cast

* fix build: Replace M_LOG2E

* Add doc

* Revert "Remove bit_cast"

This reverts commit 9662fa86bb7c57c1a015ac0bf52cb52940fbbf80.

* Explicit casts to int32_t for windows build

Co-authored-by: danthe3rd <danthe3rd>
 2022-10-17 10:49:33 -04:00
dan_the_3rdanddanthe3rd 25ebf15d02 Ensure all arch::Mma specializations have ElementC set (#576) 
Co-authored-by: danthe3rd <danthe3rd@users.noreply.github.com>
 2022-07-22 23:53:03 -04:00