HandH1998 0dd6cda288 Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

..

attention

Support nextn for flashinfer mla attention backend (#4218)

2025-03-09 00:01:54 -08:00

moe

[Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888)

2025-03-06 20:54:52 -08:00

quantization

Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

activation.py

fix: support gelu_new activation function in gpt2 (#3712)

2025-03-04 04:09:52 -08:00

dp_attention.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

layernorm.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

linear.py

Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

logits_processor.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

parameter.py

Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

pooler.py

Rename InputMetadata -> ForwardBatch (#1543)

2024-09-30 02:41:11 -07:00

radix_attention.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

rotary_embedding.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

sampler.py

[EAGLE] many fixes for eagle (#4195)

2025-03-07 22:12:13 -08:00

torchao_utils.py

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

vocab_parallel_embedding.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00