Lianmin Zheng 5493c3343e Fix data parallel + tensor parallel (#4499)

2025-03-17 05:13:16 -07:00

..

attention

Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418)

2025-03-17 00:03:43 -07:00

moe

Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418)

2025-03-17 00:03:43 -07:00

quantization

Support Online Quantization for W8A8 (#4485)

2025-03-17 00:28:56 -07:00

activation.py

cleanup deps 2/n (#4464)

2025-03-15 23:06:17 -07:00

dp_attention.py

Fix data parallel + tensor parallel (#4499)

2025-03-17 05:13:16 -07:00

elementwise.py

Add some fused elementwise kernels for grok-1 (#4398)

2025-03-13 13:39:10 -07:00

layernorm.py

Urgent model support: support gemma-3-it (#4424)

2025-03-16 17:37:32 -07:00

linear.py

Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

logits_processor.py

Improve DP attention (#4390)

2025-03-13 08:23:56 -07:00

parameter.py

Apply sgl w8a8 fp8 kernel (#3148)

2025-03-09 00:03:32 -08:00

pooler.py

Rename InputMetadata -> ForwardBatch (#1543)

2024-09-30 02:41:11 -07:00

radix_attention.py

[QUANT] Add GPTQModel Dynamic Quantization + lm_head Quantization (#3790)

2025-03-05 01:11:00 -08:00

rotary_embedding.py

Urgent model support: support gemma-3-it (#4424)

2025-03-16 17:37:32 -07:00

sampler.py

Move output processing logic from scheduler.py into a separate file (#4354)

2025-03-12 16:21:49 -07:00

torchao_utils.py

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

vocab_parallel_embedding.py

Clean up fp8 support (#4230)

2025-03-09 21:46:35 -07:00