Lianmin Zheng bd6196163e Small fix for the order of apply_torchao_config (#2495)

2024-12-16 19:21:11 -08:00

..

configs

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

constrained

[Minor] Improve code style (#2422)

2024-12-09 06:30:35 -08:00

distributed

adapt vllm distributed module to sglang (#2244)

2024-12-01 15:54:52 +08:00

layers

Revert "Small fixes for torchao quant" (#2493)

2024-12-16 15:04:16 -08:00

lora

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

managers

Add a benchmark script for in-batch prefix caching (#2494)

2024-12-16 18:49:02 -08:00

mem_cache

[Core] in batch prefix caching by delay scheduling (#2442)

2024-12-11 12:51:50 -08:00

metrics

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_executor

Small fix for the order of apply_torchao_config (#2495)

2024-12-16 19:21:11 -08:00

model_loader

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

models

[Minor] Fix grok model loader (#2473)

2024-12-12 14:34:47 -08:00

openai_api

Fix logprob for completions (#2301)

2024-12-01 05:17:05 -08:00

sampling

Fix the overlap for xgrammar (#2377)

2024-12-06 05:49:29 -08:00

_custom_ops.py

adapt vllm distributed module to sglang (#2244)

2024-12-01 15:54:52 +08:00

conversation.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

hf_transformers_utils.py

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

server_args.py

Simplify pytorch sampling kernel and logit processor (#2491)

2024-12-16 14:11:09 -08:00

server.py

[Minor] Improve code style (#2422)

2024-12-09 06:30:35 -08:00

utils.py

hotfix: checking for HIP (#2485)

2024-12-15 02:47:26 +08:00