Lianmin Zheng 3a6e8b6d78 [Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

..

configs

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

constrained

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

layers

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

managers

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

mem_cache

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

model_executor

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

models

Support MiniCPM3 (#1371)

2024-09-10 19:57:52 +10:00

openai_api

Support OpenAI API json_schema response format (#1363)

2024-09-09 19:08:25 -07:00

sampling

Fix vocab mask update bug (#1376)

2024-09-10 13:10:36 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Add torchao quant (int4/int8/fp8) to llama models (#1341)

2024-09-09 05:32:41 -07:00

server.py

Fix CORS compatibility with OpenAI, vLLM, TGI, LMDeploy (#1373)

2024-09-11 02:35:03 +10:00

utils.py

fix: multimodal_config in monkey_patch_vllm_dummy_weight_loader (#1260)

2024-08-30 16:43:41 +10:00