chenchenghao/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,169 Commits 1 Branch 0 Tags
770529a73172499a4d3e6135b1c63f70d63e1f5d
Commit Graph

259 Commits

Author SHA1 Message Date
cctry
b0b4f71679 [Fix] memory leak by overlap + retract (#11981) 
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
 2025-10-23 22:59:23 +08:00
Liangsheng Yin
9d61205dac [lint] improve ruff check (#11922) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-10-22 11:32:50 +08:00
huangtingwei
cae3956585 check master server for mooncake store (#10510) 2025-10-20 09:37:09 +08:00
Teng Ma
585e1223f0 [HiCache] feat: add more eviction policy (#11506) 2025-10-18 15:49:45 +00:00
Chang Su
627974405d [Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685) 2025-10-17 16:49:46 -07:00
Baizhou Zhang
b0d1d717e1 Revert "make radix cache deterministic" (#11728) 2025-10-16 14:36:15 -07:00
Even Zhou
3cceaa381a [Bugfix] Fix Qwen3/DSV3/DSV3.2 model support (#11510) 2025-10-16 15:14:09 +08:00
Shangming Cai
868403f642 [PD] Add PD support for hybrid model (Qwen3-Next, DeepSeek V3.2 Exp) (#10912) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2025-10-15 18:59:14 -07:00
cctry
1d7f783501 Refactor kv cache free (#11351) 2025-10-14 17:45:19 -07:00
Alex Chi Z
dc965db0e0 make radix cache deterministic (#10721) 
Signed-off-by: Alex Chi Z <iskyzh@gmail.com>
 2025-10-14 21:01:52 +08:00
Liangsheng Yin
516738b096 Depreate global_server_args_dict (#11528) 2025-10-13 19:34:43 +08:00
hzh0425
318424e2c8 [HICache]: Support 3FS-Store with page_first_direct layout (#11460) 2025-10-13 15:47:22 +08:00
Yi Zhang
a55cf5304a [Feature] Support mamba radix cache v0 (#11214) 
Co-authored-by: hanming-lu <hanming@x.ai>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: thalahors <ericalcaide1@gmail.com>
 2025-10-12 20:57:15 -07:00
Yuanhang Sun
19ba16aa3d [Fix]: add missing device attribute to ChunkCache (#11493) 2025-10-12 20:49:59 -07:00
Cheng Wan
1bdd010291 Revert "Deprecate global_server_args_dict" (#11520) 2025-10-12 17:40:40 -07:00
Liangsheng Yin
1083e7e3df Deprecate global_server_args_dict (#11331) 2025-10-13 01:20:47 +08:00
hzh0425
f5b34a510c Bugfix: Fix Type consistency for KV indices in SWARadixCache (#11452) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-10-12 23:19:44 +08:00
cctry
b36afed4a7 Separate allocation logic from scheduler (#11313) 2025-10-10 17:38:54 -07:00
JinYan Su
9aa4502d11 feat(mooncake): support GB suffix for global_segment_size (#10745) 
Signed-off-by: Jinyang Su <751080330@qq.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2025-10-10 17:38:25 -07:00
Teng Ma
9082a7d323 [HiCache] feat: add multi tenant with prefix tag (#9256) 2025-10-11 00:23:28 +08:00
ziruiliu
8df4945559 fix file and object naming scheme in HiCacheNixl to avoid data corruption (#10969) 
Signed-off-by: Zirui Liu <ziliu@ddn.com>
 2025-10-10 00:23:10 -07:00
hzh0425
ee3bd8a1c8 feat(hicache): Support passing prefix keys for l3 store. (#9045) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-10 00:22:05 -07:00
Lianmin Zheng
9b8ebb2798 move more files under srt/utils (#11285) 2025-10-09 16:46:15 -07:00
Trevor Morris
a4b424c632 [DeepSeek-V3.2] Include indexer kv cache when estimating kv cache size (#11309) 2025-10-08 23:59:46 -07:00
Netanel Haber
d6837aea4d model: Support Hybrid Mamba2 NemotronHForCausalLM (nvidia/NVIDIA-Nemotron-Nano-9B-v2) (#10909) 
Signed-off-by: Netanel Haber <nhaber@nvidia.com>
 2025-10-09 00:37:38 +08:00
YAMY
5a9170d993 Optimize copy_kv_cache for spec decoding (#11126) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-10-08 10:43:30 +08:00
cctry
f3764c26a3 Clean match_prefix and prepare_for_extend for mem cache V2 (#11200) 2025-10-07 17:54:18 -07:00
Ke Bao
24bc3fb0f9 EAGLE cache fix for SWARadixCache (#11231) 
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-10-07 18:21:37 +08:00
fzyzcjy
efbc687c28 Support DeepSeek V3.2 Exp (#11061) 
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-06 00:24:15 -07:00
Ke Bao
31b49c0b51 EAGLE cache fix for HiCache (#11215) 2025-10-04 16:53:53 -07:00
pansicheng
08af8ffb5c fix 3fs indices (#10855) 2025-10-04 00:06:38 +08:00
Alex Chi Z
1a31229cd4 fix: radix cache memory accounting (#10637) 
Signed-off-by: Alex Chi Z <iskyzh@gmail.com>
 2025-10-02 22:47:33 -07:00
Liangsheng Yin
7d00479950 Clean up ascend allocator (#11152) 2025-10-02 20:34:26 +08:00
Zhang Junda
0b2aa8a70c Intoduce cpu tensor as metadata to avoid blocking gpu kernel launch (#10720) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2025-10-02 10:51:25 +08:00
ykwd
bfa274380b [HiCache] Configurable and Dynamic Prefetch Timeout (#10512) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-01 08:44:10 -05:00
Shisong Ma
86cb4db058 [Feature] Add EIC as sglang HiCache Storage backend (#10271) 
Co-authored-by: mashisong <mashisong@bytedance.com>
 2025-10-01 08:43:34 -05:00
Ke Bao
91847e382a Fix eagle radix cache (#10846) 2025-09-30 22:59:20 +08:00
huangtingwei
e05555fad8 [HiCacheStorage] mooncake store support page_first_direct layout (#10591) 2025-09-28 20:45:48 -07:00
Teng Ma
9816989bff [HiCache] bug: fix mooncake store batch set v1 (#11013) 2025-09-28 23:18:48 +08:00
hzh0425
c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
yi wang
fce170480a integrate AIBrix KVcache (#10376) 2025-09-25 14:47:09 +08:00
Zhiqiang Xie
3d40794fcf [HiCache] Cleaning the deprecated host memory state (#10778) 2025-09-25 14:43:53 +08:00
pansicheng
d4041a5eeb refactor zero copy (#10300) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-22 15:17:31 -07:00
Xinyuan Tong
12d6cf18f0 Refactors radix cache for extra key support (#10317) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-22 02:16:16 +08:00
huangtingwei
7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) 2025-09-19 05:47:16 -07:00
FlyPanda
8b713c7248 Hicache L3 backend mooncake optimization configuration reading method (#10319) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: shicang <shicang@shicang>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-19 12:25:01 +08:00
Xuchun Shang
1ccd59c715 [HICache] introduce evict policy (#10190) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-18 11:10:20 +08:00
Lianmin Zheng
f949ad5794 [Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-16 17:06:43 -07:00
ykwd
4bb08f6e07 [Hicache] Evaluate Per-Round Metrics in Multiturn Bench (#10203) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-15 19:34:40 -07:00
Binyao Jiang
9752861002 [Fix] Support qwen3-next MTP+DP (#10392) 2025-09-13 17:45:04 +08:00