 pansicheng
|
97b2a89334
|
[RadixTree][8/N Refactor]: unify lock interface (#20330)
|
2026-03-16 11:49:51 +08:00 |
|
|
cctry
|
027f314050
|
[Fix] data race in req_to_token pool (#17850)
|
2026-02-02 14:38:15 -08:00 |
|
|
zhangheng
|
f33022d039
|
[RadixTree][3/N Refactor]：Support unified insert/evict params (#17401)
|
2026-01-22 17:36:31 +08:00 |
|
|
Yi Zhang
|
236772c0e1
|
[RadixTree][2/N Refactor]: swa cache init tiny refactor (#17397)
|
2026-01-21 15:48:30 +08:00 |
|
|
|
20b0523eca
|
[RadixTree][1/N Refactor]: Support unified match_prefix params (#17142)
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
|
2026-01-19 22:39:40 +08:00 |
|
|
Ke Bao
|
ce8a6ac690
|
Evict swa kv cache during decoding (#17220)
|
2026-01-19 22:36:52 +08:00 |
|
|
Ke Bao
|
7f8a58fffb
|
Refactor prefix cache type checking (#17028)
|
2026-01-15 11:28:13 +08:00 |
|
|
Ke Bao
|
02722b9113
|
Fix evict swa for overlap scheduler and page size > 1 (#16507)
|
2026-01-06 22:29:40 +08:00 |
|
|
Ke Bao
|
76bc07a335
|
Move swa memory pool to a seperate file (#16347)
|
2026-01-04 22:39:30 +08:00 |
|
|
Ke Bao
|
7b7e357f61
|
Separate swa and local attention chunk cache eviction (#15820)
|
2025-12-26 09:34:22 +08:00 |
|
|
+6
|
160a06cab2
|
[Feature] Xiaomi MiMo-V2-Flash day0 support (#15207)
Co-authored-by: 谢学扬 <xiexueyang@xiaomi.com>
Co-authored-by: tz <tangzhen3@xiaomi.com>
Co-authored-by: 李家乐 <lijiale10@xiaomi.com>
Co-authored-by: 张晨 <zhangchen50@xiaomi.com>
Co-authored-by: Shaohui Liu <liushaohui3@xiaomi.com>
Co-authored-by: 王晨 <wangchen77@xiaomi.com>
Co-authored-by: jiangzihan <jiangzihan@xiaomi.com>
Co-authored-by: xiexueyang <xyxie_wangyi@163.com>
Co-authored-by: Linghao Zhang <zhanglinghao@xiaomi.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: JoyFuture <35593546+JoyFuture@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: root <root@bj9-ml-g8h20e-k8s-slave106-20251106.alicn.idc.xiaomi.com>
|
2025-12-19 11:40:07 +08:00 |
|
|
Liangsheng Yin
|
ecb401ed42
|
Enhance runtime memory check in CI (#15192)
|
2025-12-16 21:40:38 +08:00 |
|
|
Liangsheng Yin
|
b2f7b08c49
|
Refactor cache init logic (#13800)
|
2025-11-24 11:41:46 +08:00 |
|
|
Liangsheng Yin
|
665416f6dd
|
Unify memory management across (overlap, non-overlap) x (page>=1) x (spec, non-spec, spec v2) x (retract, finished) (#12224)
|
2025-11-11 02:56:22 +08:00 |
|
|
cctryandLiangsheng Yin
|
b0b4f71679
|
[Fix] memory leak by overlap + retract (#11981)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-10-23 22:59:23 +08:00 |
|
|
cctry
|
1d7f783501
|
Refactor kv cache free (#11351)
|
2025-10-14 17:45:19 -07:00 |
|
 leavelet
|
19ba16aa3d
|
[Fix]: add missing device attribute to ChunkCache (#11493)
|
2025-10-12 20:49:59 -07:00 |
|
|
cctry
|
f3764c26a3
|
Clean match_prefix and prepare_for_extend for mem cache V2 (#11200)
|
2025-10-07 17:54:18 -07:00 |
|
|
Lianmin Zhengandgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
f949ad5794
|
[Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
2025-09-16 17:06:43 -07:00 |
|
|
Zhiqiang Xie
|
54e872d343
|
[HiCache] resolve conflict between chunked-prefill and hicache hit count (#9776)
|
2025-08-30 01:30:54 +08:00 |
|
|
Lianmin Zhengandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
9e426466af
|
Clean up allocators (#9134)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-13 13:56:04 -07:00 |
|
|
Hanming LuandYing Sheng
|
9379da77de
|
SWA Prefix Cache (#7367)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
|
2025-07-13 12:31:07 -07:00 |
|
|
|
eb6c2c1663
|
Hybrid kv cache for LLaMA4 (#6563)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-06-27 18:58:55 -07:00 |
|
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
|
DarkSharpnessandZhiqiang Xie
|
47367b768d
|
[Refactor] Clean up radix cache related API (#7303)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-06-20 00:58:48 +08:00 |
|
|
Baizhou Zhang
|
d2679f5109
|
Fix ChunkCache object has no attribute 'disable' (#7217)
|
2025-06-15 20:55:15 -07:00 |
|
|
Byron Hsu
|
0a4fc73b48
|
[PD] Fix failure abort (#6535)
|
2025-05-22 20:32:03 -07:00 |
|
|
Zhiqiang Xie
|
f8e460930a
|
Fix prefill OOM error in the case of large page size (#5081)
|
2025-05-05 16:02:55 -07:00 |
|
|
Lianmin Zheng
|
c76040e31b
|
Support page size > 1 (#4356)
|
2025-03-12 22:22:39 -07:00 |
|
|
Ying ShengandKe Bao
|
d3d4d76758
|
[Eagle] Refactor eagle speculative decoding (#3986)
Co-authored-by: Ke Bao <ISPObaoke@163.com>
|
2025-03-05 08:06:07 -08:00 |
|
|
|
ac2387279e
|
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
|
2025-03-03 00:12:04 -08:00 |
|
|
Zhiqiang XieandLianmin Zheng
|
08104b56de
|
Sanity check to prevent performance regression (#3171)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-01-27 12:28:17 -08:00 |
|
|
SangBin Cho
|
9208618b3e
|
[Core] in batch prefix caching by delay scheduling (#2442)
|
2024-12-11 12:51:50 -08:00 |
|
|
Lianmin Zheng
|
769bf11c05
|
Fix the race condition in overlap mode (#1712)
|
2024-10-19 06:50:56 -07:00 |
|
|
Lianmin Zheng
|
9116b2896f
|
Add a new event loop (#1677)
|
2024-10-16 01:33:20 -07:00 |
|
|
Lianmin Zheng
|
c877292cc1
|
Re-organize CI tests (#1052)
|
2024-08-12 03:39:01 -07:00 |
|
|
Liangsheng Yin
|
fb7421db0d
|
minor: some potential bugs (#1044)
|
2024-08-12 05:35:44 +00:00 |
|
|
Lianmin Zheng
|
9dae407812
|
Improve type annotation (#1029)
|
2024-08-11 02:44:59 -07:00 |
|
|
Liangsheng Yin
|
fcc0f5ed99
|
Fix wrong assert (#1028)
|
2024-08-11 09:22:16 +00:00 |
|
|
Liangsheng Yin
|
43fbb6d919
|
Fix input_ids && rename to fill_ids (#1021)
|
2024-08-10 16:24:12 -07:00 |
|
|
Liangsheng Yin
|
6ed4e3b8fb
|
Fix chunked prefill (#984)
|
2024-08-07 22:28:42 -07:00 |
|
|
Liangsheng Yin
|
7623091d97
|
RadixCache method adjust (#977)
|
2024-08-07 15:52:24 -07:00 |
|
|
Liangsheng Yin
|
c020f9ceda
|
Support chunked prefill when radix cache is disabled (#811)
|
2024-08-01 00:29:01 -07:00 |
|