4 Commits

Author	SHA1	Message	Date
Yineng Zhang	6b8f66efe1	misc: update cuda graph capture exception log (#894)	2024-08-03 00:40:52 +10:00
Liangsheng Yin	6b0f2e9088	Add --max-total-tokens (#840)	2024-07-30 13:33:55 -07:00
Ying Sheng	e7487b08bc	Adjust default mem fraction to avoid OOM (#823)	2024-07-30 01:58:31 -07:00
Liangsheng Yin	cdcbde5fc3	Code structure refactor (#807)	2024-07-29 23:04:48 -07:00