[AMD][Quantization] Add int4fp8_moe online quantization on ROCm (#7392)
Co-authored-by: Dehua Tang <dehtang@amd.com> Co-authored-by: HAI <hixiao@gmail.com> Co-authored-by: YC Tseng <yctseng@amd.com>
This commit is contained in:
@@ -120,6 +120,12 @@ FP8_E4M3_MIN = -FP8_E4M3_MAX
|
||||
builtins.FP8_E4M3_MAX = FP8_E4M3_MAX
|
||||
builtins.FP8_E4M3_MIN = FP8_E4M3_MIN
|
||||
|
||||
# explicitly use pure text format, with a newline at the end
|
||||
# this makes it impossible to see the animation in the progress bar
|
||||
# but will avoid messing up with ray or multiprocessing, which wraps
|
||||
# each line of output with some prefix.
|
||||
BAR_FORMAT = "{desc}: {percentage:3.0f}% Completed | {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]\n" # noqa: E501
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def is_cuda():
|
||||
|
||||
Reference in New Issue
Block a user