mindspore_gs.ptq.GPTQQuantConfig

class mindspore_gs.ptq.GPTQQuantConfig(block_size=128, desc_act=False, damp_percent=0.01, static_groups=False)

Config for gptq quant algorithm.

Parameters

• block_size (int, optional) – The size of block compensation in precision recovery. Default value: 128.

• desc_act (bool, optional) – Whether to perform importance sorting on the Hessian matrix. Default value: False.

• damp_percent (float, optional) – The percentage of the average of the diagonal elements of the Hessian matrix during numerical stable computations. Default value: 0.01.

• static_groups (bool, optional) – Whether to perform per_group calculation before precision recovery in the GPTQ algorithm. Default value: False.

Raises

• TypeError – If block_size is not type int.

• TypeError – If desc_act is not type bool.

• TypeError – If damp_percent is not type float.

• TypeError – If static_groups is not type bool.

• ValueError – If block_size is less than 0.

• ValueError – If damp_percent is less than 0 or greater than 1.

Examples

>>> from mindspore_gs.ptq import GPTQQuantConfig
>>> GPTQQuantConfig(block_size=128, desc_act=False, damp_percent=0.01, static_groups=False)