One would intuitively expect that the next token predictions from MTP layers become worse as you quantize them. So by determining the acceptance rate vs. the size one can create a benchmark for quantization formats. This would not be directly comparable between models but it would be very cheap to run so it may be useful as a proxy for more expensive but better benchmarks.
One would intuitively expect that the next token predictions from MTP layers become worse as you quantize them. So by determining the acceptance rate vs. the size one can create a benchmark for quantization formats. This would not be directly comparable between models but it would be very cheap to run so it may be useful as a proxy for more expensive but better benchmarks.