-
Notifications
You must be signed in to change notification settings - Fork 19
Open
Description
To improve the 8bit quantization in the RTL mode with ½ LSB and round it to the nearest LSB I would suggest to replace the lines
*(data++) = (unsigned char)(((*xi << sample_shift) >> 8) + 128);
*(data++) = (unsigned char)(((*xq << sample_shift) >> 8) + 128);
with
*(data++) = (unsigned char)((((*xi << sample_shift) + 0x80) >> 8) + 128);
*(data++) = (unsigned char)((((*xq << sample_shift) + 0x80) >> 8) + 128);
or even more optimized
*(data++) = (unsigned char)(((*xi << sample_shift) + 0x8080) >> 8);
*(data++) = (unsigned char)(((*xq << sample_shift) + 0x8080) >> 8);
Cheers
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels