ardour/livetrax
13
0
Fork 0
Code

Fix sndfile_data_width use of low bits of subtype of libsndfile format

Browse Source 
The libsndfile format was masked with 0xf instead of the usual
SF_FORMAT_SUBMASK. It seems like the end result generally was correct
anyway, since no supported format used subtypes that used the low bits
for anything else. Most formats use SF_FORMAT_PCM subtypes. (Only Ogg
Vorbis uses a subtype, but that happens to have 0 in the low bits and
ended up in the "this will never happen branch" ... which happened to
work too.)

This could however be a real problem when SF_FORMAT_MPEG_LAYER_III with
value 0x82 is supported ... unless worked around in some way.

I don't see anything anywhere that could stuff anything in the high bits
of the subtype, so this trivially fixed by using SF_FORMAT_SUBMASK
correctly.
This commit is contained in:
Mads Kiilerich 2022-10-13 16:38:35 +02:00 committed by Robin Gareus
parent 6531b24e9a
commit 652a2aa3ac
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
libs/ardour/sndfile_helpers.cc
View File

@ -121,7 +121,7 @@ sndfile_endian_format_by_index (int index)
int
sndfile_data_width (int format)
{
int tval = format & 0xf;
int tval = format & SF_FORMAT_SUBMASK;
switch (tval) {
case SF_FORMAT_PCM_S8: