Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio: MDRC: Restructure Multiband DRC for more effective memory alloc… #9195

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ShriramShastry
Copy link
Contributor

@ShriramShastry ShriramShastry commented Jun 5, 2024

This check-in attempts to decrease memory allocation overhead, enhance cache efficiency through data locality, and lessen heap fragmentation. Within the MDRC component, combine memory allocations for the crossover, emphasis, and deemphasis filter coefficients into a single block.

By streamlining memory management, the update lowers the possibility of memory leaks.

@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch 2 times, most recently from d6d6854 to 012e192 Compare June 5, 2024 04:33
@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch 2 times, most recently from a8a3a75 to aee6da8 Compare June 5, 2024 05:44

struct sof_eq_iir_biquad *coefficients_block =
rzalloc(SOF_MEM_ZONE_RUNTIME, 0, SOF_MEM_CAPS_RAM,
total_coefficients_size);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now this seems to be allocated twice: first time inside multiband_drc_init() and a second time here.

Copy link
Contributor Author

@ShriramShastry ShriramShastry Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your feedback regarding the memory allocation within the multiband DRC module

The allocation of memory for coefficients_block in multiband_drc_init_coef() is separate from the allocation of multiband_drc_comp_data in multiband_drc_init(). The latter is the component data for the multiband
DRC module, and it is allocated only once during module initialization.

I 've cross checked again.

The multiband_drc_comp_data allocation is for storing the component's state and is done once upon initialization (multiband_drc_init()).
The coefficients_block allocation is specifically for filter coefficients and occurs later during the module's preparation (multiband_drc_init_coef()).
The allocation for coefficients_block is not done in multiband_drc_init() and hence there is no duplication.
It looks like all allocations and initializations are following the correct sequence and avoiding double allocation for the same data structures. If there are any further concerns or points that require clarification, please let me know.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShriramShastry Maybe I didn't explain it well. See line 132 below that you're removing

crossover = config->crossover_coef;
- that's where previously crossover was coming from. Now you added an allocation for it. But I don't see where you removed that original data block? So the original array seems to still be there and you add a new one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks ! I'II make the adjustments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done ! Please have a look.

@ShriramShastry ShriramShastry marked this pull request as ready for review June 5, 2024 14:18
@ShriramShastry ShriramShastry requested a review from a team as a code owner June 5, 2024 14:18
* deemphasis is situated after the emphasis coefficients.
* This ensures all filter coefficients are stored contiguously.
*/
crossover = coefficients_block;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a modification to the datatypes rather than just packing them into a single buffer and crossing fingers no one screws up the math in the future

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review and comments. The changes to the allocation and pointer setup are intended to enhance memory efficiency and cache performance for the MDRC component.

Here’s a summary of the changes and the rationale behind them:

Memory Allocation Consolidation: By allocating a single block of memory for crossover, emphasis, and deemphasis filter coefficients, we avoid separate heap allocations which can incur additional overhead and fragmentation.

size_t total_coefficients_size = sizeof(struct sof_eq_iir_biquad) *
                                 num_bands * nch * 3;

struct sof_eq_iir_biquad *coefficients_block =
    rzalloc(SOF_MEM_ZONE_RUNTIME, 0, SOF_MEM_CAPS_RAM,
            total_coefficients_size);

if (!coefficients_block) {
    comp_err(dev, "multiband_drc_init_coef(), failed to allocate memory for coefficients");
    return -ENOMEM;
}

Pointer Assignment: After the allocation, pointers for crossover, emphasis, and deemphasis filters are pointed to the appropriate locations within the single allocated block.

crossover = coefficients_block;
emphasis = coefficients_block + num_bands * nch;
deemphasis = emphasis + num_bands * nch;

This design assures that the data used concurrently during processing is also kept in memory, which will assist with cache usage when processing audio data.

Error Handling: A clear error handling path (goto err;) ensures that the allocated memory block is freed in the event of an error occurring after the allocation, preventing memory leaks.

if (ret < 0) {
    comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d", ch);
    rfree(coefficients_block);
    goto err;
}

The check-in ensure both accuracy and safety, the pointer arithmetic is based on the size, count, and channel configurations of the coefficients.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the motivation, but the code you have written here (by nature) is much more fragile than if you simply made a toplevel struct which contained everything and just allocated that directly.

Pointer math is hard enough to debug, pointer math on an embedded device is much worse. Lets not add any more than we absolutely should.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that I'm organizing coefficients in contiguous memory, should multiband_drc_init_coef() be designed to handle separate coefficients instantiation for each channel and band?

If our architecture allows for different filters per channel/band, this could mean dynamically revising the processing loop in multiband_drc_init_coef() to fetch and apply the correct filter coefficients for each specific band.

Could you shed some light on whether unique filters per channel/band are an aspect of our system's design? If so, are there any particular considerations or adjustments to the coefficient fetch and allocation logic within multiband_drc_init_coef() that I should be aware of to implement this correctly?

/** 
 *static int multiband_drc_init_coef(struct processing_module *mod, int16_t nch, uint32_t rate)
 *{
 *	struct comp_dev *dev = mod->dev;
 *	struct multiband_drc_comp_data *cd = module_get_private_data(mod);
 *	struct sof_eq_iir_biquad *crossover;
 *	struct sof_eq_iir_biquad *emphasis;
 *	struct sof_eq_iir_biquad *deemphasis;
 *	struct sof_multiband_drc_config *config = cd->config;
 *	struct multiband_drc_state *state = &cd->state;
 *	uint32_t sample_bytes = get_sample_bytes(cd->source_format);
 *	int i, ch, ret, num_bands;
 *
 *	if (!config) {
 *		comp_err(dev, "multiband_drc_init_coef(), no config is set");
 *		return -EINVAL;
 *	}
 *
 *	num_bands = config->num_bands;
 *
 *	// Sanity checks 
 *	if (nch > PLATFORM_MAX_CHANNELS) {
 *		comp_err(dev,
 *			 "multiband_drc_init_coef(), invalid channels count(%i)", nch);
 *		return -EINVAL;
 *	}
 *	if (config->num_bands > SOF_MULTIBAND_DRC_MAX_BANDS) {
 *		comp_err(dev, "multiband_drc_init_coef(), invalid bands count(%i)",
 *			 config->num_bands);
 *		return -EINVAL;
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing %i-way crossover",
 *		  config->num_bands);
 *
 *	// Crossover: collect the coef array and assign it to every channel 
 *	crossover = config->crossover_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = crossover_init_coef_ch(crossover, &state->crossover[ch],
 *					     config->num_bands);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev,
 *				 "multiband_drc_init_coef(), could not assign coeffs to ch %d", ch);
 *			goto err;
 *		}
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing emphasis_eq");
 *
 *	// Emphasis: collect the coef array and assign it to every channel 
 *	emphasis = config->emp_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = multiband_drc_eq_init_coef_ch(emphasis, &state->emphasis[ch]);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d",
 *				 ch);
 *			goto err;
 *		}
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing deemphasis_eq");
 *
 *	// Deemphasis: collect the coef array and assign it to every channel 
 *	deemphasis = config->deemp_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = multiband_drc_eq_init_coef_ch(deemphasis, &state->deemphasis[ch]);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d",
 *				 ch);
 *			goto err;
 *		}
 *	}
 *
 *	// Allocate all DRC pre-delay buffers and set delay time with band number 
 *	for (i = 0; i < num_bands; i++) {
 *		comp_info(dev, "multiband_drc_init_coef(), initializing drc band %d", i);
 *
 *		ret = drc_init_pre_delay_buffers(&state->drc[i], (size_t)sample_bytes, (int)nch);
 *		if (ret < 0) {
 *			comp_err(dev,
 *				 "multiband_drc_init_coef(), could not init pre delay buffers");
 *			goto err;
 *		}
 *
 *		ret = drc_set_pre_delay_time(&state->drc[i],
 *					     cd->config->drc_coef[i].pre_delay_time, rate);
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not set pre delay time");
 *			goto err;
 *		}
 *	}
 *
 *	return 0;
 *
 *err:
 *	multiband_drc_reset_state(state);
 *	return ret;
 *} 
 */

static int multiband_drc_setup(struct processing_module *mod, int16_t channels, uint32_t rate)
{
	struct multiband_drc_comp_data *cd = module_get_private_data(mod);
	int ret;

	/* Reset any previous state */
	multiband_drc_reset_state(&cd->state);

	/* Setup Crossover, Emphasis EQ, Deemphasis EQ, and DRC */
	ret = multiband_drc_init_coef(mod, channels, rate);

}

Currently, the coefficients for each of the emphasis/de-emphasis filters and crossovers per channel are initialised, but it appears that the multiband DRC assumes a configured number of filters applied uniformly across channels; however, if individual adjustments/adapt to filters per channel/band are required, the code should fetch and set up the coefficients accordingly within the multiband_drc_setup and related functions.

Your expertise in this area will help to clarify the best course of action and ensure that our implementation remains efficient and in line with our system's architectural goals.

Copy link
Contributor Author

@ShriramShastry ShriramShastry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let me know if you have any specific concerns or suggestions for further improvements. Your feedback is extremely helpful, and I am willing to make any necessary changes to this patch..

Thank you for review.

* deemphasis is situated after the emphasis coefficients.
* This ensures all filter coefficients are stored contiguously.
*/
crossover = coefficients_block;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review and comments. The changes to the allocation and pointer setup are intended to enhance memory efficiency and cache performance for the MDRC component.

Here’s a summary of the changes and the rationale behind them:

Memory Allocation Consolidation: By allocating a single block of memory for crossover, emphasis, and deemphasis filter coefficients, we avoid separate heap allocations which can incur additional overhead and fragmentation.

size_t total_coefficients_size = sizeof(struct sof_eq_iir_biquad) *
                                 num_bands * nch * 3;

struct sof_eq_iir_biquad *coefficients_block =
    rzalloc(SOF_MEM_ZONE_RUNTIME, 0, SOF_MEM_CAPS_RAM,
            total_coefficients_size);

if (!coefficients_block) {
    comp_err(dev, "multiband_drc_init_coef(), failed to allocate memory for coefficients");
    return -ENOMEM;
}

Pointer Assignment: After the allocation, pointers for crossover, emphasis, and deemphasis filters are pointed to the appropriate locations within the single allocated block.

crossover = coefficients_block;
emphasis = coefficients_block + num_bands * nch;
deemphasis = emphasis + num_bands * nch;

This design assures that the data used concurrently during processing is also kept in memory, which will assist with cache usage when processing audio data.

Error Handling: A clear error handling path (goto err;) ensures that the allocated memory block is freed in the event of an error occurring after the allocation, preventing memory leaks.

if (ret < 0) {
    comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d", ch);
    rfree(coefficients_block);
    goto err;
}

The check-in ensure both accuracy and safety, the pointer arithmetic is based on the size, count, and channel configurations of the coefficients.

Copy link
Contributor Author

@ShriramShastry ShriramShastry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing the code, I 'm in need of your/Google guidance on Coefficient allocation logic for Channel/Band-Specific Filters in multiband_drc_init_coef().

Because the task requires us to properly allocate and assign emphasis/de-emphasis filter coefficients within a shared memory block.

However, I need to know if our architecture requires these filters to be unique per channel/band, which will affect our allocation strategy and coefficient fetch logic.

* deemphasis is situated after the emphasis coefficients.
* This ensures all filter coefficients are stored contiguously.
*/
crossover = coefficients_block;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that I'm organizing coefficients in contiguous memory, should multiband_drc_init_coef() be designed to handle separate coefficients instantiation for each channel and band?

If our architecture allows for different filters per channel/band, this could mean dynamically revising the processing loop in multiband_drc_init_coef() to fetch and apply the correct filter coefficients for each specific band.

Could you shed some light on whether unique filters per channel/band are an aspect of our system's design? If so, are there any particular considerations or adjustments to the coefficient fetch and allocation logic within multiband_drc_init_coef() that I should be aware of to implement this correctly?

/** 
 *static int multiband_drc_init_coef(struct processing_module *mod, int16_t nch, uint32_t rate)
 *{
 *	struct comp_dev *dev = mod->dev;
 *	struct multiband_drc_comp_data *cd = module_get_private_data(mod);
 *	struct sof_eq_iir_biquad *crossover;
 *	struct sof_eq_iir_biquad *emphasis;
 *	struct sof_eq_iir_biquad *deemphasis;
 *	struct sof_multiband_drc_config *config = cd->config;
 *	struct multiband_drc_state *state = &cd->state;
 *	uint32_t sample_bytes = get_sample_bytes(cd->source_format);
 *	int i, ch, ret, num_bands;
 *
 *	if (!config) {
 *		comp_err(dev, "multiband_drc_init_coef(), no config is set");
 *		return -EINVAL;
 *	}
 *
 *	num_bands = config->num_bands;
 *
 *	// Sanity checks 
 *	if (nch > PLATFORM_MAX_CHANNELS) {
 *		comp_err(dev,
 *			 "multiband_drc_init_coef(), invalid channels count(%i)", nch);
 *		return -EINVAL;
 *	}
 *	if (config->num_bands > SOF_MULTIBAND_DRC_MAX_BANDS) {
 *		comp_err(dev, "multiband_drc_init_coef(), invalid bands count(%i)",
 *			 config->num_bands);
 *		return -EINVAL;
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing %i-way crossover",
 *		  config->num_bands);
 *
 *	// Crossover: collect the coef array and assign it to every channel 
 *	crossover = config->crossover_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = crossover_init_coef_ch(crossover, &state->crossover[ch],
 *					     config->num_bands);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev,
 *				 "multiband_drc_init_coef(), could not assign coeffs to ch %d", ch);
 *			goto err;
 *		}
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing emphasis_eq");
 *
 *	// Emphasis: collect the coef array and assign it to every channel 
 *	emphasis = config->emp_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = multiband_drc_eq_init_coef_ch(emphasis, &state->emphasis[ch]);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d",
 *				 ch);
 *			goto err;
 *		}
 *	}
 *
 *	comp_info(dev, "multiband_drc_init_coef(), initializing deemphasis_eq");
 *
 *	// Deemphasis: collect the coef array and assign it to every channel 
 *	deemphasis = config->deemp_coef;
 *	for (ch = 0; ch < nch; ch++) {
 *		ret = multiband_drc_eq_init_coef_ch(deemphasis, &state->deemphasis[ch]);
 *		// Free all previously allocated blocks in case of an error 
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not assign coeffs to ch %d",
 *				 ch);
 *			goto err;
 *		}
 *	}
 *
 *	// Allocate all DRC pre-delay buffers and set delay time with band number 
 *	for (i = 0; i < num_bands; i++) {
 *		comp_info(dev, "multiband_drc_init_coef(), initializing drc band %d", i);
 *
 *		ret = drc_init_pre_delay_buffers(&state->drc[i], (size_t)sample_bytes, (int)nch);
 *		if (ret < 0) {
 *			comp_err(dev,
 *				 "multiband_drc_init_coef(), could not init pre delay buffers");
 *			goto err;
 *		}
 *
 *		ret = drc_set_pre_delay_time(&state->drc[i],
 *					     cd->config->drc_coef[i].pre_delay_time, rate);
 *		if (ret < 0) {
 *			comp_err(dev, "multiband_drc_init_coef(), could not set pre delay time");
 *			goto err;
 *		}
 *	}
 *
 *	return 0;
 *
 *err:
 *	multiband_drc_reset_state(state);
 *	return ret;
 *} 
 */

static int multiband_drc_setup(struct processing_module *mod, int16_t channels, uint32_t rate)
{
	struct multiband_drc_comp_data *cd = module_get_private_data(mod);
	int ret;

	/* Reset any previous state */
	multiband_drc_reset_state(&cd->state);

	/* Setup Crossover, Emphasis EQ, Deemphasis EQ, and DRC */
	ret = multiband_drc_init_coef(mod, channels, rate);

}

Currently, the coefficients for each of the emphasis/de-emphasis filters and crossovers per channel are initialised, but it appears that the multiband DRC assumes a configured number of filters applied uniformly across channels; however, if individual adjustments/adapt to filters per channel/band are required, the code should fetch and set up the coefficients accordingly within the multiband_drc_setup and related functions.

Your expertise in this area will help to clarify the best course of action and ensure that our implementation remains efficient and in line with our system's architectural goals.

@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch 2 times, most recently from 3b6a9cd to b5919ce Compare June 10, 2024 12:36
struct sof_multiband_drc_config *config = cd->config;
struct multiband_drc_state *state = &cd->state;
uint32_t sample_bytes = get_sample_bytes(cd->source_format);
int i, ch, ret, num_bands;
bool alloc_success = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need this, you can just check if coefficients_block == NULL in the error path.

struct sof_eq_iir_biquad crossover[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
struct sof_eq_iir_biquad emphasis[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
struct sof_eq_iir_biquad deemphasis[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
} __packed;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't packing cause a lot of slow reads/writes because things might not be aligned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While check-in I was getting the error if I do __attribute__((packed)); instead of "} __packed; so I drop the attribute.

"

No codespell typos will be found - file '/usr/share/codespell/dictionary.txt': No such file or directory
WARNING: Prefer __packed over __attribute__((packed))
#10: FILE: src/audio/multiband_drc/multiband_drc.h:28:
+} __attribute__((packed));

total: 0 errors, 1 warnings, 8 lines checked

Understood. We're using __packed to save memory, but could you confirm if the potential misalignment outweighs the memory gain in our context? And whether our target platform can handle unaligned accesses efficiently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShriramShastry we mainly using packing for any data shared with host, but in this case isn't this data provate to FW only ? If so, we can drop the packed and allow compiler to best organise the data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShriramShastry please read this

The odds here are you saving at most 10s of bytes and paying back way worse in code size and access times

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, packed isn't needed. It should only be used where data format is important and must be preserved, e.g. when sending data over networks, or between the host and the DSP

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, removed packed

struct multiband_drc_comp_data *cd = rzalloc(SOF_MEM_ZONE_RUNTIME, 0,
SOF_MEM_CAPS_RAM, sizeof(*cd));
if (!cd) {
comp_err(dev, "multiband_drc_init(), allocation for multiband_drc_comp_data failed");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unneeded

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please clarify if the removal of the NULL check after rzalloc is due to guaranteed error handling within the allocator, or are we following a specific coding standard that omits such checks?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the change is unneeded (i.e. the trace), the check is still very much needed


/* Allocation for coefficients_block */
if (!cd->coefficients_block) {
cd->coefficients_block = rballoc(0, SOF_MEM_CAPS_RAM,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just allocate this upfront directly into cd so you don't have to constantly check this.

Copy link
Contributor Author

@ShriramShastry ShriramShastry Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you provide more detail on how to allocate coefficients_block directly with cd upfront, given that we typically need to check and allocate each separately in standard C?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just do it at the module creation step so you don't have to check if the memory exists when you are parsing params

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch from b5919ce to c86a1f7 Compare June 11, 2024 08:14
Copy link
Contributor Author

@ShriramShastry ShriramShastry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocation of coefficients_block is currently conditional on whether it is NULL. Would you prefer that this allocation be performed unconditionally at the time of cd allocation, ensuring that coefficients_block is never NULL and eliminating all subsequent NULL checks on it?

@lgirdwood
Copy link
Member

The allocation of coefficients_block is currently conditional on whether it is NULL. Would you prefer that this allocation be performed unconditionally at the time of cd allocation, ensuring that coefficients_block is never NULL and eliminating all subsequent NULL checks on it?

I would allocate it when needed if it relies on IPC configuration data for allocation size, if its always constant size you could allocate per instance when pipeline is created.

@cujomalainey
Copy link
Member

The allocation of coefficients_block is currently conditional on whether it is NULL. Would you prefer that this allocation be performed unconditionally at the time of cd allocation, ensuring that coefficients_block is never NULL and eliminating all subsequent NULL checks on it?

I would allocate it when needed if it relies on IPC configuration data for allocation size, if its always constant size you could allocate per instance when pipeline is created.

+1, I think the only thing that would possibly vary is the crossover bands and that is fairly small

@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch from c86a1f7 to f770cd2 Compare June 14, 2024 06:33
struct multiband_drc_comp_data *cd = rzalloc(SOF_MEM_ZONE_RUNTIME, 0,
SOF_MEM_CAPS_RAM, sizeof(*cd));
if (!cd) {
comp_err(dev, "multiband_drc_init(), allocation for multiband_drc_comp_data failed");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the change is unneeded (i.e. the trace), the check is still very much needed

struct sof_eq_iir_biquad crossover[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
struct sof_eq_iir_biquad emphasis[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
struct sof_eq_iir_biquad deemphasis[SOF_MULTIBAND_DRC_MAX_BANDS * PLATFORM_MAX_CHANNELS];
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the more I read this code the more I think this change is incorrect. Look at the struct directly below. This is a copy of that struct which is already directly embedded in the component data. How does this benefit anything when the data was already allocated in place in a single block?

Copy link
Contributor Author

@ShriramShastry ShriramShastry Jun 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks ! Please take a look at the latest changes. The patch saves 54 MCPS in TGL i.e. from 188 to 134

@singalsu
Copy link
Collaborator

singalsu commented Jun 18, 2024

This PR version has an issue with TGL HiFi3 build. If I set with topology sof-hda-benchmark-drc_multiband32.tplg the control amixer cset name='Analog Playback MULTIBAND_DRC enable' on the playback becomes silent. Sound can be heard with setting amixer cset name='Analog Playback MULTIBAND_DRC enable' off. The current DRC versin does not follow the switch control in runtime, the control needs to be applied when streaming is stopped (or stop the stream to get impact of previously applied control).

I have no idea if MTL HiFi4 has the issue, I don't have a suitable device to check own FW builds.

@cujomalainey
Copy link
Member

This PR version has an issue with TGL HiFi3 build. If I set with topology sof-hda-benchmark-drc_multiband32.tplg the control amixer cset name='Analog Playback MULTIBAND_DRC enable' on the playback becomes silent. Sound can be heard with setting amixer cset name='Analog Playback MULTIBAND_DRC enable' off. The current DRC versin does not follow the switch control in runtime, the control needs to be applied when streaming is stopped (or stop the stream to get impact of previously applied control).

I have no idea if MTL HiFi4 has the issue, I don't have a suitable device to check own FW builds.

@singalsu no that is by design for our code that the on/off switch is applied while the stream is open.

@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch from f770cd2 to 9ce82c1 Compare June 24, 2024 13:08
int i, ch, ret, num_bands;
struct sof_eq_iir_biquad *crossover, *emphasis, *deemphasis;
int i, ch, ret;
int num_bands = config ? config->num_bands : 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to check for config twice. The original code, checking config != NULL in line 137 below and then assigning num_bands unconditionally was fine. This change isn't an improvement IMHO.

@@ -21,6 +21,7 @@
/**
* Stores the state of the sub-components in Multiband DRC
*/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid drive-by changes

* phase. This allows efficient access to filter parameters required for audio
* processing across all channels and bands.
*/
struct multiband_drc_coefficients *coefficients_block; /**< Filter coefficients */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this adds a member of a structure called coefficients_block and I don't see it being used anywhere. And this is a 1-commit PR. This doesn't seem right.

@@ -28,6 +28,7 @@
#include <sof/ut.h>
#include <user/eq.h>
#include <user/trace.h>
#include <stdbool.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this commit adding any uses of this header, or was it missing before?
To the commit text: sorry, I don't see in the code what is described in the commit text

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was a previous version, this needs to be removed

Remove redundant struct multiband_drc_coefficients and simplify memory
management by handling memory allocation directly.

1. Removed `struct multiband_drc_coefficients` from multiband_drc.h and
   multiband_drc.c.
2. Updated `multiband_drc_init` function to eliminate the allocation of
   the obsolete `coefficients_block`.
3. Adjusted `multiband_drc_free` to no longer check and free the
   now-nonexistent `coefficients_block`.
4. Directly handled memory allocation and initialization of crossover,
   emphasis, and de-emphasis coefficients in `multiband_drc_init_coef`.
5. Simplified overall memory management, removing unnecessary
   layers of indirection.

This change reduces memory allocation overhead, improves data locality
to enhance cache efficiency, and mitigates heap fragmentation. The
refactoring of memory management adds robustness to the component by
reducing the chances of memory leaks and simplifying the initialization
and cleanup process.

By streamlining memory management, the update lowers the possibility
of memory leaks.

Ensured thorough testing to verify no functionality is broken or new
issues introduced.

Signed-off-by: Shriram Shastry <[email protected]>
@ShriramShastry ShriramShastry force-pushed the improve_multiband_drc_memory_optimization branch from 9ce82c1 to 65260b2 Compare June 27, 2024 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants