Skip to content

buffer: prevent cache corruption - #7808

Merged
kv2019i merged 1 commit into
thesofproject:mainfrom
lyakh:buffer
Jun 30, 2023
Merged

kv2019i merged 1 commit into
thesofproject:mainfrom
lyakh:buffer

Conversation

@lyakh

@lyakh lyakh commented Jun 15, 2023

Copy link
Copy Markdown
Collaborator

Recent research discovered a set of potential issues related with cache prefetch. Specifically it seems like uncached access to memory can cause cache prefetch. This can cause problems in buffer_attach() and buffer_detach() where buffers are added to or removed from lists respectively via uncached addresses, after which they can be used via cached addresses. Add proper cache synchronisation and interrupt locking to protect against such memory corruption.

Recent research discovered a set of potential issues related with
cache prefetch. Specifically it seems like uncached access to memory
can cause cache prefetch. This can cause problems in buffer_attach()
and buffer_detach() where buffers are added to or removed from lists
respectively via uncached addresses, after which they can be used via
cached addresses. Add proper cache synchronisation and interrupt
locking to protect against such memory corruption.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
@btian1

btian1 commented Jun 16, 2023

Copy link
Copy Markdown
Contributor

"Recent research discovered a set of potential issues related with cache prefetch. Specifically it seems like uncached access to memory can cause cache prefetch. "

is this reasonable? if access memory in uncached, then dsp should perform cache prefetch, right?

@lyakh

lyakh commented Jun 16, 2023

Copy link
Copy Markdown
Collaborator Author

is this reasonable? if access memory in uncached, then dsp should perform cache prefetch, right?

@btian1 #7191 (comment)

@btian1

btian1 commented Jun 19, 2023

Copy link
Copy Markdown
Contributor

I would suggest wait for some time for #7191 , if still reproducible, we can ask keqiao help for reproduce with this PR.

Comment thread src/audio/buffer.c
goto free;
}

irq_local_disable(flags);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when turn off irq, means some or one of irq have negative impact with buffer_attach.
this part of change can make buffer_attach more secure, just not sure it is must or not.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is needed because we're processing an IPC here and we might get interrupted by the scheduler, already running a pipeline to which we're attaching an additional source or sink.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does feel out-of-place in module_adapter.c level code. Given both IPC and pipeline are run in regular threads, we should be able to use plain mutex'es for exclusive access here. This would make it also userspace compatible for future-proofing this code.

But maybe beyond this PR, especially given pipeline_connect() already does the IRQ enable/disable dance, so the same should be done here to be safe.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does feel out-of-place in module_adapter.c level code. Given both IPC and pipeline are run in regular threads, we should be able to use plain mutex'es for exclusive access here. This would make it also userspace compatible for future-proofing this code.

@kv2019i maybe mutexes would work too, but would take context switching:

  1. binding of an additional pipeline is taking place
  2. a timer interrupt arrives
  3. the copying is scheduled
  4. since the LL scheduler has a higher priority than the IPC thread we switch to copying
  5. we try to take the mutex but it's held by the IPC thread
  6. the copying thread is suspended and the IPC thread is scheduled (priority inversion)
  7. the IPC thread finishes binding and releases the mutex
  8. we switch back to the copying thread

So, this should work, but would involve a bunch of switching. If the protected area is small enough, locking interrupts should be simple enough. But if we shouldn't use priority calls, then we can use a mutex

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lyakh Ack. Let's revisit this when we introduce user-space, this is anyways not the only place where we have explicit irq control in the SOF audio code.

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, took a long time to review, but this is a complex piece of code to review. I'm leaning towards approving as this does problems in current codebase. See more comments inline.

Comment thread src/audio/buffer.c
goto free;
}

irq_local_disable(flags);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does feel out-of-place in module_adapter.c level code. Given both IPC and pipeline are run in regular threads, we should be able to use plain mutex'es for exclusive access here. This would make it also userspace compatible for future-proofing this code.

But maybe beyond this PR, especially given pipeline_connect() already does the IRQ enable/disable dance, so the same should be done here to be safe.

@kv2019i

kv2019i commented Jun 30, 2023

Copy link
Copy Markdown
Collaborator

@kv2019i
kv2019i merged commit 3e3d0cd into thesofproject:main Jun 30, 2023
@lyakh
lyakh deleted the buffer branch June 30, 2023 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants