[GSoc] Spatial Audio Discussion

Developers
#1

I have been looking at Jitsi Meet’s architecture and codebase to help me draft a proposal for the spatial audio project and figure out a plan of action. I had some questions and would love it if any of the mentors for the project can provide some guidance in that respect.

  1. Since audio spatialisation will be different for different participants, I’m thinking we can receive all the audio streams and spatialise them at the client side. Does every participant receive individual streams or is it one master track of some kind?
  2. Does the project involve implementing spatial audio only for Jitsi Meet web or the native apps as well?
#2

Each participant is 1 audio track.

On web for now, since we might not have the necessary apps yet.

#3

Got it. Thank you!

#4

I was thinking about the flow for turning on spatial audio in a video call. Two ways cropped up:

  1. Initialising the conference with spatial audio turned on/off.
  2. Turning on/off spatial audio after the conference is initialised.

Audio/video calls using Dolby.io’s APIs utilise option no. 1.
Option no. 2 allows for turning spatial audio on/off as the call is going on.

Can you please suggest which option would be better suited for Jitsi Meet? I would love to hear if you had something else in mind as well.

#5

I think it will need to be togglable by the user. Depending on the layout they chose it may or may not make sense.

If you are watching 12 users in tile view, spatial audio would distribute the audio based on the tile location, but that doesn’t make sense for stage view (single speaker in the large view).

#6

Got it. I was just going to ask about what should happen in case the view gets changed… Thank you.

#7

Hello!
I finished drafting my proposal and have email it to gsoc@jitsi.org. I would appreciate it if you could take a look at it and share your thoughts when you have some time.

#8

Hello @saghul
Sorry for tagging you, but can you confirm whether the draft was received successfully? I can send it here as well if that’s more convenient.

#9

Yep, we got it.

1 Like