My goal is to stream a canvas content using the Jitsi video stream.
I am using JitsiMeetExternalAPI to setup the video session.
An option would be to create a new command and call the conference.replaceTrack() but I cannot find a way to expose the instance of API.js to my code, so I can take the content of const stream = canvas.captureStream(); and assign to the existing video stream
Ways I tried to access the APP
window.APP - not available before the video session, not available after the video session starts
via _transport - transport has no idea about the subscribers, so not a good idea, but revealed a way to get inner elements from JitsiMeetExternalAPI, using JitsiMeetExternalAPI.prototype.getInternalAPI = function () { return this._transport }
via JitsiMeetJS.app - only has setExternalTransportBackend method, is an instance of JitsiMeetExternalAPI, nothing to see here…
What about exposing the instance of API.js to MeetExternalAPI? I know it would violate the current design, which is oriented by commands, but that would allow lots of custom behaviors to be built on top of MeetExternalAPI.
Please elaborate, as far as I can see in the documentation if I have an instance of JitsiConference, I can manipulate the stream or even replace it with another stream, isn’t how background blur works?
That is true, no access to the JitsiConference from outside, what about add a command “registerVideoStreamFilter” where a pipeline to process the stream before is sent is defined, to be used with Window.postMessage() - Web APIs | MDN
input event name - the event where the parent will receive the content of the stream
output event name - the event where the Iframe will receive an updated version of the stream to finally send it
That would enable other extensions of the video, all sort of effects or updates to the video stream without recreating the existing video UI, that is very cool btw
I’m not sure that would be enough since a pipeline would likely be a function, which cannot be serialized over postMessage. There is some movement in the screen-sharing area at the w3c. Maybe some new APIs will help with this.
I was able to capture the canvas from other Iframe using windows events, will try to send the data thru the text stream instead of the video as it is available.
