I am hoping to generate audio/video stream with gstreamer and send the stream to a jitsi meet like a user/browser would. so a gstreamer sink.
and also I would like to connect to the server and ingest the stream as a gstreamer source.
This is to add some automation around using jitsi for conference presentations.
Currently a collection of humans work together and figure it out, but we would like to automate whatever we can, and replace things like [browser, vlc, vnc, human hits Play button at the right time] with a script that runs at the right time and doesn’t require the amount of config and apt to app connections and permissions that are needed for jitsi in browser to read from what vlc has rendered to it’s window.
and: I want a count down HH:MM:SS clock so we all know when the next presentation starts, using a clock we are all in sync with. currently a person in the meeting is in charge of talking this information, and sometimes they don’t for all the reasons humans don’t do what they should.
We also want to send a single composited stream to a CDN with a standard html 5 player embeded in a web page, or people can use whatever media player they want.
searching around, I found:
“I finally make simple gstreamer webrtcbin video/audio connecting to jicofo/jvb works.”
But 2 years later: “Can you share the code” [jitsi-dev] rtcp-mux without bundle
I am also interested working in having a gstreamer sink that could be configured to connect to a Jitsi Meeting. I really don’t get why this feature seems to get ignored, several people have expressed interest in such a use case, but the developers seem to respond with either ignorance or by suggesting subpar options like screen-sharing or non-portable work-arounds using loopback devices.
Anyway, if I can find another developer to collaborate with, I would be willing to throw a couple days at this to see if it is possible to get something basic working. @CarlFK do you have any experience with Gstreamer/coding skills?
So my interest in this has just dropped a bit, but I’ll be happy to test things if someone else does the real work.
Record a .webm file for each of the other participants in the conference, containing VP8 video and Opus audio, without needing to do any transcoding. A modest system can record a very large number of participants this way, since no transcoding is required.
Passing through colibri events to allow the pipeline to respond to changes in who is speaking etc
Building video mixing elements for stage view and tile view, and implement Jibri-style recording in terms of those — it should be much more efficient than Jibri since no browser + screen capture is involved
Adding RTX and TCC support
We are really keen to hear any suggestions for other ways this framework could be used. PRs and issues (both feature requests and bugs) are most welcomed!
A feature that might be interesting to me would be to be able to limit reception to 1 participant (given their endpoint ID). This would involve setting last N to 1, and selecting that endpoint. In addition, selecting the receive quality would be nice, to choose in what (video) quality one would like to receive that participant.
I have been meaning to learn some Rust, this may be the right time to get my feet wet! No promises though, I need to find the time first!
Hi @Damien_FETIS. For the XMPP connection, we’ve only implemented websockets, so if your server only supports the older BOSH connection, it won’t work. If you’re using websockets for XMPP and it still doesn’t work properly, please file an issue on GitHub and include the output from running gst-meet with --verbose (added in 0.2.1).
What is your recommended JItsi version to test it?
Any recent version of Jicofo, Prosody & JVB should be fine, but there are a lot of variables with a Jitsi deployment and we’ve only tested with our own platform and a handful of other deployments, so undoubtedly there are incompatibilities yet to be found!
We’ve been busy on other things for a few months, but now that GStreamer 1.20 has been released, I spent some more time on this today.
TWCC feedback is now working, so the received video doesn’t freeze any more, and the JVB steps up to forwarding higher layers if bandwidth is sufficient. For recording received streams it’s now quite usable.
I’ve also implemented XMPP pings from the client to the server, which should help keep the connection active when there is a NAT/firewall/LB in the path with a short timeout.
On the debugging side, there’s now the ability to log information about every RTP and RTCP packet (which was very helpful with getting TWCC working).