Scaling JVBs behind NAT (on Kubernetes)

Hello,

Lets create the following scenario:
3 videobridges (jvb01-03) in a kubernetes cluster, they only have internal IPs and all listen on port 10000. Jicofo, prosody and the Webinterface are on different containers but all in the same subnet.
They all NAT over the same public IP, lets just assume its 1.2.3.4

Jicofo would load balance new conferences to the least used JVB. Now I got assigned to JVB01.
My browser attempts to send UDP data to 1.2.3.4 port 10000, correct? That wouldn’t really work as there are three bridges listening.

Is there any way at all to do load balancing if all JVBs share the same public IP?

Would it be possible to do the jvb “loadbalancing” via kubernetes “NodePorts”?
These would round-robin all traffic to the underlaying JVBs, each packet goes to a different JVB. if the JVBs are connected via octo, would that still work?

Or is there just no way around having a public IP dedicated to each JVB.

2 Likes

Actually, to go more in depth, the kubernetes approach with horizontal scaling, in this case, would be to assign a node port to a service and use a selector for the underlaying jvb pod.
Now all worker nodes have, lets say port 32000, being viped or proxyed to the jvb pod.

Works fine as long as there is only one jvb pod available for that selector.

As soon as you scale up the jvb pod to more than one replica things break because kubernetes service type nodeport balances round robin to multiple identical jvb instances.

One would need to make the replicas register as individual instances, with individual nicknames. Still it would probably not work because the jvb instances still share the same external ips (node ip) and the same port (nodeport)

All members of a conference need to be on the same jvb instance, so round robin Balancing to jvb simply wont work since these instances dont know each other, you cant let a loadbalancer decide where to put the udp datagrams since jvb is not a “cluster”. Right.?

2 Likes