Call not ending immediately or delayed after hitting red "end call" icon

With my setup of Jitsi, after I hit end call (red icon), it takes 15 to 20 seconds for the call to actually end and go to homepage.
Why is this so? Is there a configuration that helps change this?

On the other hand, if I start a call at and end it, it immediately closes the call and takes back to home page without delays.

@damencho are there any options that control this?