To whoever's interested, I've fixed an issue in the bouncy castle library that breaks the DTLS negotiation but it's not yet accepted from upstream (https://github.com/bcgit/bc-java/pull/111). We need this fix in the JVB.
They just released v1.52 so I doubt this will get released anytime soon. I've setup a repository to hold the patched library here https://github.com/gpolitis/bc-java/tree/r1rv51-patched. Unless you think this is a bad idea, I'll update the JVB later today with a home baked version of the bouncycastle.jar.