Installation problems on Ubuntu 22.04 Jammy Jellyfish

Hello,

Ubuntu 22.04 is now less than one month away, and it looks not promising. It looks in fact that the best option may be to update the handbook to preconise Debian only.

There are 2 problems, both related to the Ubuntu tendency to update to shiny new things without doing the work of updating stuff to adapt to these shiny new things.

Problems

  1. Java 18 breaks Jitsi-videobridge2 package: there is a circular dependency in Ubuntu (and Debian of course) between Java and ca-certificates-java. It seems that it is making the order of installation between these 2 packages not deterministic. Usually Java gets installed first (no idea why), but some combination of circonstances can make that ca-certificates-java is installed before Java is fully configured. That’s why (I guess) there is a strange path setting loop in the ca-certificates-jave postinst to find the appropriage Java to run.
    However, these packages are managed by Debian teams and Debian has not upgraded to Java 18 even on the unstable version (Sid/Bookworm). And installing jitsi-videobridge2 is one of these unhappy circonstnaces that are causing this installation inversion.
    So installing jitsi-videobridge2 fails:
head: cannot open '/etc/ssl/certs/java/cacerts' for reading: No such file or directory
/var/lib/dpkg/info/ca-certificates-java.postinst: line 101: java: command not found
dpkg: error processing package ca-certificates-java (--configure):
 installed ca-certificates-java package post-installation script subprocess returned error exit status 127
dpkg: dependency problems prevent configuration of openjdk-18-jre-headless:amd64:
 openjdk-18-jre-headless:amd64 depends on ca-certificates-java (>= 20190405~); however:
  Package ca-certificates-java is not configured yet.

dpkg: error processing package openjdk-18-jre-headless:amd64 (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 ca-certificates-java
 openjdk-18-jre-headless:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)

I have filed this Ubuntu bug.

  1. Openssl 3.01: Coturn can’t compile out of the box under Openssl 3 and as such the package has been dropped. Jitsi-meet installation can go on without Coturn, but of course Coturn functionality is missing, and as an added fun, Nginx AND Apache are both installed. This lack of Coturn is something Jitsi-meet packages have not envisioned it seems.
    Same things as Java, Debian is managing this package and has not upgraded OpenSSL in latest unstable, so no problem for Debian.
    I have filed this Ubuntu bug.

Workarounds

  1. Java 18: explicitly install a previous Java version
    sudo apt install default-jre jitsi-meet
    This is peculiar because the jitsi-meet is asking to install java8 OR java11 and the result is tha the latest Java is installed (Java 17 for Ubuntu 20.04 for instance).
    But asking for default-jre will install Java 11. Just the one Jitsi-meet love to install and advise.
    In fact I’d say that if Jitsi-meet devs love old things (when it’s Java, for other systems it’s the opposite), changing the package dependency to default-jre could be the best option.
    If people want newer Java they can install it explicitly.

  2. coturn: there is no workaround, other than asking to not install it.
    So the final Ubuntu 22.04 installation magic command is:
    sudo apt install default-jre jitsi-meet --no-install-recommends
    this does not install Coturn of course, but at least it does not install Nginx AND Apache2.
    It’s possible to rebuild a Coturn package from the latest source (21.10) by patching the package of course (the fix is in the Ubuntu bug I filed), but it’s more involved and much worse, there is the problem of security vulnerabilities to follow up. Coturn is a package written in a not memory safe language, very complex with tons of options, and a such has already suffered quite a few security vulnerabilities. This is not a great option for a small organization.

1 Like

There is more work to be done than just bumping the version. It is not that we love old, it is time and resources. To bump a java version you need long loading test and monitoring with many participants, monitoring the app, tuning java garbage collection and the app … Currently if you use and the supported java 11 and change the garbage collection algorithm you will experience frequent freezes and bad quality all the time.

Any recommendations for good turnserver alternative are welcome? We were not having problems with coturn in small and big deployment environments, so I don’t see the point of changing it.

Any problems the package has, will be fixed.

You better stick to LTS versions of Ubuntu if you are looking for stability, my two cents.

We had addressed specific Debian and Ubuntu versions in the past in the handbook. So any warning or purposed workarounds are welcome there.
I don’t see the point of saying Ubuntu is not supported due to problems with a particular version, which is not even LTS.

I’m not looking for stability, I’m trying to foresee the problems that will come in less than one month, when 22.04 will be the current LTS version.

Uuu, thank you. I haven’t check that it will be LTS… Sorry.

Well in that case I will work on updating the handbook.
Thanks for doing that and filing the ubuntu bug report.

WDYT about changing the Java requirements for jitsi-videobridge from

Pre-Depends: java8-runtime-headless | java8-runtime | java11-runtime-headless | java11-runtime

to

Pre-Depends: default-jre

it would install java 11 as is the official advice, and not the last version as it happens currently.

(about Java you have to understand I am sometimes indulging in light teasing, I have felt some discrepancy between the careful testing for Java and the jumping right into the next Prosody version before, well, testing it in depth).

I don’t understand why java8-runtime-headless | java8-runtime | java11-runtime-headless | java11-runtime doesn’t insta;; java 11, but default-jre will install it?
Did they change the name of the package?

Well :slight_smile: seems I will be testing this today :slight_smile: Thank you

This should fix the java problem.

I still don’t get it, after the changes it still insists on openjdk-18-jre-headless …
Checking dependencies with apt-rdepends shows correctly java11 :frowning:

This seems to work though

I have tried to peek into the Apt doc but it would need some hours to get it. My current tentative interpretation based on behaviour and vague intuition about the field names provided by apt-cache is that you are asking for Java 11 but Java 11 is REPLACED by more recent Java versions. And Apt is picking the more recent based on the Version field. As the first 4 packages listed are virtual, it may mean that Debian packaging system is picking freely from the virtual packages first, and falling back to the non- packages you added if it can’t satisfy using one of the virtual ones. Very hypothetical but it would make sense.
All I can say is that when I have tried to install with sudo apt install default-jre jitsi-videobridge2 it has picked the Java 11 version.

Yeah, I don’t know either. I looked for debugging this, to get more information on why it chooses 18 … where default is 11 … my guess based on this:

# apt depends jicofo
jicofo
 |Depends: debconf (>= 0.5)
  Depends: <debconf-2.0>
    cdebconf
    debconf
 |Depends: openjdk-11-jre-headless
 |Depends: openjdk-11-jre
 |Depends: <java11-runtime-headless>
    default-jre-headless
    openjdk-11-jre-headless
    openjdk-17-jre-headless
    openjdk-18-jre-headless
  Depends: <java11-runtime>
    default-jre
    openjdk-11-jre
    openjdk-17-jre
    openjdk-18-jre
  Depends: ruby-hocon
  Depends: jq

Is the first one does not exist:

# apt install java11-runtime-headless
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package java11-runtime-headless is a virtual package provided by:
  openjdk-18-jre-headless 18~36ea-1
  openjdk-17-jre-headless 17.0.2+8-1
  openjdk-11-jre-headless 11.0.14+9-0ubuntu2
  default-jre-headless 2:1.11-72build1
You should explicitly select one to install.

E: Package 'java11-runtime-headless' has no installation candidate

And it chooses the first one from the list …
The strange thing here is openjdk-18 and openjdk-17 provides package with name java11-runtime-headless, I would say that is a bug …
But yeah, moving the existing packages first fixes the issue in my testing and I think is not causing issues on older Ubuntu/Debian versions … once the packages are in unstable, I will test it on 20.04.

In control file of openjdk-18-jre-headless :

Provides: java-runtime-headless, java10-runtime-headless, java11-runtime-headless, java12-runtime-headless, java13-runtime-headless, java14-runtime-headless, java15-runtime-headless, java16-runtime-headless, java17-runtime-headless, java2-runtime-headless, java5-runtime-headless, java6-runtime-headless, java7-runtime-headless, java8-runtime-headless, java9-runtime-headless

And the first non-existing choice is provided by this package and that’s why it offers it to me … without checking the rest … which is weird though …

I think that is the answer

Now why it prefers apache than nginx :man_facepalming:

That’s the other problem, coturn I think. The package config asks for jitsi-meet-turnserver OR apache2, since coturn don’t exist in 22.04, it can’t install jitsi-meet-turnserver hence it installs apache2 (AND nginx…)
Now what I don’t understand is this: jitsi-meet-turnserver OR apache2 ? why???

Package: jitsi-meet
Architecture: all
Version: 2.0.7129-1
(...)
Recommends: jitsi-meet-turnserver (= 1.0.6020-1) | apache2  <==== ????

Yeah, I was about to test to add | nginx before apache.

This is a thing from the past that for coturn we were using ALPN and nginx config, but that became problematic because of some mobile stuff and turns implementation in chrom is still missing correct ALPN so it wasn’t working correctly and we dropped it …
That alpn thingy is dependent on a nginx minimum version and that’s why it was if we can use nginx or drop to apache …

Maybe I will just get rid of the | apache.

@gpatel-fr I think latest from unstable is fine. It uses java 11 and nginx on a new install. Probably will push it to stable next week.
Thank you for the help!

Tested the new version on Ubuntu 20.04, 22.04, Debian 11 and 12 and all install fine. There is no Coturn in 22.04 of course.

2 Likes