[jitsi-dev] Reproducible build (with git submodules)


#1

Hi,

I'm trying to make jitsi (jvb & jicofo) reproducible, ie build all
"SNAPSHOT" packages from sources
I'm using git submodules and a master pom file and i'm now able to build
almost every jitsi package from sources, with no changes in the repos or pom

I've 4 PR pending:
https://github.com/jitsi/jain-sip/pull/1
https://github.com/jitsi/jitsi-universe/pull/3
https://github.com/jitsi/smack_3_2_2/pull/1
https://github.com/jitsi/jicoco/pull/2

And i'm also looking for the source of 3 artifacts (callstats, fmj, jnsapi):
https://github.com/jitsi/jitsi-maven-repository/issues/3

to try (see my PR when it fails to compile):
git clone --recursive https://github.com/champtar/jitsi-submodules
cd jitsi-submodules
rm -rf ~/.m2
mvn install -DskipTests -Dmaven.javadoc.skip=true -Dgpg.skip=true
-Dassembly.skipAssembly=false

The idea for this repo is to be auto updated and tagged (with jenkins?)
when one of the submodules is updated,
so we can reproduce jvb version 532 if we want

Etienne


#2

Hi,

I'm trying to make jitsi (jvb & jicofo) reproducible, ie build all
"SNAPSHOT" packages from sources
I'm using git submodules and a master pom file and i'm now able to build
almost every jitsi package from sources, with no changes in the repos or pom

I've 4 PR pending:
https://github.com/jitsi/jain-sip/pull/1
https://github.com/jitsi/jitsi-universe/pull/3
https://github.com/jitsi/smack_3_2_2/pull/1
https://github.com/jitsi/jicoco/pull/2

And i'm also looking for the source of 3 artifacts (callstats, fmj, jnsapi):

Callstats: https://github.com/callstats-io/callstats.java
FMJ: http://sourceforge.net/p/fmj/code/HEAD/tree/

I don't know about jnsapi.

https://github.com/jitsi/jitsi-maven-repository/issues/3

to try (see my PR when it fails to compile):
git clone --recursive https://github.com/champtar/jitsi-submodules
cd jitsi-submodules
rm -rf ~/.m2
mvn install -DskipTests -Dmaven.javadoc.skip=true -Dgpg.skip=true
-Dassembly.skipAssembly=false

The idea for this repo is to be auto updated and tagged (with jenkins?)
when one of the submodules is updated,
so we can reproduce jvb version 532 if we want

This is very interesting! Thank you for working on it and sharing your efforts!

A related update from our side:
We had a discussion with Lyubomir and other people from the team, and we agree that reproducible builds are something we want to have (again).

The idea we have for fixing the problems with the current maven based builds, as far as I understand it, is to pin the dependency versions in pom.xml and have an automated way of bumping them. One problem we was with this was the number of useless "version bump" commits.

We have not considered using git submodules so far (at least I don't know that we have), but they seem to solve alot of problems in a simple way.

Regards,
Boris

···

On 28/12/15 20:21, Etienne Champetier wrote:


#3

Hi,

Hi,

I'm trying to make jitsi (jvb & jicofo) reproducible, ie build all
"SNAPSHOT" packages from sources
I'm using git submodules and a master pom file and i'm now able to build
almost every jitsi package from sources, with no changes in the repos or
pom

I've 4 PR pending:
https://github.com/jitsi/jain-sip/pull/1
https://github.com/jitsi/jitsi-universe/pull/3
https://github.com/jitsi/smack_3_2_2/pull/1
https://github.com/jitsi/jicoco/pull/2

And i'm also looking for the source of 3 artifacts (callstats, fmj,
jnsapi):

Callstats: https://github.com/callstats-io/callstats.java

are you sure jitsi jenkins pull from this url ? (i will add it)

FMJ: http://sourceforge.net/p/fmj/code/HEAD/tree/

I need a git repo for git submodules :frowning:
can someone setup a svn mirror on github.com/jitsi/?

I don't know about jnsapi.

lyubomir must know (last commit on jitsi-maven-repo for jnsapi)
https://github.com/jitsi/jitsi-maven-repository/commit/9fef8a6f276ca7324e9427ab418fc6409a0981b6

https://github.com/jitsi/jitsi-maven-repository/issues/3

to try (see my PR when it fails to compile):
git clone --recursive https://github.com/champtar/jitsi-submodules
cd jitsi-submodules
rm -rf ~/.m2
mvn install -DskipTests -Dmaven.javadoc.skip=true -Dgpg.skip=true
-Dassembly.skipAssembly=false

The idea for this repo is to be auto updated and tagged (with jenkins?)
when one of the submodules is updated,
so we can reproduce jvb version 532 if we want

This is very interesting! Thank you for working on it and sharing your
efforts!

A related update from our side:
We had a discussion with Lyubomir and other people from the team, and we
agree that reproducible builds are something we want to have (again).

good choice :slight_smile:

The idea we have for fixing the problems with the current maven based
builds, as far as I understand it, is to pin the dependency versions in
pom.xml and have an automated way of bumping them. One problem we was with
this was the number of useless "version bump" commits.

We have not considered using git submodules so far (at least I don't know
that we have), but they seem to solve alot of problems in a simple way.

I new to maven, and don't know all it capabilities / weeknesses yet.

Some ideas:
-artifact should have a strict version, or should be a git submodule

-all submodules should be available on github.com/jitsi/ (so we are sure we
have all sources, they won't go away, ...)

-the "master" repo should be auto updated when a submodules is updated,
tagged (daily with the date?), and releases should use "master" repo tag
(what was the version of all dependencies of jvb version 532 again?)

-if possible release only what changed (don't know if maven can help here),
else release everything (jvb, jicofo, ...)

-release binaries/packages should really include release version into them
and not 0.1-SNAPSHOT
(in some of my apps first log is "starting app version xyz")

-maybe we can replace all version tag with a variable sourced from
jitsi-universe, so we only have one repo with all version bumps

I will restart my work on monday,
In the meantime happy new year
Etienne

···

2015-12-29 16:46 GMT+01:00 Boris Grozev <boris@jitsi.org>:

On 28/12/15 20:21, Etienne Champetier wrote:

Regards,
Boris