[jitsi-dev] Compression for jitsi bundle jars


#1

Hey,

I just noticed that we generate the jars for the Jitsi bundles without compression. I see how that makes sense in Jitsi where they are not stored in the repository, and are distributed in a package which is itself compressed. However, now that we're using them in other projects (videobridge, jigasi, jicofo) where they are stored as blobs I think it would be good to compress them (libjitsi.jar is now 6.6MB, while 3.4MB compressed).

What do you think? Am I missing something here (I assume the load time different for the jvm is negligible), or should I open an issue about this?

Boris


#2

I just noticed that we generate the jars for the Jitsi bundles without
compression. I see how that makes sense in Jitsi where they are not
stored in the repository, and are distributed in a package which is
itself compressed. However, now that we're using them in other projects
(videobridge, jigasi, jicofo) where they are stored as blobs I think it
would be good to compress them (libjitsi.jar is now 6.6MB, while 3.4MB
compressed).

What do you think? Am I missing something here (I assume the load time
different for the jvm is negligible), or should I open an issue about this?

a) As Git compresses binaries and only stores deltas, it shouldn't make a difference if we compress the jars before committing or not. It's possibly even backfiring if we'd do so because the delta between two compressed binaries is probably bigger than between two uncompressed ones.

b) We should make more libraries and reference the projects instead of copying blobs around projects. For a start, this could be done with Git submodules [1] and importing Jitsi's Ant script into Videobridge, Jicofo, etc. A proper modularization into small, distinct libraries with whatever build system could come later.

Boris

Ingo

[1] http://git-scm.com/docs/git-submodule


#3

In line ...

I just noticed that we generate the jars for the Jitsi bundles without
compression. I see how that makes sense in Jitsi where they are not
stored in the repository, and are distributed in a package which is
itself compressed. However, now that we're using them in other projects
(videobridge, jigasi, jicofo) where they are stored as blobs I think it
would be good to compress them (libjitsi.jar is now 6.6MB, while 3.4MB
compressed).

What do you think? Am I missing something here (I assume the load time
different for the jvm is negligible), or should I open an issue about this?

a) As Git compresses binaries and only stores deltas, it shouldn't make a difference if we compress the jars before committing or not. It's possibly even backfiring if we'd do so because the delta between two compressed binaries is probably bigger than between two uncompressed ones.

Also, if libraries are uncompressed, then it might be way easier for git
to store compress, given that the content of the file reflects the
actual changes in the code. I mean, if you use compression, then given
by how the dictionary is composed, contents might change significantly.
While, on the other hand, an uncompressed library will basically have
the same content, only maybe slightly offsetted.

b) We should make more libraries and reference the projects instead of copying blobs around projects. For a start, this could be done with Git submodules [1] and importing Jitsi's Ant script into Videobridge, Jicofo, etc. A proper modularization into small, distinct libraries with whatever build system could come later.

Have you worked with submodules / subtrees before? Whenever I read into
the subject I get confused by which is better in the "general" case and
how one should use them. I don't mind using them, but I don't have a
clear image of how we should. (I've also read on occasion hints on how
"the other one" is better, but without a real clarification.)

Danny

···

On 13-04-15 09:22, Ingo Bauersachs wrote:

Boris

Ingo

[1] http://git-scm.com/docs/git-submodule

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#4

Forgot to include reference to git subtrees.

Git subtree spec:
https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt

···

On 14-04-15 22:12, Danny van Heumen wrote:

In line ...

On 13-04-15 09:22, Ingo Bauersachs wrote:

I just noticed that we generate the jars for the Jitsi bundles without
compression. I see how that makes sense in Jitsi where they are not
stored in the repository, and are distributed in a package which is
itself compressed. However, now that we're using them in other projects
(videobridge, jigasi, jicofo) where they are stored as blobs I think it
would be good to compress them (libjitsi.jar is now 6.6MB, while 3.4MB
compressed).

What do you think? Am I missing something here (I assume the load time
different for the jvm is negligible), or should I open an issue about this?

a) As Git compresses binaries and only stores deltas, it shouldn't make a difference if we compress the jars before committing or not. It's possibly even backfiring if we'd do so because the delta between two compressed binaries is probably bigger than between two uncompressed ones.

Also, if libraries are uncompressed, then it might be way easier for git
to store compress, given that the content of the file reflects the
actual changes in the code. I mean, if you use compression, then given
by how the dictionary is composed, contents might change significantly.
While, on the other hand, an uncompressed library will basically have
the same content, only maybe slightly offsetted.

b) We should make more libraries and reference the projects instead of copying blobs around projects. For a start, this could be done with Git submodules [1] and importing Jitsi's Ant script into Videobridge, Jicofo, etc. A proper modularization into small, distinct libraries with whatever build system could come later.

Have you worked with submodules / subtrees before? Whenever I read into
the subject I get confused by which is better in the "general" case and
how one should use them. I don't mind using them, but I don't have a
clear image of how we should. (I've also read on occasion hints on how
"the other one" is better, but without a real clarification.)

Danny

Boris

Ingo

[1] http://git-scm.com/docs/git-submodule

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#5

b) We should make more libraries and reference the projects instead of
copying blobs around projects. For a start, this could be done with Git
submodules [1] and importing Jitsi's Ant script into Videobridge, Jicofo,
etc. A proper modularization into small, distinct libraries with whatever
build system could come later.

Have you worked with submodules / subtrees before? Whenever I read into
the subject I get confused by which is better in the "general" case and
how one should use them. I don't mind using them, but I don't have a
clear image of how we should. (I've also read on occasion hints on how
"the other one" is better, but without a real clarification.)

No, I haven't used either of them. But the description of git-subtree makes me feel a bit more awkward compared to git-submodules. E.g. with subtree you can commit into the pulled in module alongside with changes in the "parent". The "split" for pushing those changes upstream seems weird.
A submodule on the other hand points to a specific commit of another project that is then kind-of symlinked into a subdirectory.

Yet another option is to actually require jitsi and libjitsi in a folder next to Jicofo/Videobridge etc as a build requirement, just as we do require a clone of some other repos for the deb-src target.

Whatever is easier for those who still don't like Git I guess (as long as those binaries vanish at some point).

Danny

Boris

Ingo

[1] http://git-scm.com/docs/git-submodule

Ingo


#6

b) We should make more libraries and reference the projects instead of
copying blobs around projects. For a start, this could be done with Git
submodules [1] and importing Jitsi's Ant script into Videobridge, Jicofo,
etc. A proper modularization into small, distinct libraries with whatever
build system could come later.

Have you worked with submodules / subtrees before? Whenever I read into
the subject I get confused by which is better in the "general" case and
how one should use them. I don't mind using them, but I don't have a
clear image of how we should. (I've also read on occasion hints on how
"the other one" is better, but without a real clarification.)

No, I haven't used either of them. But the description of git-subtree makes me feel a bit more awkward compared to git-submodules. E.g. with subtree you can commit into the pulled in module alongside with changes in the "parent". The "split" for pushing those changes upstream seems weird.
A submodule on the other hand points to a specific commit of another project that is then kind-of symlinked into a subdirectory.

The submodule solution sounds like it's more obvious one, I guess.

Yet another option is to actually require jitsi and libjitsi in a folder next to Jicofo/Videobridge etc as a build requirement, just as we do require a clone of some other repos for the deb-src target.

This kind of sounds like git subtree IMO. Only you'd need to parent
directory in which all the subtrees are created.

You should also keep in mind that these things (probably) have to be
initialized in some way. I mean, if you don't do the initialization, you
start with a kind of "half-way" checkout, ... I think.

I guess you'd just need to try to see what works in this case.

Danny

···

On 14-04-15 23:14, Ingo Bauersachs wrote:

Whatever is easier for those who still don't like Git I guess (as long as those binaries vanish at some point).

Danny

Boris

Ingo

[1] http://git-scm.com/docs/git-submodule

Ingo

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev