[sip-comm-dev] Performance GSOC project status


#1

Hello all,

In a first part of GSOC I was dealing with profiling sip-communicator. I was looking for laggy slow places in sip-comm and tried to profile them and then optimize. The found problems from user perspective were:
1. slow sip-communicator loading
2. graphics slow loading
3. big chat history slow processing

The starting up of the sip-communicator is the slowest part of it, it takes about 10 - 15 seconds. It uses apache felix for loading up the bundles, it takes up to 330 milliseconds for every bundle to start up. Other time for starting up is going to the drawing swing/awt components and loading up images. When profiling the functionality I just see Felix and JDK methods invocations, it is up profiling JDK and Felix, their methods are time-consumptive.

Very similar situation was with big chat history slow processing. The slowest part is the text parsing with javax.swing.text.html.HTMLFactory (see net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner class HTMLFactoryX). There is two ways of eliminating the problem:
- wait for updated and more optimized java version
- to profile jdk
- to use other technology for text formating

Then I just monitored the sip-communicator with a profile to search for anomalies (took the 10 most expensive methods). I found that processSmilies() method from ChattConversationPanel class class is also consumpitve with big texts, thist uses regular expressions for smilies finding and the regular expression is very complex for it.

I noticed that net.java.sip.communicator.util.ScLogFormatter is also time-consumptive, it takes about 2-3% of sip-communicator execution. The problems are format(LogRecord record) and inferCaller(LogRecord record) methods. First is using DecimalFormat for formatting is required time to process and also sending the messages to the output. The second is using StackTraceElement objects and processings, that are time consumptive. These methods are made with help of JDK and those are not very effective.

There are Thread.sleep's in sip-comm code which are look like performance problems in profiler (look in net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or just search for .sleep in sip-comm's source, you will found some), is the value of the sleeping time correct ?

There are minor time problems that are also connected with libraries that are slow. I attach a profiler's report if you want to see all the problems, it was made by means of JProfiler.

···

*
Notes for developers.*
What I can suggest to community is to continue adhere design patterns. Also quite effective is not to return a new created object, but reuse an already existing one - a creation of an object take more time and makes the heap larger. Of course it is not often possible, for example in net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner class HTMLFactoryX.

static class HTMLFactoryX extends HTMLFactory
        implements ViewFactory {
                public View create(Element elem) {
                 View v = super.create(elem);

            if(v instanceof ImageView){
                return new SIPCommImageView(elem);
            }
            else if (v instanceof ParagraphView) {
                return new ParagraphViewX(elem);
            }
            return v;
        }
    }

Namely View is created by create(Element element) method from javax.swing.text.html.HTMLFactory class, the Element object is necessary for the new instance of view , and there is no possible to clean up and return existing View class.

But if we see the createDefaultDocument() in c class, we see that HTMLDocument doc variable can be reused - there is no in parameter for the object:
        public Document createDefaultDocument() {
        if(doc != null) return doc;
        StyleSheet styles = getStyleSheet();
        StyleSheet ss = new StyleSheet();

        ss.addStyleSheet(styles);

        doc = new HTMLDocument(ss);
        doc.setParser(getParser());
        doc.setAsynchronousLoadPriority(4);
        doc.setTokenThreshold(100);
        return doc;
    }

Also I reused the creation of new SIPCommHTMLEditorKit classes in ChatConversationPanel. Actually the fix is not very helpful, because the method createDefaultDocument() is called only once from ChatConversationPanel but the same logic you can use in your projects.

Cheers,
Vladimir �karupelov


#2

Vladimir,

Thank you for the detailed description!

1. Which profilers did you use in your work?

Going by the mentioning of JProfiler, I presume I figure at least it
was used. I personally haven't used it so I cannot comment on its
accuracy on CPU profiling.

However, I've used YourKit Java Profiler, NetBeans Profiler and
Eclipse TPTP (well, a long time ago because it still doesn't run on
Mac OS X) and I can certainly say that CPU profiles are different
depending on the profiler in use. For example, I find YourKit Java
Profiler to have a tendency to overly penalize frequently executed
methods (it's a pity because I'm used to the way it presents heap
dumps and I don't think NetBeans Profilers comes any closer with
respect to usability) which in a way ruins the usefulness the
hot-spot statistics. In that respect, I find NetBeans Profiler to be
better at it.

Anyway, I think we should be creating and comparing CPU profiles in
more than one profiler in order to get a more fine-grained sense as to
where optimizations can and are worthy to be made. (Of course, I
presume the major bottlenecks are obvious regardless of the profiler
in use and we want to move forward.)

2. What about threads?

I often look at the number of threads in the running instance of SIP
Communicator and I'm totally puzzled why would a messenger with a
couple of registered accounts need more than 40 threads (I have 50 at
the time of this writing). For example, I'm using the Transmission
BitTorrent client on Mac OS X at the moment to download 3 torrents and
it has only 12 threads. (I'm citing the current numbers but the
average observations are close because I often look at these.)

When optimizing based on CPU profiles, I'd say one needs the
statistics about which threads consume the precious time in order to:
- focus on optimizations that would yield better perceived execution times, and
- suggest implementation and/or architecture changes if local method
optimizations aren't enough.

Have you been able to observe these and draw conclusions from them?

3. And the memory?

While I see you've touched on the recommendations for reusing
instances, I didn't see an analysis on the object creation, class
loading, etc.

And if the above aren't of interest to you for general optimizations
(and I think they should), are there memory leaks? I would personally
recommend YourKit Java Profiler in the area and I've been able to get
good return of effort from SAP/Eclipse Memory Analyzer
(http://www.eclipse.org/mat/).

Best regards,
Lubo

···

On Thu, Jul 17, 2008 at 2:11 PM, Vladimir Škarupelov <voviss@gmail.com> wrote:

Hello all,

In a first part of GSOC I was dealing with profiling sip-communicator. I was
looking for laggy slow places in sip-comm and tried to profile them and then
optimize. The found problems from user perspective were:
1. slow sip-communicator loading
2. graphics slow loading
3. big chat history slow processing

The starting up of the sip-communicator is the slowest part of it, it takes
about 10 - 15 seconds. It uses apache felix for loading up the bundles, it
takes up to 330 milliseconds for every bundle to start up. Other time for
starting up is going to the drawing swing/awt components and loading up
images. When profiling the functionality I just see Felix and JDK methods
invocations, it is up profiling JDK and Felix, their methods are
time-consumptive.

Very similar situation was with big chat history slow processing. The
slowest part is the text parsing with javax.swing.text.html.HTMLFactory (see
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner class
HTMLFactoryX). There is two ways of eliminating the problem:
- wait for updated and more optimized java version
- to profile jdk
- to use other technology for text formating

Then I just monitored the sip-communicator with a profile to search for
anomalies (took the 10 most expensive methods). I found that
processSmilies() method from ChattConversationPanel class class is also
consumpitve with big texts, thist uses regular expressions for smilies
finding and the regular expression is very complex for it.

I noticed that net.java.sip.communicator.util.ScLogFormatter is also
time-consumptive, it takes about 2-3% of sip-communicator execution. The
problems are format(LogRecord record) and inferCaller(LogRecord record)
methods. First is using DecimalFormat for formatting is required time to
process and also sending the messages to the output. The second is using
StackTraceElement objects and processings, that are time consumptive. These
methods are made with help of JDK and those are not very effective.

There are Thread.sleep's in sip-comm code which are look like performance
problems in profiler (look in
net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or just
search for .sleep in sip-comm's source, you will found some), is the value
of the sleeping time correct ?

There are minor time problems that are also connected with libraries that
are slow. I attach a profiler's report if you want to see all the problems,
it was made by means of JProfiler.

Notes for developers.
What I can suggest to community is to continue adhere design patterns. Also
quite effective is not to return a new created object, but reuse an already
existing one - a creation of an object take more time and makes the heap
larger. Of course it is not often possible, for example in
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner class
HTMLFactoryX.

static class HTMLFactoryX extends HTMLFactory
        implements ViewFactory {

        public View create(Element elem) {

            View v = super.create(elem);

            if(v instanceof ImageView){
                return new SIPCommImageView(elem);
            }
            else if (v instanceof ParagraphView) {
                return new ParagraphViewX(elem);
            }
            return v;
        }
    }

Namely View is created by create(Element element) method from
javax.swing.text.html.HTMLFactory class, the Element object is necessary for
the new instance of view , and there is no possible to clean up and return
existing View class.

But if we see the createDefaultDocument() in c class, we see that
HTMLDocument doc variable can be reused - there is no in parameter for the
object:

    public Document createDefaultDocument() {
        if(doc != null) return doc;
        StyleSheet styles = getStyleSheet();
        StyleSheet ss = new StyleSheet();

        ss.addStyleSheet(styles);

        doc = new HTMLDocument(ss);
        doc.setParser(getParser());
        doc.setAsynchronousLoadPriority(4);
        doc.setTokenThreshold(100);
        return doc;
    }

Also I reused the creation of new SIPCommHTMLEditorKit classes in
ChatConversationPanel. Actually the fix is not very helpful, because the
method createDefaultDocument() is called only once from
ChatConversationPanel but the same logic you can use in your projects.

Cheers,
Vladimir Škarupelov

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hello Lubomir,

For profiling sip-communicator I used JProfiler and Eclipse TPTP profilers, the first is not free, but it is faster than the others I tried, the TPTP project is very handy for me, because I used to work with eclipse, but this profiler is slow on my computer - it requires a lot of RAM. I tried also tried to use hprof and netbeans profiler for it, but it was enough for me to use that two ones, there were a lot code to test. When testing the code I used old but always working System.currentTimeMillis() method, for getting the timings, and put a counter for objects invocations. The time of method executions were mostly the same in TPTP and JProfiler, moreover I compared them with currentTimeMillis(), of course the the time of currentTimeMillis was always a little bit smaller (up to 5%) than the profilers one. Only problem that way can be compared only methods that take more than one millisecond. I think I try also NetBeans for timing testing.

About threads.
Thanks for the pointing to the problem, I mostly tested cpu and memory executions of sip-comm. There is something really very weird with the threads: if you start chatting with somebody you get 4 new threads in waiting status, if you close your chatting window, the threads stay alive. And you get new 4 threads if you start the next conversation, and so on ...

Memory leaks.
Sip-communicator was working pretty well without heavy traffic at night, the maximum heap size was only 20 mb, every 3 hours (when the size had reached ~ 20 mb) the garbage was collected and the heap size had dropped for 10 mb.
But when chatting the number of String and dependent on it char[] is becoming bigger. Also becoming more byte[] objects, it seems that the information is sent in bytes and not correctly garbage collected.
If you see the objects in memory, you see there (memoryAnalysis.png), for example, big number of javax.swing.text.View[] objects, these objects are not directly invoked by sip-communicator, but with the technology that sip-communicator use (HTMLDocument for the View arrays, for example, see attached memoryAnalysis2.png).
But anyway, I think I'll try the soft you sent to me.

Thanks for your interest,
Vladimir �karupelov

Lubomir Marinov wrote:

···

Vladimir,

Thank you for the detailed description!

1. Which profilers did you use in your work?

Going by the mentioning of JProfiler, I presume I figure at least it
was used. I personally haven't used it so I cannot comment on its
accuracy on CPU profiling.

However, I've used YourKit Java Profiler, NetBeans Profiler and
Eclipse TPTP (well, a long time ago because it still doesn't run on
Mac OS X) and I can certainly say that CPU profiles are different
depending on the profiler in use. For example, I find YourKit Java
Profiler to have a tendency to overly penalize frequently executed
methods (it's a pity because I'm used to the way it presents heap
dumps and I don't think NetBeans Profilers comes any closer with
respect to usability) which in a way ruins the usefulness the
hot-spot statistics. In that respect, I find NetBeans Profiler to be
better at it.

Anyway, I think we should be creating and comparing CPU profiles in
more than one profiler in order to get a more fine-grained sense as to
where optimizations can and are worthy to be made. (Of course, I
presume the major bottlenecks are obvious regardless of the profiler
in use and we want to move forward.)

2. What about threads?

I often look at the number of threads in the running instance of SIP
Communicator and I'm totally puzzled why would a messenger with a
couple of registered accounts need more than 40 threads (I have 50 at
the time of this writing). For example, I'm using the Transmission
BitTorrent client on Mac OS X at the moment to download 3 torrents and
it has only 12 threads. (I'm citing the current numbers but the
average observations are close because I often look at these.)

When optimizing based on CPU profiles, I'd say one needs the
statistics about which threads consume the precious time in order to:
- focus on optimizations that would yield better perceived execution times, and
- suggest implementation and/or architecture changes if local method
optimizations aren't enough.

Have you been able to observe these and draw conclusions from them?

3. And the memory?

While I see you've touched on the recommendations for reusing
instances, I didn't see an analysis on the object creation, class
loading, etc.

And if the above aren't of interest to you for general optimizations
(and I think they should), are there memory leaks? I would personally
recommend YourKit Java Profiler in the area and I've been able to get
good return of effort from SAP/Eclipse Memory Analyzer
(http://www.eclipse.org/mat/).

Best regards,
Lubo

On Thu, Jul 17, 2008 at 2:11 PM, Vladimir �karupelov <voviss@gmail.com> wrote:
  

Hello all,

In a first part of GSOC I was dealing with profiling sip-communicator. I was
looking for laggy slow places in sip-comm and tried to profile them and then
optimize. The found problems from user perspective were:
1. slow sip-communicator loading
2. graphics slow loading
3. big chat history slow processing

The starting up of the sip-communicator is the slowest part of it, it takes
about 10 - 15 seconds. It uses apache felix for loading up the bundles, it
takes up to 330 milliseconds for every bundle to start up. Other time for
starting up is going to the drawing swing/awt components and loading up
images. When profiling the functionality I just see Felix and JDK methods
invocations, it is up profiling JDK and Felix, their methods are
time-consumptive.

Very similar situation was with big chat history slow processing. The
slowest part is the text parsing with javax.swing.text.html.HTMLFactory (see
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner class
HTMLFactoryX). There is two ways of eliminating the problem:
- wait for updated and more optimized java version
- to profile jdk
- to use other technology for text formating

Then I just monitored the sip-communicator with a profile to search for
anomalies (took the 10 most expensive methods). I found that
processSmilies() method from ChattConversationPanel class class is also
consumpitve with big texts, thist uses regular expressions for smilies
finding and the regular expression is very complex for it.

I noticed that net.java.sip.communicator.util.ScLogFormatter is also
time-consumptive, it takes about 2-3% of sip-communicator execution. The
problems are format(LogRecord record) and inferCaller(LogRecord record)
methods. First is using DecimalFormat for formatting is required time to
process and also sending the messages to the output. The second is using
StackTraceElement objects and processings, that are time consumptive. These
methods are made with help of JDK and those are not very effective.

There are Thread.sleep's in sip-comm code which are look like performance
problems in profiler (look in
net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or just
search for .sleep in sip-comm's source, you will found some), is the value
of the sleeping time correct ?

There are minor time problems that are also connected with libraries that
are slow. I attach a profiler's report if you want to see all the problems,
it was made by means of JProfiler.

Notes for developers.
What I can suggest to community is to continue adhere design patterns. Also
quite effective is not to return a new created object, but reuse an already
existing one - a creation of an object take more time and makes the heap
larger. Of course it is not often possible, for example in
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner class
HTMLFactoryX.

static class HTMLFactoryX extends HTMLFactory
        implements ViewFactory {

        public View create(Element elem) {

            View v = super.create(elem);

            if(v instanceof ImageView){
                return new SIPCommImageView(elem);
            }
            else if (v instanceof ParagraphView) {
                return new ParagraphViewX(elem);
            }
            return v;
        }
    }

Namely View is created by create(Element element) method from
javax.swing.text.html.HTMLFactory class, the Element object is necessary for
the new instance of view , and there is no possible to clean up and return
existing View class.

But if we see the createDefaultDocument() in c class, we see that
HTMLDocument doc variable can be reused - there is no in parameter for the
object:

    public Document createDefaultDocument() {
        if(doc != null) return doc;
        StyleSheet styles = getStyleSheet();
        StyleSheet ss = new StyleSheet();

        ss.addStyleSheet(styles);

        doc = new HTMLDocument(ss);
        doc.setParser(getParser());
        doc.setAsynchronousLoadPriority(4);
        doc.setTokenThreshold(100);
        return doc;
    }

Also I reused the creation of new SIPCommHTMLEditorKit classes in
ChatConversationPanel. Actually the fix is not very helpful, because the
method createDefaultDocument() is called only once from
ChatConversationPanel but the same logic you can use in your projects.

Cheers,
Vladimir �karupelov

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

As the developers tracking the SVN commits have probably noticed
already, I've started slowly fixing some memory leaks. Since I'm
already used to the heap-dump browsing in YourKit Java Profiler, I
applied for a 15-day evaluation license for the product in question.
In a few days I got contacted through e-mail by sales@yourkit.com in
search of feedback about my evaluation. After answering the questions
and mentioning that I was using the product to profile SIP
Communicator, I was given a 1-year license for free and I was told
that should any other SIP Communicator developers need licenses, I
should only send the list of their names and e-mails and the licenses
would be sent to them. Please let me know if you're interested in such
an offer.

Best regards,
Lubo

···

On Thu, Jul 17, 2008 at 5:01 PM, Vladimir Škarupelov <voviss@gmail.com> wrote:

Hello Lubomir,

For profiling sip-communicator I used JProfiler and Eclipse TPTP profilers,
the first is not free, but it is faster than the others I tried, the TPTP
project is very handy for me, because I used to work with eclipse, but this
profiler is slow on my computer - it requires a lot of RAM. I tried also
tried to use hprof and netbeans profiler for it, but it was enough for me to
use that two ones, there were a lot code to test. When testing the code I
used old but always working System.currentTimeMillis() method, for getting
the timings, and put a counter for objects invocations. The time of method
executions were mostly the same in TPTP and JProfiler, moreover I compared
them with currentTimeMillis(), of course the the time of currentTimeMillis
was always a little bit smaller (up to 5%) than the profilers one. Only
problem that way can be compared only methods that take more than one
millisecond. I think I try also NetBeans for timing testing.

About threads.
Thanks for the pointing to the problem, I mostly tested cpu and memory
executions of sip-comm. There is something really very weird with the
threads: if you start chatting with somebody you get 4 new threads in
waiting status, if you close your chatting window, the threads stay alive.
And you get new 4 threads if you start the next conversation, and so on ...

Memory leaks.
Sip-communicator was working pretty well without heavy traffic at night, the
maximum heap size was only 20 mb, every 3 hours (when the size had reached ~
20 mb) the garbage was collected and the heap size had dropped for 10 mb.
But when chatting the number of String and dependent on it char[] is
becoming bigger. Also becoming more byte[] objects, it seems that the
information is sent in bytes and not correctly garbage collected.
If you see the objects in memory, you see there (memoryAnalysis.png), for
example, big number of javax.swing.text.View[] objects, these objects are
not directly invoked by sip-communicator, but with the technology that
sip-communicator use (HTMLDocument for the View arrays, for example, see
attached memoryAnalysis2.png).
But anyway, I think I'll try the soft you sent to me.

Thanks for your interest,
Vladimir Škarupelov

Lubomir Marinov wrote:

Vladimir,

Thank you for the detailed description!

1. Which profilers did you use in your work?

Going by the mentioning of JProfiler, I presume I figure at least it
was used. I personally haven't used it so I cannot comment on its
accuracy on CPU profiling.

However, I've used YourKit Java Profiler, NetBeans Profiler and
Eclipse TPTP (well, a long time ago because it still doesn't run on
Mac OS X) and I can certainly say that CPU profiles are different
depending on the profiler in use. For example, I find YourKit Java
Profiler to have a tendency to overly penalize frequently executed
methods (it's a pity because I'm used to the way it presents heap
dumps and I don't think NetBeans Profilers comes any closer with
respect to usability) which in a way ruins the usefulness the
hot-spot statistics. In that respect, I find NetBeans Profiler to be
better at it.

Anyway, I think we should be creating and comparing CPU profiles in
more than one profiler in order to get a more fine-grained sense as to
where optimizations can and are worthy to be made. (Of course, I
presume the major bottlenecks are obvious regardless of the profiler
in use and we want to move forward.)

2. What about threads?

I often look at the number of threads in the running instance of SIP
Communicator and I'm totally puzzled why would a messenger with a
couple of registered accounts need more than 40 threads (I have 50 at
the time of this writing). For example, I'm using the Transmission
BitTorrent client on Mac OS X at the moment to download 3 torrents and
it has only 12 threads. (I'm citing the current numbers but the
average observations are close because I often look at these.)

When optimizing based on CPU profiles, I'd say one needs the
statistics about which threads consume the precious time in order to:
- focus on optimizations that would yield better perceived execution times,
and
- suggest implementation and/or architecture changes if local method
optimizations aren't enough.

Have you been able to observe these and draw conclusions from them?

3. And the memory?

While I see you've touched on the recommendations for reusing
instances, I didn't see an analysis on the object creation, class
loading, etc.

And if the above aren't of interest to you for general optimizations
(and I think they should), are there memory leaks? I would personally
recommend YourKit Java Profiler in the area and I've been able to get
good return of effort from SAP/Eclipse Memory Analyzer
(http://www.eclipse.org/mat/).

Best regards,
Lubo

On Thu, Jul 17, 2008 at 2:11 PM, Vladimir Škarupelov <voviss@gmail.com> > wrote:

Hello all,

In a first part of GSOC I was dealing with profiling sip-communicator. I was
looking for laggy slow places in sip-comm and tried to profile them and then
optimize. The found problems from user perspective were:
1. slow sip-communicator loading
2. graphics slow loading
3. big chat history slow processing

The starting up of the sip-communicator is the slowest part of it, it takes
about 10 - 15 seconds. It uses apache felix for loading up the bundles, it
takes up to 330 milliseconds for every bundle to start up. Other time for
starting up is going to the drawing swing/awt components and loading up
images. When profiling the functionality I just see Felix and JDK methods
invocations, it is up profiling JDK and Felix, their methods are
time-consumptive.

Very similar situation was with big chat history slow processing. The
slowest part is the text parsing with javax.swing.text.html.HTMLFactory (see
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner class
HTMLFactoryX). There is two ways of eliminating the problem:
- wait for updated and more optimized java version
- to profile jdk
- to use other technology for text formating

Then I just monitored the sip-communicator with a profile to search for
anomalies (took the 10 most expensive methods). I found that
processSmilies() method from ChattConversationPanel class class is also
consumpitve with big texts, thist uses regular expressions for smilies
finding and the regular expression is very complex for it.

I noticed that net.java.sip.communicator.util.ScLogFormatter is also
time-consumptive, it takes about 2-3% of sip-communicator execution. The
problems are format(LogRecord record) and inferCaller(LogRecord record)
methods. First is using DecimalFormat for formatting is required time to
process and also sending the messages to the output. The second is using
StackTraceElement objects and processings, that are time consumptive. These
methods are made with help of JDK and those are not very effective.

There are Thread.sleep's in sip-comm code which are look like performance
problems in profiler (look in
net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or just
search for .sleep in sip-comm's source, you will found some), is the value
of the sleeping time correct ?

There are minor time problems that are also connected with libraries that
are slow. I attach a profiler's report if you want to see all the problems,
it was made by means of JProfiler.

Notes for developers.
What I can suggest to community is to continue adhere design patterns. Also
quite effective is not to return a new created object, but reuse an already
existing one - a creation of an object take more time and makes the heap
larger. Of course it is not often possible, for example in
net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner class
HTMLFactoryX.

static class HTMLFactoryX extends HTMLFactory
        implements ViewFactory {

        public View create(Element elem) {

            View v = super.create(elem);

            if(v instanceof ImageView){
                return new SIPCommImageView(elem);
            }
            else if (v instanceof ParagraphView) {
                return new ParagraphViewX(elem);
            }
            return v;
        }
    }

Namely View is created by create(Element element) method from
javax.swing.text.html.HTMLFactory class, the Element object is necessary for
the new instance of view , and there is no possible to clean up and return
existing View class.

But if we see the createDefaultDocument() in c class, we see that
HTMLDocument doc variable can be reused - there is no in parameter for the
object:

    public Document createDefaultDocument() {
        if(doc != null) return doc;
        StyleSheet styles = getStyleSheet();
        StyleSheet ss = new StyleSheet();

        ss.addStyleSheet(styles);

        doc = new HTMLDocument(ss);
        doc.setParser(getParser());
        doc.setAsynchronousLoadPriority(4);
        doc.setTokenThreshold(100);
        return doc;
    }

Also I reused the creation of new SIPCommHTMLEditorKit classes in
ChatConversationPanel. Actually the fix is not very helpful, because the
method createDefaultDocument() is called only once from
ChatConversationPanel but the same logic you can use in your projects.

Cheers,
Vladimir Škarupelov

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#5

Hello,

Sure, I am interested. Name: Vladimir Shkarupelov.

Thanks,
Vladimir

···

On Sun, Aug 3, 2008 at 10:22 PM, Lubomir Marinov <lubomir.marinov@gmail.com>wrote:

As the developers tracking the SVN commits have probably noticed
already, I've started slowly fixing some memory leaks. Since I'm
already used to the heap-dump browsing in YourKit Java Profiler, I
applied for a 15-day evaluation license for the product in question.
In a few days I got contacted through e-mail by sales@yourkit.com in
search of feedback about my evaluation. After answering the questions
and mentioning that I was using the product to profile SIP
Communicator, I was given a 1-year license for free and I was told
that should any other SIP Communicator developers need licenses, I
should only send the list of their names and e-mails and the licenses
would be sent to them. Please let me know if you're interested in such
an offer.

Best regards,
Lubo

On Thu, Jul 17, 2008 at 5:01 PM, Vladimir Škarupelov <voviss@gmail.com> > wrote:
> Hello Lubomir,
>
> For profiling sip-communicator I used JProfiler and Eclipse TPTP
profilers,
> the first is not free, but it is faster than the others I tried, the TPTP
> project is very handy for me, because I used to work with eclipse, but
this
> profiler is slow on my computer - it requires a lot of RAM. I tried also
> tried to use hprof and netbeans profiler for it, but it was enough for me
to
> use that two ones, there were a lot code to test. When testing the code I
> used old but always working System.currentTimeMillis() method, for
getting
> the timings, and put a counter for objects invocations. The time of
method
> executions were mostly the same in TPTP and JProfiler, moreover I
compared
> them with currentTimeMillis(), of course the the time of
currentTimeMillis
> was always a little bit smaller (up to 5%) than the profilers one. Only
> problem that way can be compared only methods that take more than one
> millisecond. I think I try also NetBeans for timing testing.
>
> About threads.
> Thanks for the pointing to the problem, I mostly tested cpu and memory
> executions of sip-comm. There is something really very weird with the
> threads: if you start chatting with somebody you get 4 new threads in
> waiting status, if you close your chatting window, the threads stay
alive.
> And you get new 4 threads if you start the next conversation, and so on
...
>
> Memory leaks.
> Sip-communicator was working pretty well without heavy traffic at night,
the
> maximum heap size was only 20 mb, every 3 hours (when the size had
reached ~
> 20 mb) the garbage was collected and the heap size had dropped for 10 mb.
> But when chatting the number of String and dependent on it char[] is
> becoming bigger. Also becoming more byte[] objects, it seems that the
> information is sent in bytes and not correctly garbage collected.
> If you see the objects in memory, you see there (memoryAnalysis.png), for
> example, big number of javax.swing.text.View[] objects, these objects are
> not directly invoked by sip-communicator, but with the technology that
> sip-communicator use (HTMLDocument for the View arrays, for example, see
> attached memoryAnalysis2.png).
> But anyway, I think I'll try the soft you sent to me.
>
> Thanks for your interest,
> Vladimir Škarupelov
>
> Lubomir Marinov wrote:
>
> Vladimir,
>
> Thank you for the detailed description!
>
> 1. Which profilers did you use in your work?
>
> Going by the mentioning of JProfiler, I presume I figure at least it
> was used. I personally haven't used it so I cannot comment on its
> accuracy on CPU profiling.
>
> However, I've used YourKit Java Profiler, NetBeans Profiler and
> Eclipse TPTP (well, a long time ago because it still doesn't run on
> Mac OS X) and I can certainly say that CPU profiles are different
> depending on the profiler in use. For example, I find YourKit Java
> Profiler to have a tendency to overly penalize frequently executed
> methods (it's a pity because I'm used to the way it presents heap
> dumps and I don't think NetBeans Profilers comes any closer with
> respect to usability) which in a way ruins the usefulness the
> hot-spot statistics. In that respect, I find NetBeans Profiler to be
> better at it.
>
> Anyway, I think we should be creating and comparing CPU profiles in
> more than one profiler in order to get a more fine-grained sense as to
> where optimizations can and are worthy to be made. (Of course, I
> presume the major bottlenecks are obvious regardless of the profiler
> in use and we want to move forward.)
>
> 2. What about threads?
>
> I often look at the number of threads in the running instance of SIP
> Communicator and I'm totally puzzled why would a messenger with a
> couple of registered accounts need more than 40 threads (I have 50 at
> the time of this writing). For example, I'm using the Transmission
> BitTorrent client on Mac OS X at the moment to download 3 torrents and
> it has only 12 threads. (I'm citing the current numbers but the
> average observations are close because I often look at these.)
>
> When optimizing based on CPU profiles, I'd say one needs the
> statistics about which threads consume the precious time in order to:
> - focus on optimizations that would yield better perceived execution
times,
> and
> - suggest implementation and/or architecture changes if local method
> optimizations aren't enough.
>
> Have you been able to observe these and draw conclusions from them?
>
> 3. And the memory?
>
> While I see you've touched on the recommendations for reusing
> instances, I didn't see an analysis on the object creation, class
> loading, etc.
>
> And if the above aren't of interest to you for general optimizations
> (and I think they should), are there memory leaks? I would personally
> recommend YourKit Java Profiler in the area and I've been able to get
> good return of effort from SAP/Eclipse Memory Analyzer
> (http://www.eclipse.org/mat/).
>
> Best regards,
> Lubo
>
> On Thu, Jul 17, 2008 at 2:11 PM, Vladimir Škarupelov <voviss@gmail.com> > > wrote:
>
>
> Hello all,
>
> In a first part of GSOC I was dealing with profiling sip-communicator. I
was
> looking for laggy slow places in sip-comm and tried to profile them and
then
> optimize. The found problems from user perspective were:
> 1. slow sip-communicator loading
> 2. graphics slow loading
> 3. big chat history slow processing
>
> The starting up of the sip-communicator is the slowest part of it, it
takes
> about 10 - 15 seconds. It uses apache felix for loading up the bundles,
it
> takes up to 330 milliseconds for every bundle to start up. Other time for
> starting up is going to the drawing swing/awt components and loading up
> images. When profiling the functionality I just see Felix and JDK methods
> invocations, it is up profiling JDK and Felix, their methods are
> time-consumptive.
>
> Very similar situation was with big chat history slow processing. The
> slowest part is the text parsing with javax.swing.text.html.HTMLFactory
(see
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner
class
> HTMLFactoryX). There is two ways of eliminating the problem:
> - wait for updated and more optimized java version
> - to profile jdk
> - to use other technology for text formating
>
>
> Then I just monitored the sip-communicator with a profile to search for
> anomalies (took the 10 most expensive methods). I found that
> processSmilies() method from ChattConversationPanel class class is also
> consumpitve with big texts, thist uses regular expressions for smilies
> finding and the regular expression is very complex for it.
>
> I noticed that net.java.sip.communicator.util.ScLogFormatter is also
> time-consumptive, it takes about 2-3% of sip-communicator execution. The
> problems are format(LogRecord record) and inferCaller(LogRecord record)
> methods. First is using DecimalFormat for formatting is required time to
> process and also sending the messages to the output. The second is using
> StackTraceElement objects and processings, that are time consumptive.
These
> methods are made with help of JDK and those are not very effective.
>
> There are Thread.sleep's in sip-comm code which are look like performance
> problems in profiler (look in
> net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or
just
> search for .sleep in sip-comm's source, you will found some), is the
value
> of the sleeping time correct ?
>
> There are minor time problems that are also connected with libraries that
> are slow. I attach a profiler's report if you want to see all the
problems,
> it was made by means of JProfiler.
>
> Notes for developers.
> What I can suggest to community is to continue adhere design patterns.
Also
> quite effective is not to return a new created object, but reuse an
already
> existing one - a creation of an object take more time and makes the heap
> larger. Of course it is not often possible, for example in
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner class
> HTMLFactoryX.
>
> static class HTMLFactoryX extends HTMLFactory
> implements ViewFactory {
>
> public View create(Element elem) {
>
> View v = super.create(elem);
>
> if(v instanceof ImageView){
> return new SIPCommImageView(elem);
> }
> else if (v instanceof ParagraphView) {
> return new ParagraphViewX(elem);
> }
> return v;
> }
> }
>
> Namely View is created by create(Element element) method from
> javax.swing.text.html.HTMLFactory class, the Element object is necessary
for
> the new instance of view , and there is no possible to clean up and
return
> existing View class.
>
> But if we see the createDefaultDocument() in c class, we see that
> HTMLDocument doc variable can be reused - there is no in parameter for
the
> object:
>
> public Document createDefaultDocument() {
> if(doc != null) return doc;
> StyleSheet styles = getStyleSheet();
> StyleSheet ss = new StyleSheet();
>
> ss.addStyleSheet(styles);
>
> doc = new HTMLDocument(ss);
> doc.setParser(getParser());
> doc.setAsynchronousLoadPriority(4);
> doc.setTokenThreshold(100);
> return doc;
> }
>
> Also I reused the creation of new SIPCommHTMLEditorKit classes in
> ChatConversationPanel. Actually the fix is not very helpful, because the
> method createDefaultDocument() is called only once from
> ChatConversationPanel but the same logic you can use in your projects.
>
> Cheers,
> Vladimir Škarupelov
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#6

Hi Lubo,

How it is going the licenses? Can I get a year long one =) ?

Best,
Vladimir

···

On Mon, Aug 4, 2008 at 1:36 PM, Vladimir Shkarupelov <voviss@gmail.com>wrote:

Hello,

Sure, I am interested. Name: Vladimir Shkarupelov.

Thanks,
Vladimir

  On Sun, Aug 3, 2008 at 10:22 PM, Lubomir Marinov < > lubomir.marinov@gmail.com> wrote:

As the developers tracking the SVN commits have probably noticed
already, I've started slowly fixing some memory leaks. Since I'm
already used to the heap-dump browsing in YourKit Java Profiler, I
applied for a 15-day evaluation license for the product in question.
In a few days I got contacted through e-mail by sales@yourkit.com in
search of feedback about my evaluation. After answering the questions
and mentioning that I was using the product to profile SIP
Communicator, I was given a 1-year license for free and I was told
that should any other SIP Communicator developers need licenses, I
should only send the list of their names and e-mails and the licenses
would be sent to them. Please let me know if you're interested in such
an offer.

Best regards,
Lubo

On Thu, Jul 17, 2008 at 5:01 PM, Vladimir Škarupelov <voviss@gmail.com> >> wrote:
> Hello Lubomir,
>
> For profiling sip-communicator I used JProfiler and Eclipse TPTP
profilers,
> the first is not free, but it is faster than the others I tried, the
TPTP
> project is very handy for me, because I used to work with eclipse, but
this
> profiler is slow on my computer - it requires a lot of RAM. I tried also
> tried to use hprof and netbeans profiler for it, but it was enough for
me to
> use that two ones, there were a lot code to test. When testing the code
I
> used old but always working System.currentTimeMillis() method, for
getting
> the timings, and put a counter for objects invocations. The time of
method
> executions were mostly the same in TPTP and JProfiler, moreover I
compared
> them with currentTimeMillis(), of course the the time of
currentTimeMillis
> was always a little bit smaller (up to 5%) than the profilers one. Only
> problem that way can be compared only methods that take more than one
> millisecond. I think I try also NetBeans for timing testing.
>
> About threads.
> Thanks for the pointing to the problem, I mostly tested cpu and memory
> executions of sip-comm. There is something really very weird with the
> threads: if you start chatting with somebody you get 4 new threads in
> waiting status, if you close your chatting window, the threads stay
alive.
> And you get new 4 threads if you start the next conversation, and so on
...
>
> Memory leaks.
> Sip-communicator was working pretty well without heavy traffic at night,
the
> maximum heap size was only 20 mb, every 3 hours (when the size had
reached ~
> 20 mb) the garbage was collected and the heap size had dropped for 10
mb.
> But when chatting the number of String and dependent on it char[] is
> becoming bigger. Also becoming more byte[] objects, it seems that the
> information is sent in bytes and not correctly garbage collected.
> If you see the objects in memory, you see there (memoryAnalysis.png),
for
> example, big number of javax.swing.text.View[] objects, these objects
are
> not directly invoked by sip-communicator, but with the technology that
> sip-communicator use (HTMLDocument for the View arrays, for example, see
> attached memoryAnalysis2.png).
> But anyway, I think I'll try the soft you sent to me.
>
> Thanks for your interest,
> Vladimir Škarupelov
>
> Lubomir Marinov wrote:
>
> Vladimir,
>
> Thank you for the detailed description!
>
> 1. Which profilers did you use in your work?
>
> Going by the mentioning of JProfiler, I presume I figure at least it
> was used. I personally haven't used it so I cannot comment on its
> accuracy on CPU profiling.
>
> However, I've used YourKit Java Profiler, NetBeans Profiler and
> Eclipse TPTP (well, a long time ago because it still doesn't run on
> Mac OS X) and I can certainly say that CPU profiles are different
> depending on the profiler in use. For example, I find YourKit Java
> Profiler to have a tendency to overly penalize frequently executed
> methods (it's a pity because I'm used to the way it presents heap
> dumps and I don't think NetBeans Profilers comes any closer with
> respect to usability) which in a way ruins the usefulness the
> hot-spot statistics. In that respect, I find NetBeans Profiler to be
> better at it.
>
> Anyway, I think we should be creating and comparing CPU profiles in
> more than one profiler in order to get a more fine-grained sense as to
> where optimizations can and are worthy to be made. (Of course, I
> presume the major bottlenecks are obvious regardless of the profiler
> in use and we want to move forward.)
>
> 2. What about threads?
>
> I often look at the number of threads in the running instance of SIP
> Communicator and I'm totally puzzled why would a messenger with a
> couple of registered accounts need more than 40 threads (I have 50 at
> the time of this writing). For example, I'm using the Transmission
> BitTorrent client on Mac OS X at the moment to download 3 torrents and
> it has only 12 threads. (I'm citing the current numbers but the
> average observations are close because I often look at these.)
>
> When optimizing based on CPU profiles, I'd say one needs the
> statistics about which threads consume the precious time in order to:
> - focus on optimizations that would yield better perceived execution
times,
> and
> - suggest implementation and/or architecture changes if local method
> optimizations aren't enough.
>
> Have you been able to observe these and draw conclusions from them?
>
> 3. And the memory?
>
> While I see you've touched on the recommendations for reusing
> instances, I didn't see an analysis on the object creation, class
> loading, etc.
>
> And if the above aren't of interest to you for general optimizations
> (and I think they should), are there memory leaks? I would personally
> recommend YourKit Java Profiler in the area and I've been able to get
> good return of effort from SAP/Eclipse Memory Analyzer
> (http://www.eclipse.org/mat/).
>
> Best regards,
> Lubo
>
> On Thu, Jul 17, 2008 at 2:11 PM, Vladimir Škarupelov <voviss@gmail.com> >> > wrote:
>
>
> Hello all,
>
> In a first part of GSOC I was dealing with profiling sip-communicator. I
was
> looking for laggy slow places in sip-comm and tried to profile them and
then
> optimize. The found problems from user perspective were:
> 1. slow sip-communicator loading
> 2. graphics slow loading
> 3. big chat history slow processing
>
> The starting up of the sip-communicator is the slowest part of it, it
takes
> about 10 - 15 seconds. It uses apache felix for loading up the bundles,
it
> takes up to 330 milliseconds for every bundle to start up. Other time
for
> starting up is going to the drawing swing/awt components and loading up
> images. When profiling the functionality I just see Felix and JDK
methods
> invocations, it is up profiling JDK and Felix, their methods are
> time-consumptive.
>
> Very similar situation was with big chat history slow processing. The
> slowest part is the text parsing with javax.swing.text.html.HTMLFactory
(see
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner
class
> HTMLFactoryX). There is two ways of eliminating the problem:
> - wait for updated and more optimized java version
> - to profile jdk
> - to use other technology for text formating
>
>
> Then I just monitored the sip-communicator with a profile to search for
> anomalies (took the 10 most expensive methods). I found that
> processSmilies() method from ChattConversationPanel class class is also
> consumpitve with big texts, thist uses regular expressions for smilies
> finding and the regular expression is very complex for it.
>
> I noticed that net.java.sip.communicator.util.ScLogFormatter is also
> time-consumptive, it takes about 2-3% of sip-communicator execution. The
> problems are format(LogRecord record) and inferCaller(LogRecord record)
> methods. First is using DecimalFormat for formatting is required time to
> process and also sending the messages to the output. The second is using
> StackTraceElement objects and processings, that are time consumptive.
These
> methods are made with help of JDK and those are not very effective.
>
> There are Thread.sleep's in sip-comm code which are look like
performance
> problems in profiler (look in
> net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or
just
> search for .sleep in sip-comm's source, you will found some), is the
value
> of the sleeping time correct ?
>
> There are minor time problems that are also connected with libraries
that
> are slow. I attach a profiler's report if you want to see all the
problems,
> it was made by means of JProfiler.
>
> Notes for developers.
> What I can suggest to community is to continue adhere design patterns.
Also
> quite effective is not to return a new created object, but reuse an
already
> existing one - a creation of an object take more time and makes the heap
> larger. Of course it is not often possible, for example in
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner
class
> HTMLFactoryX.
>
> static class HTMLFactoryX extends HTMLFactory
> implements ViewFactory {
>
> public View create(Element elem) {
>
> View v = super.create(elem);
>
> if(v instanceof ImageView){
> return new SIPCommImageView(elem);
> }
> else if (v instanceof ParagraphView) {
> return new ParagraphViewX(elem);
> }
> return v;
> }
> }
>
> Namely View is created by create(Element element) method from
> javax.swing.text.html.HTMLFactory class, the Element object is necessary
for
> the new instance of view , and there is no possible to clean up and
return
> existing View class.
>
> But if we see the createDefaultDocument() in c class, we see that
> HTMLDocument doc variable can be reused - there is no in parameter for
the
> object:
>
> public Document createDefaultDocument() {
> if(doc != null) return doc;
> StyleSheet styles = getStyleSheet();
> StyleSheet ss = new StyleSheet();
>
> ss.addStyleSheet(styles);
>
> doc = new HTMLDocument(ss);
> doc.setParser(getParser());
> doc.setAsynchronousLoadPriority(4);
> doc.setTokenThreshold(100);
> return doc;
> }
>
> Also I reused the creation of new SIPCommHTMLEditorKit classes in
> ChatConversationPanel. Actually the fix is not very helpful, because the
> method createDefaultDocument() is called only once from
> ChatConversationPanel but the same logic you can use in your projects.
>
> Cheers,
> Vladimir Škarupelov
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#7

Hey Vladimir,

I'm sorry to hear there's still no answer on the side of YourKit. I
sent your contact details to sales@yourkit.com, at which I had a
continuous e-mail exchange with Vladimir Kondratyev prior to
forwarding the offer on this mailing list, on August 13, 2008.

I'm not too surprised though. In his last e-mail Kondratyev asked for
details on an issue I alluded to in an earlier feedback e-mail of
mine, then I explained at length and presented examples but I haven't
heard from him ever since. I logged a feature request in their forums
and I didn't get an answer to it there as well. However, I logged a
bug report in the same forum and it got answered by Kondratyev the
very same day... Thus I'm left unaware of the cause of the lack of
response.

I'll resend your contact information to sales@yourkit.com now.

Best regards,
Lubo

···

On Thu, Aug 21, 2008 at 11:46 AM, Vladimir Shkarupelov <voviss@gmail.com> wrote:

Hi Lubo,

How it is going the licenses? Can I get a year long one =) ?

Best,
Vladimir

On Mon, Aug 4, 2008 at 1:36 PM, Vladimir Shkarupelov <voviss@gmail.com> > wrote:

Hello,

Sure, I am interested. Name: Vladimir Shkarupelov.

Thanks,
Vladimir

On Sun, Aug 3, 2008 at 10:22 PM, Lubomir Marinov >> <lubomir.marinov@gmail.com> wrote:

As the developers tracking the SVN commits have probably noticed
already, I've started slowly fixing some memory leaks. Since I'm
already used to the heap-dump browsing in YourKit Java Profiler, I
applied for a 15-day evaluation license for the product in question.
In a few days I got contacted through e-mail by sales@yourkit.com in
search of feedback about my evaluation. After answering the questions
and mentioning that I was using the product to profile SIP
Communicator, I was given a 1-year license for free and I was told
that should any other SIP Communicator developers need licenses, I
should only send the list of their names and e-mails and the licenses
would be sent to them. Please let me know if you're interested in such
an offer.

Best regards,
Lubo

On Thu, Jul 17, 2008 at 5:01 PM, Vladimir Škarupelov <voviss@gmail.com> >>> wrote:
> Hello Lubomir,
>
> For profiling sip-communicator I used JProfiler and Eclipse TPTP
> profilers,
> the first is not free, but it is faster than the others I tried, the
> TPTP
> project is very handy for me, because I used to work with eclipse, but
> this
> profiler is slow on my computer - it requires a lot of RAM. I tried
> also
> tried to use hprof and netbeans profiler for it, but it was enough for
> me to
> use that two ones, there were a lot code to test. When testing the code
> I
> used old but always working System.currentTimeMillis() method, for
> getting
> the timings, and put a counter for objects invocations. The time of
> method
> executions were mostly the same in TPTP and JProfiler, moreover I
> compared
> them with currentTimeMillis(), of course the the time of
> currentTimeMillis
> was always a little bit smaller (up to 5%) than the profilers one. Only
> problem that way can be compared only methods that take more than one
> millisecond. I think I try also NetBeans for timing testing.
>
> About threads.
> Thanks for the pointing to the problem, I mostly tested cpu and memory
> executions of sip-comm. There is something really very weird with the
> threads: if you start chatting with somebody you get 4 new threads in
> waiting status, if you close your chatting window, the threads stay
> alive.
> And you get new 4 threads if you start the next conversation, and so on
> ...
>
> Memory leaks.
> Sip-communicator was working pretty well without heavy traffic at
> night, the
> maximum heap size was only 20 mb, every 3 hours (when the size had
> reached ~
> 20 mb) the garbage was collected and the heap size had dropped for 10
> mb.
> But when chatting the number of String and dependent on it char[] is
> becoming bigger. Also becoming more byte[] objects, it seems that the
> information is sent in bytes and not correctly garbage collected.
> If you see the objects in memory, you see there (memoryAnalysis.png),
> for
> example, big number of javax.swing.text.View[] objects, these objects
> are
> not directly invoked by sip-communicator, but with the technology that
> sip-communicator use (HTMLDocument for the View arrays, for example,
> see
> attached memoryAnalysis2.png).
> But anyway, I think I'll try the soft you sent to me.
>
> Thanks for your interest,
> Vladimir Škarupelov
>
> Lubomir Marinov wrote:
>
> Vladimir,
>
> Thank you for the detailed description!
>
> 1. Which profilers did you use in your work?
>
> Going by the mentioning of JProfiler, I presume I figure at least it
> was used. I personally haven't used it so I cannot comment on its
> accuracy on CPU profiling.
>
> However, I've used YourKit Java Profiler, NetBeans Profiler and
> Eclipse TPTP (well, a long time ago because it still doesn't run on
> Mac OS X) and I can certainly say that CPU profiles are different
> depending on the profiler in use. For example, I find YourKit Java
> Profiler to have a tendency to overly penalize frequently executed
> methods (it's a pity because I'm used to the way it presents heap
> dumps and I don't think NetBeans Profilers comes any closer with
> respect to usability) which in a way ruins the usefulness the
> hot-spot statistics. In that respect, I find NetBeans Profiler to be
> better at it.
>
> Anyway, I think we should be creating and comparing CPU profiles in
> more than one profiler in order to get a more fine-grained sense as to
> where optimizations can and are worthy to be made. (Of course, I
> presume the major bottlenecks are obvious regardless of the profiler
> in use and we want to move forward.)
>
> 2. What about threads?
>
> I often look at the number of threads in the running instance of SIP
> Communicator and I'm totally puzzled why would a messenger with a
> couple of registered accounts need more than 40 threads (I have 50 at
> the time of this writing). For example, I'm using the Transmission
> BitTorrent client on Mac OS X at the moment to download 3 torrents and
> it has only 12 threads. (I'm citing the current numbers but the
> average observations are close because I often look at these.)
>
> When optimizing based on CPU profiles, I'd say one needs the
> statistics about which threads consume the precious time in order to:
> - focus on optimizations that would yield better perceived execution
> times,
> and
> - suggest implementation and/or architecture changes if local method
> optimizations aren't enough.
>
> Have you been able to observe these and draw conclusions from them?
>
> 3. And the memory?
>
> While I see you've touched on the recommendations for reusing
> instances, I didn't see an analysis on the object creation, class
> loading, etc.
>
> And if the above aren't of interest to you for general optimizations
> (and I think they should), are there memory leaks? I would personally
> recommend YourKit Java Profiler in the area and I've been able to get
> good return of effort from SAP/Eclipse Memory Analyzer
> (http://www.eclipse.org/mat/).
>
> Best regards,
> Lubo
>
> On Thu, Jul 17, 2008 at 2:11 PM, Vladimir Škarupelov <voviss@gmail.com> >>> > wrote:
>
>
> Hello all,
>
> In a first part of GSOC I was dealing with profiling sip-communicator.
> I was
> looking for laggy slow places in sip-comm and tried to profile them and
> then
> optimize. The found problems from user perspective were:
> 1. slow sip-communicator loading
> 2. graphics slow loading
> 3. big chat history slow processing
>
> The starting up of the sip-communicator is the slowest part of it, it
> takes
> about 10 - 15 seconds. It uses apache felix for loading up the bundles,
> it
> takes up to 330 milliseconds for every bundle to start up. Other time
> for
> starting up is going to the drawing swing/awt components and loading up
> images. When profiling the functionality I just see Felix and JDK
> methods
> invocations, it is up profiling JDK and Felix, their methods are
> time-consumptive.
>
> Very similar situation was with big chat history slow processing. The
> slowest part is the text parsing with javax.swing.text.html.HTMLFactory
> (see
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit's inner
> class
> HTMLFactoryX). There is two ways of eliminating the problem:
> - wait for updated and more optimized java version
> - to profile jdk
> - to use other technology for text formating
>
>
> Then I just monitored the sip-communicator with a profile to search for
> anomalies (took the 10 most expensive methods). I found that
> processSmilies() method from ChattConversationPanel class class is also
> consumpitve with big texts, thist uses regular expressions for smilies
> finding and the regular expression is very complex for it.
>
> I noticed that net.java.sip.communicator.util.ScLogFormatter is also
> time-consumptive, it takes about 2-3% of sip-communicator execution.
> The
> problems are format(LogRecord record) and inferCaller(LogRecord record)
> methods. First is using DecimalFormat for formatting is required time
> to
> process and also sending the messages to the output. The second is
> using
> StackTraceElement objects and processings, that are time consumptive.
> These
> methods are made with help of JDK and those are not very effective.
>
> There are Thread.sleep's in sip-comm code which are look like
> performance
> problems in profiler (look in
> net.java.sip.communicator.plugin.statusupdate.StatusUpdateActivator or
> just
> search for .sleep in sip-comm's source, you will found some), is the
> value
> of the sleeping time correct ?
>
> There are minor time problems that are also connected with libraries
> that
> are slow. I attach a profiler's report if you want to see all the
> problems,
> it was made by means of JProfiler.
>
> Notes for developers.
> What I can suggest to community is to continue adhere design patterns.
> Also
> quite effective is not to return a new created object, but reuse an
> already
> existing one - a creation of an object take more time and makes the
> heap
> larger. Of course it is not often possible, for example in
> net.java.sip.communicator.impl.gui.utils.SIPCommHTMLEditorKit inner
> class
> HTMLFactoryX.
>
> static class HTMLFactoryX extends HTMLFactory
> implements ViewFactory {
>
> public View create(Element elem) {
>
> View v = super.create(elem);
>
> if(v instanceof ImageView){
> return new SIPCommImageView(elem);
> }
> else if (v instanceof ParagraphView) {
> return new ParagraphViewX(elem);
> }
> return v;
> }
> }
>
> Namely View is created by create(Element element) method from
> javax.swing.text.html.HTMLFactory class, the Element object is
> necessary for
> the new instance of view , and there is no possible to clean up and
> return
> existing View class.
>
> But if we see the createDefaultDocument() in c class, we see that
> HTMLDocument doc variable can be reused - there is no in parameter for
> the
> object:
>
> public Document createDefaultDocument() {
> if(doc != null) return doc;
> StyleSheet styles = getStyleSheet();
> StyleSheet ss = new StyleSheet();
>
> ss.addStyleSheet(styles);
>
> doc = new HTMLDocument(ss);
> doc.setParser(getParser());
> doc.setAsynchronousLoadPriority(4);
> doc.setTokenThreshold(100);
> return doc;
> }
>
> Also I reused the creation of new SIPCommHTMLEditorKit classes in
> ChatConversationPanel. Actually the fix is not very helpful, because
> the
> method createDefaultDocument() is called only once from
> ChatConversationPanel but the same logic you can use in your projects.
>
> Cheers,
> Vladimir Škarupelov
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net