Log Based Monitoring Approach

@Magic_V that sounds great! If you need a hand with it, let me know.

@Sudha_Singhal Can you send the output from this
docker logs "$(docker ps | grep jicofo | cut -c 1-12)" 0>1

@MarcL Super, thanks for thinking along!

@MarcL It’s pretty huge log file. I should not paste that huge logs here.

@Sudha_Singhal You can upload it as a file. Or at least the last 10 lines.

@MarcL Here is the log file

logs.txt (9.2 KB)

@MarcL I’m also geeting this error
No such file or directory
image

Just sent you a download link for you to see our jicofo.log.total file, @Magic_V.
Thank you.

@Sudha_Singhal
Did you clone this repo? It’s currently not working with Magic_V’s work

@MarcL Yes, I cloned this repository which you’ve mentioned.

I deleted and again cloned your repo but the error is same.

@Sudha_Singhal Ouh yes, that was my fault. Fix is pushed. Give it a try.

Working now. Thanks.

@combnum
It appears that for a few of the rooms there is no closing event in the log. This could be for a many reasons, but most probably because of a server reset or some other error. The rooms were disposed of when the server closed down, but there was no message posted to the jicofo log. I’ll try to come up with a way to deal with situations like this, maybe I can detect if there has been a system startup or something (Java entries in the log).

In the meantime if you want to solve the problem right now, I would simply backup your logs if you want to keep them and delete your current log history. The program can’t report what isn’t there and should show no rooms open until you open new ones.

Alright @Magic_V, I deleted the logs.
Now, I can see this message coming back:
updating...
cat: /var/log/jitsi/jicofo.log: No such file or directory

And for sure, jicofo.log won’t exist until a new room is created and used.

I would like to thank you - it works great on my vServer. I had some concerns about the update frequency and CPU usage but at the moment i see no problems.

Is it possible to extent the script so that i can see the rooms created and the participant count for a given time interval - lets say the last day, week or month?

That would be really great.

@combnum
Yes this seems to makes sense, I remember having deleted them once and restart the server for the log file to be created again. I wouldn’t recommend deleting the jicofo.log file unless necessary btw, just the other jicofo.log.* files. The logsnatch script combines them all together.

@LuckyGeorge75
Yes, I had the exact same concerns at first, and if I had the time I would not have written it in a slow language like bash to begin with. And there is a lot of performance to be gained from improving the logsnatch script to only pull in truncated log files when they have changed. But it seems to work sufficiently fast to be useful, and the update interval could be adjusted in the main script by setting the 2 sleep parameters, (I’ll set a reminder to make that a bit easier to change through a variable). Sadly the quit button event is linked to the speed of the update so you might have to wait 5 seconds till the app closes.

Agreed it would be useful to be able to adjust the time period, not sure yet about the best way to go about it (code logic vs trimming the log.total file into a second file). Maybe both.

Hello, I am new here… :slight_smile:

Firstly, many thanks for this wonderful idea to get information about existing rooms from the logfile!

Inspired by this I’ve written a little C program for this. To increase performance further I thought it would be best not to examine the logs after they have been written but to chain-in a program which does linux-style filtering.

JitsiSoftware -> ROOMLISTPROGRAM -> jicofo.log

How to do this? One possible way would be to delete jicofo.log and create a named pipe instead:

mkfifo jicofo.log

Then start the ROOMLISTPROGRAM and feed it from the pipe. The program itself would continuously read from the pipe and write the log into a regular file.

roomlistprogram <jicofo.log >>jicofologfile.log

While just copying stdin to stdout this program would also inspect the stream and write the room list into a file.

However, I am puzzled because Jicofo seems to do some kind of “log rotation”. this might conflict with the “pipe idea”…
What do you think – is it worth a try? How does the log rotation work? Can it be deactivated?

Well – after one night of lots of coding, it seems to work very good.

The named-pipe thing runs smoothly and the room list is up to date at any time. No polling necessary. Example:

Jitsi Room list (creation_time room_name [owner] member_count)
2020-04-23_14:02:10 room1 [me] 2
2020-04-23_14:05:58 someroom [ben] 5
2020-04-23_14:09:25 nowhere [ben] 3

I’ve added a room log file which keeps a room history. This file is updated each time a room is closed (and therefore no longer on the room list). Example:

2020-04-23_13:50:33 2020-04-23_13:41:42 someroom [me] 2
2020-04-23_14:20:01 2020-04-23_14:02:10 room1 [me] 4
2020-04-23_14:20:11 2020-04-23_14:05:58 someroom [ben] 5
2020-04-23_14:20:17 2020-04-23_14:09:25 nowhere [ben] 3

(termination time, creation time, room name, owner name, max number of participants)

Both files can be made accessible via www – secured by .htpasswd if you want.

Edit: Well, I seem to be too stupid to do this. Jitsi catches any page request, thus my web page at /var/www/html/status/… is ignored.

2nd edit: Got it. Works fine.

2 Likes

@Marqqs Any plans to open source the script?

Sure. Currently testing on a local virtual machine. I guess my program might still be buggy. Are you interested in helping to test it? Do you have some Linux background experience? That would be very welcome, of course.

If not, perhaps Magic_V’s script is better suited for your purpose. It is older and probably more reliable (see link to Gitlab above).

2 Likes

Good news: Test was successful.
Room List program is now running on a live server. Seems to work so far. :smile: