Log Based Monitoring Approach

Hello, I am new here… :slight_smile:

Firstly, many thanks for this wonderful idea to get information about existing rooms from the logfile!

Inspired by this I’ve written a little C program for this. To increase performance further I thought it would be best not to examine the logs after they have been written but to chain-in a program which does linux-style filtering.

JitsiSoftware -> ROOMLISTPROGRAM -> jicofo.log

How to do this? One possible way would be to delete jicofo.log and create a named pipe instead:

mkfifo jicofo.log

Then start the ROOMLISTPROGRAM and feed it from the pipe. The program itself would continuously read from the pipe and write the log into a regular file.

roomlistprogram <jicofo.log >>jicofologfile.log

While just copying stdin to stdout this program would also inspect the stream and write the room list into a file.

However, I am puzzled because Jicofo seems to do some kind of “log rotation”. this might conflict with the “pipe idea”…
What do you think – is it worth a try? How does the log rotation work? Can it be deactivated?

Well – after one night of lots of coding, it seems to work very good.

The named-pipe thing runs smoothly and the room list is up to date at any time. No polling necessary. Example:

Jitsi Room list (creation_time room_name [owner] member_count)
2020-04-23_14:02:10 room1 [me] 2
2020-04-23_14:05:58 someroom [ben] 5
2020-04-23_14:09:25 nowhere [ben] 3

I’ve added a room log file which keeps a room history. This file is updated each time a room is closed (and therefore no longer on the room list). Example:

2020-04-23_13:50:33 2020-04-23_13:41:42 someroom [me] 2
2020-04-23_14:20:01 2020-04-23_14:02:10 room1 [me] 4
2020-04-23_14:20:11 2020-04-23_14:05:58 someroom [ben] 5
2020-04-23_14:20:17 2020-04-23_14:09:25 nowhere [ben] 3

(termination time, creation time, room name, owner name, max number of participants)

Both files can be made accessible via www – secured by .htpasswd if you want.

Edit: Well, I seem to be too stupid to do this. Jitsi catches any page request, thus my web page at /var/www/html/status/… is ignored.

2nd edit: Got it. Works fine.

2 Likes

@Marqqs Any plans to open source the script?

Sure. Currently testing on a local virtual machine. I guess my program might still be buggy. Are you interested in helping to test it? Do you have some Linux background experience? That would be very welcome, of course.

If not, perhaps Magic_V’s script is better suited for your purpose. It is older and probably more reliable (see link to Gitlab above).

2 Likes

Good news: Test was successful.
Room List program is now running on a live server. Seems to work so far. :smile:

Can you share the script with us?

Of course. I’ve opened a new topic in this forum’s “Install & Config” section:

This does not seems to be working… has something been changed or am I missing something?

After cloning the repo sucessfully, when I type (with sudo): main -s meet.mydomain.com

it gives me an error:

Command ‘main’ not found, did you mean:

command ‘moin’ from deb python-moinmoin
command ‘maim’ from deb maim
command ‘cain’ from deb cain
command ‘mail’ from deb mailutils
command ‘amin’ from deb orville-write
command ‘rain’ from deb bsdgames
command ‘man’ from deb man-db

Try: apt install

I am relatively new to unix, so if you see any mistake at my end, pls feel free to point it out.

Many Thanks,
Rav

@raviktiwari
Nothing has been changed and all updates will be pushed to the development branch so the master branch is still the same it was 2 weeks ago.

I think what you should try first is to execute the main command as ./main instead of main. This is the way you execute programs/scripts from a Unix shell if you want to execute them from the same folder.

If you just type main it will look for the main script in your $PATH variable. If you want to see what folder are part of your $PATH variable (a list) you can type: echo $PATH and you can see where your shell searches for programs (usually /usr/bin or something like that. It is possible to add the main script to your global path but I would recommend against it and instead just navigate to the repo folder and execute from there.

Edit: I checked the Gitlab repo and have changed the lines in the README.md to include ./main instead of main to make things more clear.

I can confirm that it works as expected. Thanks!

Thanks for the prompt reply. I think this is the only group/forum in Jitsi community which is so prompt and resposnive - most of the jitsi groups are not so good (I am trying to be nice on the internet). :slight_smile:

Just so you know, I tried the command that you suggested and it DID work perfectly fine. Thanks a lot for taking out the time to make the necessary changes - really appreciate it.

However, may I ask a very basic question (which I believe someone has already asked) - can we modify the script to ask for a date range and then based on the input it can show how many meetings were held within that period and show the name of the room or user name. It will be better if we can capture the start&end time or just the duration of each meeting.

What do you think? Is it too much to ask for? Is there no point in making such script?

Many Thanks,
Rav

@raviktiwari
Good to hear you got it working!

And yes I’m working on implementing certain additional functionalities, I’m keeping track of them in the issues on the Gitlab project. Once finished I will push these updates to the development branch of the project. The thing is that I’m doing this as a little side project, and I have another big project I’m currently working on that requires my full attention. I have recently updated and cleaned up the logsnatch script, to make it a lot more efficient, this is already in the development branch. My next move will be to update the jitsihawk script. Currently there are the following changes planned:

  • ignore log files based on server restarts
  • use a config file for the regular expressions (separate the script logic from the log search queries) to make it easier for users to make changes in case the log file formatting changes
  • provide full host (authorized user) array per room instead of last host joined
  • work on the log analysis within time windows (perhaps using execution flags as well), if you have additional suggestions please message me and I’ll consider them
  • look into providing docker support using execution flag on main
  • clean up the code and documentation to make it more readable (and maintainable)

I’ll post to this thread on updates!

Hello @Magic_V,

Congratulations on this simple, yet effective solution!
I did not have the time to read through all the code, but I want to fix a little something… Maybe you will beat me to it! :stuck_out_tongue:

There is something unpleasant about the repainting of the list… It seems to be clearing the screen and relisting… Maybe there is a way to just update what is on screen? Like “atop” does… Or maybe the data is not buffered in RAM before “repainting” the screen and it appears too slow, making a weird flickering… I don’t know! Maybe you have an idea!

Also, I don’t think that having the last Host is useful
I would parse the name of the first host to have joined the room as it is probably the meeting’s host…

And also, it would be great if the first line gave the total number of participants after the total number of rooms… just a thought…

What do you think?

Works like a charm. Made a jitsi directory (sudo ssh) and made a subdirecroty lib. Put main script in jitsi. Put jitsihawk and logsnatch in jitsi/lib. I did chmod for main, jitsihawh and logsnatch to 755. I can now ssh into my server and see what is going on without needing to be root, but will need to be root to change anything.

Well, I didn’t consider the jicofo files in /var/lib being not readable to user. Still have to use sudo to run main.

quick question -

do we need to run the script on Jicofo instance ?

Yes, it uses the jicofo logs to gather data

thx

was wondering some how to capture that log data in grafana … any ideas ?