Trace user activity on external systems

Hello everyone,

I need to trace every user activity, ideally I would like to keep records of events like

  • user connected
  • user disconnected
  • room created
  • room destroyed

I saw from several old forum posts (like this one or this one)
that suggest writing his own prosody module (by hooking the events of the muc) or go trough the JICOFO logs (I would avoid the latter).

The posts are a bit old and I was wondering if meanwhile something has been made(I would prefer a REST integration, but I’m open to other options).

Did someone already approached this problem?

Writing a new module is the only solution? If this is really the only solution, could someone elaborate on how to do this with some starting tips (I’m a beginner with lua - ex. rest calls, serializing a table into a json, etc…)?

3 Likes

Jicofo log analyzer

I would really avoid parsing the jicofo logs with a bash script: It seems like it’s not a solution easy to maintain (ex. we might need to parse a potentially very long file, not talk about file locks, permissions and stuff like that) and the results would not arrive in real time.

you can enable syslog for jicofo

--- logging.properties.ori	2021-07-08 15:33:12.176664889 +0200
+++ logging.properties	2021-07-08 15:33:28.136134017 +0200
@@ -5,7 +5,7 @@
 #handlers= java.util.logging.ConsoleHandler, org.jitsi.impl.protocol.xmpp.log.XmppPacketsFileHandler
 
 # Handlers with syslog enabled:
-#handlers= java.util.logging.ConsoleHandler, com.agafua.syslog.SyslogHandler
+handlers= java.util.logging.ConsoleHandler, com.agafua.syslog.SyslogHandler
 #handlers= java.util.logging.ConsoleHandler, io.sentry.jul.SentryHandler
 
 java.util.logging.ConsoleHandler.level = ALL
@@ -25,7 +25,7 @@
 # Syslog (uncomment handler to use)
 com.agafua.syslog.SyslogHandler.transport = udp
 com.agafua.syslog.SyslogHandler.facility = local0
-com.agafua.syslog.SyslogHandler.port = 514
+com.agafua.syslog.SyslogHandler.port = 10514
 com.agafua.syslog.SyslogHandler.hostname = localhost
 com.agafua.syslog.SyslogHandler.formatter = org.jitsi.utils.logging2.JitsiLogFormatter
 com.agafua.syslog.SyslogHandler.escapeNewlines = false

then
nc -l 10514 -u

and
sudo systemctl restart jicofo
will show you the way.

Then replacing nc by some custom app should need less than 250 LOC in a high level language and maybe 3 days of dev ?

IMO using jicofo logs is better than Prosody as I think there is no way to get at the nicknames picked by users with a Prosody handler. If it’s not needed however, Prosody could be used.

I already explained why I think it’s unmainteinable and a road that cannot be used. We need to have data in real time.

IMO using jicofo logs is better than Prosody as I think there is no way to get at the nicknames picked by users with a Prosody handler. If it’s not needed however, Prosody could be used.

doesn’t jicofo receive data from Prosody? Therefore whatever is available on on jicofo should be available on Prosody.

If there was some way to instruct jicofo to send the logs through REST or some other integration method that would be great

Then replacing nc by some custom app should need less than 250 LOC in a high level language and maybe 3 days of dev ?

I am talking of a SERIOUS real time integration with a communication protocol (rest, websockests, grpc, etc…) with an EXTERNAL system that is already in place and error handling. I don’t want to read a volatile file or the system logs. I don’t want to deal with quick solutions that will create more troubles than the one they are resolving (which seems the case)

If I did not understand your need, too bad. Good luck.

Update: The plugin has been ported to GitHub - jitsi-contrib/prosody-plugins: Prosody plugins for Jitsi. Please use the one there rather than the version in the gist below.


If it is a prosody plugin you’re after, here is an example of a prosody plugin that I hastily adapted from an internal POC:

Caveats

  1. I’ve only tested this with users authenticated using JWT (no guest users)
  2. Prosody is single threaded, so firing lots of POST requests to an external endpoint on each event could cause performance issues on busy servers

What it does

Sends POST request with JSON payload to external API on occupant and room events.

  • muc-room-createdPOST $API_PREFIX/events/room/created
    • payload contains
      • room_name
      • room_jid
      • created_at
  • muc-room-destroyedPOST $API_PREFIX/events/room/destroyed
    • payload contains
      • room_name
      • room_jid
      • created_at
      • destroyed_at
      • all_occupants (list of occupant data)
  • muc-occupant-joinedPOST $API_PREFIX/events/occupant/joined
    • payload contains
      • room_name
      • room_jid
      • occupant (occupant data)
  • muc-occupant-leftPOST $API_PREFIX/events/occupant/left
    * room_name
    * room_jid
    * occupant (occupant data)

Occupant data referenced above contains:

  • occupant_jid
  • name (user.context.name from JWT token)
  • email (user.context.email from JWT token)
  • id (user.context.id from JWT token)
  • joined_at
  • left_at

Example payload

{
  "event": "muc-occupant-joined",  
  "room_name": "[tenantx]00380200-d3ce-42dd-b1d6-c09f4f0da7ee",
  "room_jid": "[tenantx]00380200-d3ce-42dd-b1d6-c09f4f0da7ee@conference.comain.com",
  "occupant": {
    "name": "James Barrow",
    "id": "00380324-a840-400d-880f-7ee0933b7556",
    "joined_at": 1625771282,
    "occupant_jid": "6ad6ac18-a3d9-492e-96e4-fbf943633709@domain.com/7fE491d4"
  }
}
{
  "event": "muc-room-destroyed",  
  "all_occupants": [
    {
      "left_at": 1625771295,
      "joined_at": 1625771282,
      "name": "James Barrow",
      "id": "00380324-a840-400d-880f-7ee0933b7556",
      "occupant_jid": "6ad6ac18-a3d9-492e-96e4-fbf943633709@domain.com/7fE491d4"
    }
  ],
  "room_jid": "[tenantx]00380200-d3ce-42dd-b1d6-c09f4f0da7ee@conference.domain.copm",
  "created_at": 1625771281,
  "room_name": "[tenantx]00380200-d3ce-42dd-b1d6-c09f4f0da7ee",
  "destroyed_at": 1625771295
}

Installation

  1. Copy file to /usr/share/jitsi-meet/prosody-plugins
  2. Add the component in /etc/prosody/conf.d/<DOMAIN>.cfg.lua
    Component "event_sync.<DOMAIN>" "event_sync_component"
       muc_component = "conference.<DOMAIN>"
    
       api_prefix = "http://your.api.server/api"
    
  3. Restart prosody

See comments in code for more optional config entries, e.g. to add custom header for API auth.


Hopefully this will give you a head start.

Good luck!

4 Likes

This looks as good candidate for Introducing the jitsi-contrib GitHub organization

:+1: I’ll see if I get a chance to clean it up over the weekend and submit a PR.

Did a quick test and the component appears to work. Issue raised on jitsi-contrib/meta to propose an entry :crossed_fingers:t4:

Thank you! I will try the solution in the weekend and let you know.

It would be great to have an official integration that we could use!

Is it possible to do it with asyc programming to solve this problem?

The net.http lib (https://prosody.im/doc/developers/net/http) used for the POST calls is already doing non-blocking calls so doesn’t block the process. But prosody is still inherently single-threaded, so having lots of async calls in flight will still have an impact on performance. I have not benchmarked this though, so cannot comment on what constitutes “a lot” and how noticeable the impact would be.

1 Like

Thank you,
your POC was really what I was looking for, it’s great to receive such an, unexpectedly complete, snippet. It’s really awesome to receive such a quick and kind community help!

I will test it, if it’s ok configure it a little and then (1/2 weeks) I might do some load testing. I will let you know :slight_smile:

1 Like

@Andrea_Montanari The plugin is now on GitHub - jitsi-contrib/prosody-plugins: Prosody plugins for Jitsi. Probably better to use the one there rather than the version I posted above.

I’ve also updated my answer above so other readers won’t inadvertently use the old version.

Anyway, I look forward to hearing how it fares in your load testing :slight_smile:

2 Likes

How we can tell who was the moderator in that conference

You’ll need to modify the plugin to add that information to stored occupant data.

I’m guessing you can infer that from event.occupant.role, and if you use JWT to set mod status, then you also have access to JWT user context from event.origin.jitsi_meet_context_user.

event.origin.jitsi_meet_context_user
is working for me thanks