[jitsi-users] Issues developing plugin for generic USB-HID support


#1

For simplicity, the plugin currently just waits for a specific HID event
and translates it into an AWT key press using java.Robot.
(This can then be used with the global key shortcuts)

Interesting idea.

Thanks. This is far from optimal, but enough for a proof of concept.

Ideally it should eventually directly trigger the answer action
directly. Also for now I needs to be configured to work with just one
HID device. In fact, a list of supported HID devices could be put in the
resources and any attached device could be identified automatically, but
that's for the final version.

I guess these would all be minor issues if everything else would be
solved...

A little help would be appreciated, how to trigger a "pick-up"/"hang-up"
action. I already found some other plugin that uses a seperate XML
config file - I would use that trick as well for build in supported
devices. Possibly adding an additional user generated file that could be
submitted to the devs for integration, when another working device has
been found by some user. but anyway...

Well for calls, basically you'd need to go through OSGi services of type ProtocolProviderService, query them for an operation set OperationSetBasicTelephony and add a CallListener to that. This is how the shortcut work. In the end it might be best to somehow interface with the code that is currently handling the shortcuts as there's quite some code for different call states in there (cf. CallShortcut). I'm not yet sure how that could be handled best though.

I added the external libs to the ant build process of the plugin as
follows:

          <target name="bundle-plugin-usbhidmapper">
              <jar compress="false"
destfile="${bundles.dest}/usbhidmapper.jar"

manifest="${src}/net/java/sip/communicator/plugin/usbhidmapper/usbhidmapp
er.m anifest.mf">
                  <zipfileset
dir="${dest}/net/java/sip/communicator/plugin/usbhidmapper"

prefix="net/java/sip/communicator/plugin/usbhidmapper"/>
                  <zipfileset src="${lib.noinst}/hid4java-0.3.1.jar"/>
                  <zipfileset src="${lib.noinst}/jna-4.1.0.jar"/>
              </jar>
          </target>
This results in a 2.5 MB JAR, containing the external java libs, native
libs and bundle code. I can add this jar as a plugin to a stable version
install.

Ahm, this might work for testing purposes, but if we want to integrate
that, these should be separate three jars: - usbhidmapper (your work) -
jna (we already have that in Jitsi, might need to be updated though) -
hid4java (run "mvn package" to get an OSGi enabled jar)

How would I configure that then?
1. where to put the external jars

- lib/installer-exclude for the raw jars
- if they come with an OSGi manifest, simply copy them in your target to ${bundlest.dest}
- otherwise add the manifest to the jar and put the output to ${bundles.dest}

There are plenty of examples in the build.xml, e.g. bundle-hsql for the first and bundle-architectureviewer for the second.

2. how to reference them

Add an entry to felix.client.run.properties, jna and hid4java must come before yours. Level 67 seems appropriate. Jna is already there (level 40) as jnalib.jar.

3. I used some mvn assembly configuration to get the hid4java-0.3.1.jar

Check and see if the created jar contains a META-INF/MANIFEST.MF file that contains definition for Bundle-Name, Bundle-SymbolicName, Export-Package, etc.

There are 2 lib jars in the jitsi source tree. But the bundle ant target
creates a single jar with dependencies (as I would expect for a plugin).

This works and is well for private plugins. But we have to consider Linux packaging and there we cannot duplicate libraries and need to reference the ones from the system. Hid4java might not have a package (yet), but jna does and we must use that (or it will not be accepted by the distro maintainers).

I'm currently having some issues/questions though:

1.) Is the MIT license "OK" for you, so the plugin could possibly be
integrated with Jitsi?

Yes, but you'd need to sign a contributor agreement [1] as well.

We can do that, when it's stable enough.

2.) On Linux, the bundle does not get any stop events when shutting down
the application. On WinXP it does. (This is an issue, because I need to
stop the thread listening on the USB device). It works when I manually
deactivate the plugin before quitting jitsi.

There's a timeout in the shutdown until the process gets killed. You might
run into that on Linux. What's the problem when you don't manually stop that
thread? After all, the application is killed. Would that leaves some native
pointers/handles to the device open?

It looks like some processes of the hid4java lib are blocking the
shutdown. The lib echos the stderr that it's unloading and the listening
thread keeps running. It won't interrupt. I think that the listening
thread blocks the shutdown process. I need to "kill -9" the jitsi
process (a simple kill is not sufficient). Can I somehow intercept the
shutdown request to manually clean up "my" things, i case the bundles
stop() method is not called?

There's a "kill-service" on Windows, but it isn't bundled on the other platforms as it wasn't needed there.

Or is there any service i can query for the
applications status? Then the listening thread itself could check
whether it should terminate, without beeing interrupted.

Did you set your thread as a daemon?

3.) On Windows, the external lib does not detect USB devices, when
running with jitsi. The plugin and external lib get loaded without
errors though. If I use the same created plugin JAR with a simple test
program by adding it to the classpath, it works. It might be due to some
strange OSGI/felix problem, but I have no clue on where to start looking.

What do you mean by "gets loaded"? Can you obtain at least some information
from hid4java? There could also be a problem from the two jna libraries,
although that is not very likely due to the OSGi isolation.

The library does not complain about anything missing. It just does not
detect any HID devices (I have a Wingman Extreme attached as well to
test device detection). Using the same lib (i.e. the plugin jar
including the external libs) with a simple test program works.

This might have something to do with the JNA library modifying the (java|jna).library.path and not succeeding under OSGi. Has the jna.jar an OSGi-Manifest specifying some native library locations? They could get lost with your packaging task.

On WinXP I'm using Jitsi 2.8.5426 with java 1.7.0_71 (oracle)
On Linux I'm using Jitsi 2.8.5426 with java 1.8.0_40 x64 (oracle)

Is your source somewhere public to take a look at?

I'm currently rebasing/merging to your current master branch. I will
send you a patch and the test program to demonstrate the issues.

A link to you github branch would be easier than a patch file on the list here.

Regards,
Peter

Ingo

[1] http://bluejimp.com/bca.pdf

Ingo

···

On 2015-04-08 15:55, Ingo Bauersachs wrote:


#2

A little help would be appreciated, how to trigger a "pick-up"/"hang-up"
action. I already found some other plugin that uses a seperate XML
config file - I would use that trick as well for build in supported
devices. Possibly adding an additional user generated file that could be
submitted to the devs for integration, when another working device has
been found by some user. but anyway...

Well for calls, basically you'd need to go through OSGi services of type ProtocolProviderService, query them for an operation set OperationSetBasicTelephony and add a CallListener to that. This is how the shortcut work. In the end it might be best to somehow interface with the code that is currently handling the shortcuts as there's quite some code for different call states in there (cf. CallShortcut). I'm not yet sure how that could be handled best though.

I've had a look on the code you mentioned. I've seen that there is some code duplication along the plugins, concerning answering/hanging up calls. Wouldn't it be sufficient to just call
net.java.sip.communicator.impl.gui.main.call.CallManager.answerCall(Call)
and the corresponding hangup method?
(btw. that class should be split up in a GUI part handling dialogues and a backend part, that could offer functionality that is (also) found in the CallShortcut).

Ahm, this might work for testing purposes, but if we want to integrate
that, these should be separate three jars:

>>> - usbhidmapper (your work)

- jna (we already have that in Jitsi, might need to be updated though) -
hid4java (run "mvn package" to get an OSGi enabled jar)

How would I configure that then?
1. where to put the external jars

- lib/installer-exclude for the raw jars
- if they come with an OSGi manifest, simply copy them in your target to ${bundlest.dest}
- otherwise add the manifest to the jar and put the output to ${bundles.dest}

There are plenty of examples in the build.xml, e.g. bundle-hsql for the first and bundle-architectureviewer for the second.

Ok, but then it is not bundled with the plugin, so before I can install the (my) plugin, I need to install the dependencies first - right?

I managed to build it and add the manifest data - see the ant target:

<!-- BUNDLE-PLUGIN-USBHIDMAPPER PLUGIN -->
<target name="bundle-plugin-usbhidmapper">
   <jar compress="false" destfile="${bundles.dest}/usbhidmapper.jar"
manifest="${src}/net/java/sip/communicator/plugin/usbhidmapper/usbhidmapper.manifest.mf">
     <zipfileset dir="${dest}/net/java/sip/communicator/plugin/usbhidmapper"
  prefix="net/java/sip/communicator/plugin/usbhidmapper"/>
   </jar>

   <!-- just copy jna-4.1.0 as it has an OSGI manifest -->
   <copy file="${lib.noinst}/jna-4.1.0.jar" tofile="${bundles.dest}/jna-4.1.0.jar"/>

   <!-- add OSGI manifest to hid4java -->
   <jar compress="false" destfile="${bundles.dest}/hid4java.jar"
       filesetmanifest="merge">

     <zipfileset src="${lib.noinst}/hid4java-0.3.1.jar"/>
     <manifest>
  
       <attribute name="Bundle-Name" value="hid4java" />
       <attribute name="Bundle-Description" value="A cross-platform Java Native Access (JNA) wrapper for the signal11/hidapi library." />
       <attribute name="Bundle-Vendor" value="https://github.com/gary-rowe/hid4java" />
       <attribute name="Bundle-Version" value="0.3.1"/>
       <attribute name="Bundle-SymbolicName" value="org.hid4java"/>
       <attribute name="Export-Package" value="org.hid4java,
    org.hid4java.event,
    org.hid4java.jna"/>
       <attribute name="Import-Package"
    value="com.sun.jna"/>
       <attribute name="Bundle-NativeCode" value="
    win32-x86/hidapi.dll; processor=x86;osname=win32,
    win32-x86-64/hidapi.dll; processor=x86-64;osname=win32,
    win32-amd64/hidapi.dll; processor=amd64;osname=win32,
    linux-x86/libhidapi.so; processor=x86;osname=linux,
    linux-x86-64/libhidapi.so; processor=x86-64;osname=linux,
    linux-amd64/libhidapi.so; processor=amd64;osname=linux,
    darwin/libhidapi.dylib; osname=macosx;processor=x86;processor=x86-64;processor=ppc" />
     </manifest>
   </jar>
</target>

2. how to reference them

Add an entry to felix.client.run.properties, jna and hid4java must come before yours. Level 67 seems appropriate. Jna is already there (level 40) as jnalib.jar.

Did so, and it seems like it's almost working. One Problem is now, that when loading the plugin, it complains about not beeing able to load the native library that is bundled with hid4java.
It's likely due to the manifest data in the build file, but I don't know what might be wrong.

The error is:
java.lang.UnsatisfiedLinkError: Unable to load library 'hidapi': libhidapi.so: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden
(English error message is: "Unable to load library 'hidapi': libhidapi.so: Can't open shared object file: file or directory not found)
  at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:163)
  at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:236)
  at com.sun.jna.Library$Handler.<init>(Library.java:140)
  at com.sun.jna.Native.loadLibrary(Native.java:379)
  at com.sun.jna.Native.loadLibrary(Native.java:364)
  at org.hid4java.jna.HidApiLibrary.<clinit>(HidApiLibrary.java:27)
  at org.hid4java.jna.HidApi.<clinit>(HidApi.java:36)
  at net.java.sip.communicator.plugin.usbhidmapper.USBHIDMapperPluginActivator.start(USBHIDMapperPluginActivator.java:77)
...

As you can see, my code is executed, the hid4java code is executed, jna is used to load the library, but then it fails.
Any suggestions how to tell him the location of the native library?
I think, it is due to the wrong version of JNA (see below).

3. I used some mvn assembly configuration to get the hid4java-0.3.1.jar

Check and see if the created jar contains a META-INF/MANIFEST.MF file that contains definition for Bundle-Name, Bundle-SymbolicName, Export-Package, etc.

No, it has no OSGI info, so I added it manually - see above.

There are 2 lib jars in the jitsi source tree. But the bundle ant target
creates a single jar with dependencies (as I would expect for a plugin).

This works and is well for private plugins. But we have to consider Linux packaging and there we cannot duplicate libraries and need to reference the ones from the system. Hid4java might not have a package (yet), but jna does and we must use that (or it will not be accepted by the distro maintainers).

As you can see, I use JNA 4.1.0.
I tried to run the whole bundle with your jna version, which leads to the above error. When I include JNA 4.1.0 with my bundle (as is did earlier) or include it with the hid4java bundle, it finds the native library, but I still have the problem on windows, that no devices are detected.
Could I somehow upgrade the JNA version of jitsi? (at least for testing to see if that helps) Or force a bundle to use a specific version of it?

I will share my code on github asap.

Thanks so far and kind regards,

Peter

···

On 2015-04-08 17:44, Ingo Bauersachs wrote: