Logging Conversations

I have setup the server to log conversations between users in a corporate environment. But the log file has a lot of stuff that I really don’'t need and it is difficult to navigate when looking for a specific item. Is there an app I can use to read the file easier or a plugin I can use to save them as a plain text conversation?

ItsNewToYou

I’‘d also be interested in a feature like this if it isn’‘t available. I’'m doing some searches for an XML parser app that I can use in the meantime.

There’‘s been some discussion before about how to improve logging and auditing (here’‘s one url=http://www.jivesoftware.org/forums/thread.jspa?threadID=14432thread[/url] in particular) but I don’'t believe anyone is actively working on anything.

Hope that helps,

Ryan

Hey ItsNewToYou,

There aren’‘t any tools for it yet, but we do hope to add some ways to view/search it in the admin console in a future release. Meanwhile, if you are logging stuff that you don’'t need you can only check to audit Message packets and leave unchecked IQ and Presence packets in the Admin Console.

Regards,

– Gato

I do have it set to only “Audit Message Packets” but there is still a ton of stuff you just don’'t need. So what I found and am using is a proxy called JabLogger and it looks like it is doing just what I need it to so far.

http://www.michaylov.com/html/jablogger.html

ItsNewToYou

Well after trying this approach it worked fine for awhile. But after about 30 people were logged into the server the connection would be lost after they send an IM to someone. So I had to revert back to the old way of doing things.

ItsNewToYou

I’'ve been doing a little work on this.

I’‘ve put together a small Perl script that parses the jive.audit log files and creates dated log files from them (default is the current date but an offset in days can be passed in). This could be run as a cron job (or scheduled task) early in the morning, specifying an offset of -1 to grab all the previous day’‘s traffic. All log files are parsed as there is no accounting for preferred rollover file sizes and also depending on heavy usage, a single day’'s traffic could span several files. The original log files remain intact.

I have submitted a feature request to have the facility to specify log rollover by time rather than by file size and Matt has kindly filed it as JM-298, so the Perl script would eventually become redundant.

I then have a program in C# - .NET but I’‘m sure it can be ported to Mono, (I’‘m all over the place with languages , except Java ) that parses the dated log file, although it can quite happily parse the original audit file as the only difference being the dated log files only contain message packets for a specific date. I’'ve not tested very large files.

Messages are presented in the form of a list, giving date, sender, message type (chat or message), subject and first 20 words of message body. Double clicking will provide the whole message. This list can then be filtered by senders and message type, i.e. being able to focus in on a chat, or just view all messages sent by a particular user.

As message packets are logged twice, ie. from the sender’‘s stream and also to the recipient’'s stream, these are ignored based on sender, subject (if one exists), and message body. Recipient is ignored to allow for broadcast messages.

I could post the code if anyone is interested, but bear in mind that I tend to take extreme programming to extremes and just omit design completely . Currently there are no configuration options so lots of stuff is hard coded, but I could fix that before posting. Also none this is internationalised in terms of date/time formats, but again, date formatting could be set as an option.

Things to do:

Stuff this all into a database to allow for easy searching, and add a liberal sprinkling of nice icons.

Cheers,

Jason.

Jason,

Sounds cool. I’'ve given you permission to post attachments in case you want to attach your app or screenshots.

Regards,

Matt

Hi,

I’'ve attached a png image of the log viewer itself to whet your appetites. It shows the main points of the application. Appearance is a little bland at the moment. No screen shot for the Perl script - no output on stdout (hopefully none on stderr either!).

The context menu - Focus this type lets you easily focus in on a chat or a series of messages between two users, without having to set the filter via the filter window. Focus all types selects all types of message between the two users. The last one is handy here as lots of users still haven’'t got to grips with the difference between chat and message. The Filter window allows you to select from all available users (that are listed in the log file) and view various types of message between them.

For the list of users, the application strips off the domain section of the JID and assumes that if the first portion has a . in it then it is Firstname.Surname and uses proper case on the username (only for display - the original message is stored intact).

Currently, the application only reads the dated log file, however, I will get around to enabling this to be entered into a database.

I’'ll do a little work on putting some of the hard coded settings in the log viewer into some sort of config file to make things easier. Hopefully being able to post the code in the next few days.

Sorry, no smilies in the message window , not yet at least. I’'m using a RichText control and displaying images is a little involved if you want to avoid using the clipboard.

I have also attached the Perl script that splits the audit logs into dated files. This should work on both Windows and Linux, the path separator really being the only difference.

The Perl script can take 4 options:

-o specify an offset in days previous to the current day. I.e -o “-1” for yesterday

Default is 0 i.e. today. I suppose that means you can supply and offset for a future date, but that won’'t get you anywhere

-O output location for the dated log file

Default is ./logs (I think this works on Windows regardless of the / separator)

-l location of the jive audit files

Default is /opt/jive_messenger/logs (no idea where the default is on Windows)

-f a template for the audit file

Default is jive.audit-*.log. The script then ‘‘globs’’ all matching files.

The output log file is named yyyy-mm-dd.log and is in the same format as the original, just containing the message packets from the specified date.

The current BIG assumption in the Perl script is that the message packet dates are in the form: Fri Jun 02 09:35:00 BST 2005, with BST and GMT being used depending on daylight savings being in effect. I’'m sure someone can point out the error of my ways here.

Actually, the Log viewer makes this assumption as well (as that is the format our logs take).

Hope this helps a few people.

Cheers,

Jason.

Message was edited by:

jasonmcclean

Ok, turned out there weren’'t that many hard coded settings

I’'ve attached the C# source code. You will need, at least, the .NET v1.1 SDK to compile. Much easier if you have something like Visual Studio .Net, or what I use, a GPL .NET IDE called SharpDevelop (http://www.icsharpcode.net/OpenSource/SD/Default.aspx).

Create a new project, add in all the source files. The file called Log Viewer.exe.config should be placed in the output folder and renamed to match the executable that you produce, i.e. yourApp.exe.config. This file contains the broadcast address, your domain (not used at moment), date/time format for display and the window caption.

I can post the executable if you wish, but you will need the .NET v1.1 runtime to run it.

The whole thing needs a good polish, but it works. Let me know when you find all the bugs.

EDIT

Just checking and noticed something. If you are going to use it on the actual log files produced by Jive messenger, use a copy. This is because in the current log file, the closing tag as not yet been added. The log viewer checks for this and adds it if it can’‘t be found. This avoids errors being thrown from the Xml classes being used. Not sure what impact that would have if Jive Messenger found the closing tag when it wasn’'t supposed to be there, unless it just blindly appends to the file. However, you would then still end up with an invalid XML file at the end due to two closing tags being present.

Sorry

End of edit

Cheers,

Jason.

Message was edited by:

jasonmcclean

edit

source added to message towards end

Any feedback?

I’'ve added support for emoticons and you can easily recreate psuedo chats from either chats or related messages.

I’‘ll update the previous post with new code if anyone interested. If not, nevermind, it’'s doing what I need

Jason.

Added image of chat window created from the logs. (couldn’‘t add it to previous post as it didn’'t originally have an attachment).

I’'ll stop now.

Jason.

Jason,

It’'s starting to look very cool! Would you be interested in doing binary releases so that people can more easily use the software?

Regards,

Matt

Looks really nice, yes!

Just tried to compile it on Mono, but in spite of a (citing the website) 98% complete implementation of System.Windows.Forms it’'s still missing the RichTextBox… sigh

“Major areas still needing work are DataGrid, RichText, MDI and Printing.”

Would be very cool to have your tool on a cross-platform framework…

Regards,

Ben

Would you be

interested in doing binary releases so that people

can more easily use the software?

Not a problem. As long as .NET v1.1 runtime is installed, there shouldn’'t be a problem.

Have attached binary.

Emoticon support was added by using the RichEdit control by Khendys Gordon (http://www.codeproject.com/cs/miscctrl/csexrichtextbox.asp)

The emoticons are taken from the Psi client (http://psi.affinix.com) as that is the client we use - using the same icondef.xml as Psi uses. That means any icon set with a corresponding icondef.xml can be used.

Just extract and run.

edit

Oh, edit the Log Viewer.exe.config file first with your broadcast address to pick up broadcast messages. Will work without, but you won’'t see broadcasts marked as such, just the first recipient message.

Jason.

Message was edited by:

jasonmcclean

edit

Seem to have lost the binary. Reattached to a message towards the end.

Looks really nice, yes!

Just tried to compile it on Mono, but in spite of a

(citing the website) 98% complete implementation of

System.Windows.Forms it’'s still missing the

RichTextBox… sigh

"Major areas still needing work are DataGrid,

RichText, MDI and Printing."

Would be very cool to have your tool on a

cross-platform framework…

Haven’'t played with Mono yet (forgot about Windows.Forms Doh!). Maybe soon

edit

don’‘t see a reason why it can’'t be re-implemented using something other than C#. That just happens to be my language of the moment.

Jason

Message was edited by:

jasonmcclean

what about *nix?

i mean bin

Haven’'t played with Mono yet (forgot about

Windows.Forms Doh!). Maybe soon

It’'s currently “in the pipeline” (Talking about RichTextBox here). But the next problem would be, that you are using the Codecentral approach of inserting icons, right? With P/Invoke on gdi/windows dlls…

edit

don’‘t see a reason why it can’'t be re-implemented

using something other than C#. That just happens to

be my language of the moment.

I do like C# as well (currently even better than Java sometimes). It certainly could be implemented cross-platform with C# / gtk# or lots of other languages of course (Well - Java would be quite a good idea for a java IM server… ponder)

wroot: No idea what you mean, sorry?

wroot: No idea what you mean, sorry?

nevermind. i guess i’'ll go to bed right now, this is the best solution so far for me:)

It’'s currently “in the pipeline” (Talking about

RichTextBox here). But the next problem would be,

that you are using the Codecentral approach of

inserting icons, right? With P/Invoke on gdi/windows

dlls…

The RichEdit control that I’'m using has extended the standard one by adding in functions to wrap up an image in a Windows MetaFile (WMF) and inserting directly into the control wrapped in RTF codes. If you looked at one of the icons in this state you would see the RTF control codes and the image represented in Hex.

The way I’‘m using it is I’'ve created a class to search for the emoticons using regexes then generate the RTF codes required and replacing in the text of the message. The message is then inserted into the RichText control as standard RTF text.

As RTF is a well known standard, as soon as Mono sort out the control, this should work (I think)