dele

Just over a year ago, I blogged about using audio and video with openfire. At that time, I implemented a SIP based softphone in Adobe Flash using AsteriskWin32 and VAC4. My argument for an open-source, standards based, no-install web-based softphone as a requirement for Web 2.0 voice applications is still valid today and the emergence of Ribbit, TringMe, Zingaya (Flashphone) and others confirmed my thinking was not isolated.

It has however been a disappointment that all implementations I have encountered to-date have been closed, proprietary and inaccessible for integration (both client and server).

The latest release of the Red5 Plugin for Openfire features a completely open-source implementation of a web-based SIP softphone written in Flex3 and should work on both Windows and Linux. It uses MjSIP as the SIP user agent in the plugin and should work with most SIP proxies, but I have only tested with Asterisk. I have also only tested 2 simultaneous users, but there is no limit and will depend on how many users and media streams Red5 can cope with before it dies. Each telephone conversation consumes 2 user RTMP connections and 4 audio streams on Red5. All source code is provided and you are free to use it in your Openfire Red5 Plugin applications. Just confirm that the open-source licenses of MjSIP and Nelly2PCM are to your liking.



I have also integrated the softphone into SparkWeb and the Openfire SIP plugin. This will enable a user to make SIP calls from Spark and SparkWeb with the same user profile. The old Red5gateway will be depreciated and in a later release for window users, I will be adding AsteriskWin32 to the plugin and provide a complete SIP solution for Openfire.

As usual, any feedback will be appreciated.

For details of how this works read on.....



The red5Phone Flex3 client makes a NetConnection with the Red5 SIP application. When it recieves a success response, it makes a remote "open" method call in the Red5 SIP application which creates a pair of SIPUser and RTMPUser objects for that user and instructs SIPUser to register the user with the specified SIP proxy. When the NetConnection is closed by the Flex3 client, the pair of objects are destroyed and the user is unregistered from the SIP proxy.

When the Flex3 client invokes "call" remotely, SIPUser starts a SIP outgoing call with the SIP proxy and exchanges RTP audio streams. It invokes "connected" on the Flex3 client and informs it of what stream names Flex3 client should use to publish from the PC microphone and play to the PC speaker. It then resamples the incoming audio RTP packets from 8KHZ to 11KHZ, converts from ulaw to ADPCM and calls a method in the RTMPUser object to publish the audio to Red5 using the same name it gave to the Flex3 client to play.

The RTMPUser objects also plays the stream being published by the Flex3 client which is in the Nellymoser ASAO codec. It calls asao2ulaw (my modified version of the open source nelly2pcm) to convert the packets to ulaw and pass to SIPUser through a PipedOutputStream.

An incoming call follows the same pattern, the incoming SIP signal appears as a remote "incoming" call on the flex3 client. The user can then pickup the call and the Red5phone Flex3 client remotely calls "accept" in SIPUser to accept the call. The audio is setup the same way as an outgoing call.

As the lead developer of the web-based version of a voice trading system used world-wide by some of the largest financial organizations, I have learned one or two things about web-based VoIP applications. When we designed our trading application, web-based VoIP was not ready for the 99% reliability and quality demanded for financial transactions. We therefore decided to keep the voice on a regular telephone (POTS) and used the web browser to handle the real-time signaling and associated data using a push method called pushlet.

My work on the Red5 plugin for Openfire is a personal attempt to investigate alternative communication technologies and get involved in the open source community. I could not have picked a better place than the Igniterealtime community. It will be interesting to see what happens when Adobe release their own VoIP enabled Flash client, but until then Flash and Red5 in my opinion is still the best way to do web-based VoIP applications.

The telephone landscape is changing. It's less about exclusive voice communication and more about real-time collaboration where voice is just one of the many communication channels used to do effective collaboration. I believe presence is a very important ingredient that brings intelligence to finding the right channel to communicate with one or many contacts in real time. This is a vision that I share with Jive Software and that why I endorse and work with Openfire on a personal basis. The plugin approach on both server and client is a very smart idea and has enabled me to integrate Flash and Red5 with Spark and Openfire with little or no effort. Implementing HTTP-binding in Openfire was icing on the cake and enabled me to add audio and video to a web-based client like JWChat with relative ease.

This brings me to the purpose of this blog.

At the moment, Spark users get a choice of SIP, Jingle and Red5 for voice. These features enable Spark to make calls to SIP phones, XMPP users and web-based XMPP users. SparkWeb does not do VOIP yet, but that will change when Jingle supports a web-compatible transport.

Web-based XMPP users could not call SIP phones until now.

I have designed a gateway (audio bridge) which converts red5 calls to and from SIP. I have now implemented a version for Windows XP/2k3 using Asteriskwin32. The basic engine is an ActiveX component written in Visual Basic and the source code is open. To solve the problem of transcoding the proprietary Nellymoser codec to SIP, I have used 8 pairs of virtual soundcards provided by a commercial product called VAC (Virtual Audio cable). It is very popular and has been used on other projects to bridge Skype for example.

I now have tight integration between Openfire, Red5 and Asterisk and this has lead to a new version of the Red5 plugin which can control the red5Gateway and implement some interesting ideas. For example, the gateway makes it possible to make every user and group JID become a public SIP address. The gateway will route and convert all SIP calls for user@domain.com to a red5 call. In reverse, if you make a Red5 call to an extension nnnnnn, the gateway will convert it to SIP and pass it on to your configured SIP service provider to route. In the case of group@example, Asterisk is used to hold the caller in a queue while every group member is called until someone answers.

Red5 gateway support is disabled by default on the Red5 plugin, as it is an optional component and is not part of the red5.war file. I am in the process of creating a hosted server to demo it over the next few weeks. If your Openfire runs on Windows and you are interested in trying it before then, send me an email and I will send you the link and documentation as soon as I finish writing it.

In the meantime, enjoy the latest version of the red5 plugin 0.0.7 which is now compatible with Openfire 3.3.0 and Spark 2.5.1. It has the latest Red5 server code: 0.6RC3. I tried to downgrade it to java5, but it was incompatible with Openfire, so I gave up trying . I have added support for viewing vcard avatars. I will add the ability to upload images in the next release so you will need Spark to upload the photographs/images for now. As I use jwchat5 myself, I have made the chat UI much like Pandion style, which I like. I only got one bug report from the previous release and have fixed it.

A big thanks to everyone at Igniterealtime, especially the folks at Jive who have added Red5 support duties to their already busy day jobs.

As always, all feedback positive or negative helps to motivate an open-source development and influence product design. The current version 0.0.7 is still far from 1.0.0, so please keep the comments coming.

-dele