Just over a year ago, I blogged about using audio and video with openfire. At that time, I implemented a SIP based softphone in Adobe Flash using AsteriskWin32 and VAC4. My argument for an open-source, standards based, no-install web-based softphone as a requirement for Web 2.0 voice applications is still valid today and the emergence of Ribbit, TringMe, Zingaya (Flashphone) and others confirmed my thinking was not isolated.
It has however been a disappointment that all implementations I have encountered to-date have been closed, proprietary and inaccessible for integration (both client and server).
The latest release of the Red5 Plugin for Openfire features a completely open-source implementation of a web-based SIP softphone written in Flex3 and should work on both Windows and Linux. It uses MjSIP as the SIP user agent in the plugin and should work with most SIP proxies, but I have only tested with Asterisk. I have also only tested 2 simultaneous users, but there is no limit and will depend on how many users and media streams Red5 can cope with before it dies. Each telephone conversation consumes 2 user RTMP connections and 4 audio streams on Red5. All source code is provided and you are free to use it in your Openfire Red5 Plugin applications. Just confirm that the open-source licenses of MjSIP and Nelly2PCM are to your liking.
I have also integrated the softphone into SparkWeb and the Openfire SIP plugin. This will enable a user to make SIP calls from Spark and SparkWeb with the same user profile. The old Red5gateway will be depreciated and in a later release for window users, I will be adding AsteriskWin32 to the plugin and provide a complete SIP solution for Openfire.
As usual, any feedback will be appreciated.
For details of how this works read on.....
The red5Phone Flex3 client makes a NetConnection with the Red5 SIP application. When it recieves a success response, it makes a remote "open" method call in the Red5 SIP application which creates a pair of SIPUser and RTMPUser objects for that user and instructs SIPUser to register the user with the specified SIP proxy. When the NetConnection is closed by the Flex3 client, the pair of objects are destroyed and the user is unregistered from the SIP proxy.
When the Flex3 client invokes "call" remotely, SIPUser starts a SIP outgoing call with the SIP proxy and exchanges RTP audio streams. It invokes "connected" on the Flex3 client and informs it of what stream names Flex3 client should use to publish from the PC microphone and play to the PC speaker. It then resamples the incoming audio RTP packets from 8KHZ to 11KHZ, converts from ulaw to ADPCM and calls a method in the RTMPUser object to publish the audio to Red5 using the same name it gave to the Flex3 client to play.
The RTMPUser objects also plays the stream being published by the Flex3 client which is in the Nellymoser ASAO codec. It calls asao2ulaw (my modified version of the open source nelly2pcm) to convert the packets to ulaw and pass to SIPUser through a PipedOutputStream.
An incoming call follows the same pattern, the incoming SIP signal appears as a remote "incoming" call on the flex3 client. The user can then pickup the call and the Red5phone Flex3 client remotely calls "accept" in SIPUser to accept the call. The audio is setup the same way as an outgoing call.
Ignite Realtime Blog
Previous Next
dele
in response to: vanaf
It is not a Win32 implementation. It is all done in Java except Nelly2PCM which is in C and compiled for both windows and Linux.
olivier2831
1. How is DTMF supported ?
2. Is this http://www.igniterealtime.org/projects/openfire/plugins-beta/red5/readme.html still valid ?
3. Do you think an H.264 and AAC compliant Red5 version is worked on ?
dele
in response to: olivier2831
1. DTMF is not supported yet, but I assume MjSIP does support it. It should be very straightfoward to put in a RTMP remote method "dtmf" in SIPUser to call from Flex3 application. The plumbing is all in place.
2. Red5 Plugin on igniterealtime plugins area is at least 2 versions behind. See readme of latest version in the Red5 plugin community area. I am dropping the red5gateway in favour of the Red5Phone (it only worked on windows) and plan to concentrate on Spark and the un-named Red5 version of SparkWeb.
3. There is an opensource RTMP server for H.264 and AAC called rubyizume (written in Ruby) that is being ported to red5. It should appear in a version soon.
topher515
You mentioned the source code, but I can't seem to find it anywhere. Is it in the openfire svn? The beta-plugins page? I'm a bit confused.
voipper
Flashphone project is created by Innovative Systems of Communication company and based on Zingaya Media Server platform (with red5 inside). It supports 2 types of DTMF (via SIP info and via RTP), registration on SIP proxies, so you can receive incoming calls, different routing types (global for server, local for users). More info... But anyway it's very good that flash SIP is also available in opensource community now.,
topher515
hey dele,
thanks for your hard work on this awesome piece of software. I've started working on porting your plugin so that it will work as a straightforward webapp on a standalone Red5 server. I was wondering if you can foresee any particular problems this might cause, based on your implementation?
dele
in response to: topher515
There should really be no problems. It is impmemented as a Red5 application and the only Openfire dependencies are calls to a static Class in the Red5 plugin to retrieve global parameters. That can be replaced with an xml or properties file. Thanks for the kind words 
barata7
in response to: dele
Congratulations for the plugin
Can you help me telling:
How to setup the whole plugin?
How to test only the SIP client? Which URL?
Best Regards,
Thiago
dele
in response to: barata7
Enable HTTP_BIND on Openfire (if disabled) and note the HTTP port.
Copy the red5.war file to the Openfire_HOME/plugins directory.
Go to the page http://your_server:your_http_bind_port/red5/index.html
See readme for more details. If you running on Linux, see additional notes by Chris.
-dele
barata7
in response to: dele
Thank you for the reply,
I got it running. But how to test SIP gateway?
I didn't see any link on the index page.
Is there an example which is just a flashphone or something like that? How did you guys test SIP stuff?
Regards,
Thiago
jadestorm
This is all very cool dele! I'll have to try it out sometime soon. It sure would be cool if I could ditch my VOIP phone client and use this instead. 
Well, I'd rather use a desktop version frankly, but hey. For what it's worth, I'm working on the desktop version of Spark (Air) lately and I believe we're planning on plugin support at some point. Wanted to let you know in case that was of interest to you in not having to hack sparkweb source itself. =)
dele
in response to: jadestorm
Hi jadestorm,
Plugin support into SparkWeb or SparkAir would be real cool . Pls let me know as soon as a design/spec is available. That would really help in adding the audo/video/telephony streaming features into SparkXXX without hacking.
Are you planning on using Flex 3 runtime-shared-libraries (RSLs)??
-dele-
jadestorm
in response to: dele
hehehe well... the "we're planning on plugin support" is about as far as we've gotten as far as I know. RSLs seem promising for this but we haven't done anything serious with it yet. Honestly, once I get the desktop version going, I imagine it'll be a bigger drive for plugin support than the web version, so once it comes about, you'll probably see plugins show up soon after in some format. But we'll definitely be in touch over it. =)
cney
Hi,
This is very cool, I have been looking for this for several months...
Do you guys have any idea of what it would take to support video
as well has audio? Is it even possible with codec issues?
Thanks,
Christophe
bugmenot user
Finally someone posted an open source solution for this kind of application
I've installed it and did some test calls (first I had to turn off the authentication on my switch since the mj sip stack does not seem to handle it very well).
The calls are established but I've got one way audio 
(there is no NAT or firewalls involved ..)
I can hear the remote side when I make call but no one can hear me.
I guess I will have to dig in into the implementation .... Unless someone has any idea ?
dele
in response to: bugmenot user
Maybe this can help you
bugmenot user
in response to: dele
Actually I've installed it on a standalone RED5 without OpenFire ... and it works ... Except that the outgoing RTP channel has very-very low volume ... The other side can barely hear me.
mgamble
in response to: bugmenot user
Can you tell me how you got it running without OpenFire? Everytime I try to register outside of Openfire I 'm getting a Service Invocation error, which is
Caused by: java.lang.NoClassDefFoundError: org/red5/server/net/rtmp/RTMPClient$INetStreamEventHandler
dele
in response to: mgamble
You need the SVN version of red5 server (red5.jar) or else you will get NoClassDefFoundErrors.
Go to http://code.google.com/p/red5phone/ for standalone version
mgamble
in response to: dele
Thanks - that did it. Now I just have to fix this no-audio issue and I'll have a working solution
nachok
Hi Dele, Thanks for this great code, the red5phone is a brilliant idea.
I´m testing the red5phone over Windows, i´m doing a call from the red5phone to a X-lite both registered in Asterik. In the flasphone i can hear the audio but in the x-lite i can´t, and i don´t see any RTP packet to the X-lite..
Someone has any idea?
mgamble
in response to: nachok
There are two possible issues:
1) The transcode script (asao2ulaw) is not complied or being executed correctly
2) You are hitting a bug I found in SIPUserAgent.java - by default the value "user_profile.audio" should be set to enable, but somewhere in the code it was getting turned off. I'm still trying to track down the source of this bug. You can patch the java file and recompile now as a workaround until a new version is released.
nachok
in response to: mgamble
Executing directly de asao2ulaw i´m getting this error:
24 main ? (716) asao2ulaw.exe: *** fatal error - system shared memory version mismatch detected - 0x75BE0084/0x8A88009C.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version. The most recent version should
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution. Rebooting is also suggested if you
are unable to find another cygwin DLL.
Should I recompile the asao2ulaw? i´m using windows at the moment
dele
in response to: nachok
You may have another version of cygwin on your PC or cygwin1.dll is in your windows/system32 folder. If you have cygwin, recompile asao2ulaw otherwise remove all other instances of cygwin1.dll
nachok
in response to: dele
Thanks a lot! the audio works ok!
I had the asterisk (it uses cygwin) intalled in the same machine that openfire, uninstalled the asterisk and problem solved!
Now i am experimenting a new issue, it seems that the red5phone dont send the register to the asterisk... but i judge that yesterday it works..
Any idea?
regards
nachok
in response to: nachok
Sorry my fault i do this test with the standalone version, the standalone version doesn´t send the Register, but the openfire version works perfectly.
Thanks a lot
lkyaya
What time will be released for the offical RED5 plugin?
I had test it.It is good except for a little qustion.example make the RED5 html very beautiful etc!
hfcoma
Hello,
this is great stuff. The "Tester Application" works without any problems (audio & video). However when trying to utilize the SIP gateway I have no audio (both ways). The SIP registration and signaling works flawlessly. The SIP server and the red5plugin/openfire run on the same machine. A firewall is involved with port 1953 opened. The red5phone application connects and dials as expected. However no audio can be heard on either end (caller and callee).
Environment: AMD_x64, OpenSuse 10.1, asterisk 1.4.20, red5plugin 0.0.21, openfire 3.5.1
Thank you
lkyaya
in response to: hfcoma
Sorry,I had tested most plugins except the SIP phone plugin.I think I can not do anything for you.very sorry!
What sip server have you utlilized?
introduce.........
hfcoma
in response to: lkyaya
Now you have me confused.
For now I want to accomplish calls between the red5phone and an asterisk server. If I understand it correctly the communication goes along this line: red5phone <- RTMP -> red5plugin <- SIP -> asterisk. So I wonder who is taking care of the SIP conversions? Until now I thought that´s the SIP phone plugin?
hfcoma
in response to: hfcoma
Okay, just a follow up!
After some digging I can confirm that asterisk sends out the RTP. However still no audio. I have the feeling that the RTMP service does not route the UDP packets correctly. The red5phone does not connect to localhost but a public IP address which is never seen within the red5 logs.
Any thoughts?
-snip-
Sent RTP packet to 127.0.0.1:3001 (type 00, seq 035012, ts 017440, len 000160)
-snip ---
The red5error.log states:
2008-06-12 17:03:03,936 AnonymousIoService-5 WARN org.red5.server.net.rtmp.RTMPClient - Unhandled ping: Ping: 0, 0, -1, -1
00 00 00 00 00 00
2008-06-12 17:03:03,953 AnonymousIoService-9 WARN org.red5.server.net.rtmp.RTMPClient - Unhandled ping: Ping: 0, 1, -1, -1
00 00 00 00 00 01
2008-06-12 17:03:06,791 pool-14-thread-4 ERROR o.r.server.service.ServiceInvoker - Error executing call: Service: null Method: publish Num Params: 10: null
2008-06-12 17:03:06,797 pool-14-thread-4 ERROR o.r.server.service.ServiceInvoker - Service invocation error
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker.java:217)
at org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.java:176)
at org.red5.server.net.rtmp.RTMPHandler.onInvoke(RTMPHandler.java:385)
at org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived(BaseRTMPHandler.java:143)
at org.red5.server.net.rtmp.RTMPMinaIoHandler.messageReceived(RTMPMinaIoHandler.java:119)
at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived(AbstractIoFilterChain.java:570)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilterChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:220)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:264)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at org.red5.server.stream.StreamService.publish(StreamService.java:265)
... 18 common frames omitted
dele
in response to: hfcoma
Post your questions here for support and read this to get started
lbiyikoglu
The plugin works almost perfect except that I can't get any ringing tone from my Asterisk pbx, if I register an other softphone (for example X-Lite) ring tone works fine. Any ideas to solve this problem?
P-o-P
Hi!
I've downloaded the standalone version and installed it on my Red5 install after having replaced the red5.jar by the appropriate one.
My SIP server is behind a proxy using the 5062 port, how do I configure the plugin/flashphone to get it to connect to my proxy?
Thanks!
Marc
Ignite Realtime Blog
Ignite Realtime Blog- Contributors:
-
srt
-
svoisen
-
ryang
-
barata7
-
jadestorm
-
dele
-
AWenckus
-
DavidSmith
-
flashgrand
-
greg
-
dombiak_gaston
-
ddman
-
dawn
-
matt
-
admin
Why win32 ?
I think it'll be better to use platform independent libraries,