Ignite Realtime Blog

Installers, Installers, Everywhere!

jadestorm — Thu, 17 Jan 2008 15:09:43 GMT

One of the things you may have noticed that appeared in the 3.4.3 release of Openfire is a couple of new installers, and some improvements to existing installers. Oddly enough, building installers can be one of the more difficult tasks a developer has. Simply putting out a tarball or zip file is easy, but it's not exactly the most pleasant thing to deal with from an administrator perspective. In the process of creating installers, you often find yourself fighting with differing standards between OS distributions, or different architectures altogether. That said, typically once you have created the installer, there's not much to do with it after, so it's generally a one time cost, so to speak, and the benefits far outweigh the time spent!

In an effort to make Openfire as easy to install as possible, we added official Debian and Solaris packages. Yes, I am aware the Solaris package is listed under Linux right now, but please ignore that for now. Are we stopping there? Not really. I'm not yet sure what other OS's we might be providing packages for at this point. FreeBSD is about the only other one I've seen a request for and there's a well maintained port (net-im/openfire) of Openfire already.

What we are investigating now is providing hosted repositories for the packages. Specifically, I'm looking at a Yum and APT repository at the moment. This would allow system administrators to point their repository configs at our repositories and be able to easily keep up to date. We are still working out the logistics of this, so stay tuned!

We're also investigating getting Openfire into more distributions. In other words, instead of having to come to our site to get Openfire, perhaps you could install it from a central Debian repository, or an extras cd, or something of that nature. There are a couple of possibilities in the works on that front, and a couple more I'd like to pursue.

So hopefully in the near future, it will be as easy as ever to get rolling with Openfire!

Jivin' Gateways and More!

jadestorm — Sun, 11 Nov 2007 18:25:04 GMT

You may or may not already be aware that I have been a full time member of the Jive family for a couple of weeks now! It's been quite interesting to see how different it is from my previous job in a university setting. It's been a lot of fun already and it's really exciting to have turned my favorite hobby into a career. =) My coworkers are great and I almost find myself wondering why I didn't do this earlier.

So what am I going to be doing? Well, the development of the IM Gateway plugin is part of my job now. We'll be setting solid goals and release dates instead of it being dependent entirely on my free time. That and Openfire are my main focuses. I'm really excited about playing a more direct role in Openfire development! One of my first tasks will be to improve the unix installers for Openfire. They have been lacking love for a while now and I have a strong unix background to bring to the table. In one of the next releases of Openfire we'll have improved packages, unix scripts, and better support for more operating system distributions. Overall, good things to come! =)

You may have heard that I have taken over as lead developer of Spark. It's been a long time since I have been involved in client development and I actually miss it. My very first XMPP related project was a client. Now, as you've heard from the Ignite Realtime post preceding this one, Spark is a low priority. My focus with it in terms of work with Jive is bug fixes, maintenance, and paying customer requirements. Beyond that, I'll likely be working on it on my own time when I need a change of pace. I am a Mac user primarily, so you may see more Mac focused fixes at first. If nothing else I'm going to make sure Spark is something I enjoy using, which coincides to a lot of things that the community has reported/requested anyway. I highly encourage folk who are interested to submit patches! The only caveat is that for patches of any size, I'll need you to sign contributor agreements, if you haven't already.

Now, since I'm involved in more than just the IM Gateway plugin now, I can't keep up with the forums as much as I did before. I try to spend some time each day looking over the forums, but with more than just the single forum, it's too much to keep up with entirely. Dawn is working hard on coming up with good ways to involve the community more and try to make sure things don't get missed! She has been speaking on this in the Jive Lounge, so please visit the lounge and contribute if you have some thoughts!

Anyway, I wanted to make sure folk understood that my role has changed and wave hi from within Jive! =) Any questions?

Openfire Unleashed

dombiak_gaston — Thu, 01 Nov 2007 00:06:32 GMT

Openfire is the award-winning instant messaging server known for its simplicity, elegance, performance and extensibility. With each new major release, scalability has been improved; however, being able to scale a lot without redundancy or high availability poses a risk to every connected user if the single server goes down.

After many months of work that risk is now part of the history. Openfire Enterprise 3.4.0 provides support for clustering. Clustering will let you run Openfire on several machines serving the same XMPP domain. Clustering can be enabled with just one click from the admin console. Machines running Openfire Enterprise will automatically meet between them to form a cluster. With clustering you not only get high availability but also improved scalability. In our internal load tests, we got more than half a million concurrent connections sending lots of packets in a cluster of just 2 nodes.

Another nice addition to Openfire Enterprise is SparkWeb. Users can now connect to the server and chat from your website. Read the SparkWeb: Next Generation blog post for more information.

On the open source side we also have excellent news. More than 30 new features and more than 30 bugs were fixed. Personal Eventing via Pubsub was added so you can now publish your geo-location, music you are listening to and let subscribers be alerted. From the admin console you can manage users roster. Moreover, it is now possible to retrieve photos from LDAP and use them as users avatars. The complete set of changes can be found here.

You can download Openfire from here. Openfire Enterprise can be downloaded from here.

Enjoy,

Openfire Team

New Community Plugin Space on Ignite

dawn — Mon, 27 Aug 2007 22:27:43 GMT

There are a bunch of great community developed plugins for Openfire and Spark floating around the net, and we thought it would be a good idea to create a space where people can upload their plugins and share them with the rest of the community.

So far, we have a Packet Filter plugin, a registration form plugin, and more. If you are interested in contributing your plugin or just want to see what others have contributed, head over to the Community Plugins space on Ignite Realtime.

Right now, these are unofficial plugins. Be sure to rate the plugins (rating located at the bottom of each document), since we will use this as a testing ground to find new plugins. We hope that some of these community plugins will eventually become official Ignite Realtime plugins.

Reflections Through The Gate

jadestorm — Thu, 09 Aug 2007 05:58:42 GMT

Monday, after a long period of heavy development, I finally put out version 1.1.0 of the IM Gateway plugin! A total of 85 issues from JIRA were closed along the way and I'm quite pleased with the results. Along the way there were a number of stumbling blocks where I would just about be ready to release and something major would come up, and I certainly did not want to release anything with serious issues going on. As development continued, more and more features became interesting to me and were implemented. Since 1.0's release, I've had a number of helpful folk step up and offer patches, testing, code, translations, and help with libraries I depend on. I want to take a moment to thank everyone who contributed in any way! You are all invaluable to me! There are a number of big plans coming for the next major release, but I wanted to highlight some of the things from 1.1.0's release and even comment on some of it!

First off, there's XMPP/Google Talk support. One might ask, why do you want XMPP support when there's s2s? That's a question that's been fought many times in the past and typically results in nothing being decided. Well I decided to implement it and then another helpful person (thanks Mehmet!) took my piddly start with it and turned it into a full on transport for the plugin. After implementing it I found myself using it with some accounts I had seen no reason to add to my Adium X config but decided hey, if I can handle it server side, then I'll just carry it around with me. Has been working out really well!

Then there's Gadu-Gadu support. This is a protocol I did not expect to ever implement. Why? Everything about it is Polish. I couldn't find my way around the web site enough to even download a copy of the client. However, I said early on that if someone would translate or help me download or generally help me get it set up and there was a good API out there, I'd do it. So thanks to Marcin for stepping up and helping me get this started! The API itself was amazingly enough the easiest API to work with yet!

And what about SIMPLE support? Thanks to Ravin and Patrick for writing this support as I wouldn't have even known where to begin! I still don't understnad SIP/SIMPLE. Who knows if I ever will. But the transport sure works with my OpenSER server! I'm hoping some folk will take a look at it so I can get a feel for what it does and does not work with and maybe I can work with them to tweak it to work correctly with various implementations.

Another interesting thing that was added is an XMLRPC interface so site administrators can write their own web front ends for users to register with the various transports. That way folk could use some various standard web look and feel for the registration, and/or their own authentication mechanism, or even just something they consider "nicer" for their users, and dodge around typical requirements of registering through a client or requiring the admin to do it for them.

A lot of the inner workings of the plugin were redone, making things a lot more efficient in terms of network traffic and overall coding structure. I don't know that anyone but me will appreciate the reworkings of the code, but hey. =)

One of the interesting things that came about is that I ended up as the lead developer for Martyr, a very cool IRC library. IRClib just wasn't getting it done for me so I tried out Martyr, adored it's structure, and offered to help improve it. It's already led to a far better IRC transport than what I had before.

Beyond that I want to thank the folk from nimbuzz for creating the wonderful project OpenYMSG (a fork of YMSG) that fixes a number of problems I kept running into in the past! Through their improvements Yahoo support in the IM Gateway plugin was promoted to being considered stable! On top of that they've helped me out some with JML, the library that handles MSN support. Great work folk!

Well that's enough of me, I'm excited about things to come though and I hope you all will join me in that excitement and continue to help me with testing, ideas, and whatever! =)

Want a free trip to OSCON or XMPP Devcon?

dawn — Tue, 03 Jul 2007 17:50:31 GMT

Want to come to Portland for XMPP Devcon or OSCON without spending money on travel? Have a cheap boss who won't foot the bill?

If you want Jive Software to pay for your travel, you just need to win the OSCON Trip Give-A-Way Contest by creating the best blog entry about how Clearspace, Jive Forums, Openfire and/or Spark have helped your organization. Your blog should be entertaining and creative while describing how you've used Jive software to make your organization better in some way.

All of the details and fine print can be found on the Jive Talks blog.

Reminder: Vote for Openfire

matt — Thu, 24 May 2007 21:17:35 GMT

You may remember me asking for everyone's help a few months back to vote for Openfire. We're entered in the Enterprise Open Source Reader???s Choice Award in the "Best Open Source Product" category. The deadline for voting is May 31, which is just one week away. If you haven't already voted, please visit the site to cast your vote. Note that the voting process started before the rename of the server, which is why you'll see the old Wildfire name.

The good news is that we're in the lead position with 198 votes. But other projects aren't far behind and I'd be thrilled to solidify our lead and hit at least 250 votes. Thanks for your help!

Openfire Powering Web 2.0

matt — Mon, 30 Apr 2007 16:56:05 GMT

I've recently run into a number of innovative sites and products that use Openfire.

!http://www.igniterealtime.org/blog/wp-content/uploads/2007/04/imified.gif![IMified|http://www.imified.com/] provides tools like task management, reminders and todo's through IM bots on AIM, MSN, and XMPP/GTalk. They've just released an API that will make it easy to write bots that work across all major IM networks. Openfire powers their XMPP back-end. !http://www.igniterealtime.org/blog/wp-content/uploads/2007/04/mosoto.gif![Mosoto|http://www.mosoto.com/] is a real-time collaboration application for Facebook that includes chat along with file and music sharing. They're one of the premier implementors of the Facebook API and have a slick Flash client UI. The service is now in Alpha, but should mature and grow quickly (especially given the size of the Facebook userbase). They use both Openfire and the XIFF Flash API.

!http://www.igniterealtime.org/blog/wp-content/uploads/2007/04/justintv.gif!Finally, we were told at the Web 2.0 conference that the chat feature on Justin.tv is powered by Openfire. I haven't had a chance to to verify that yet, but if true, it's cool that we're part of such an interesting social experiment. If you haven't seen the site yet, Justin wears a portable camera 24x7 that streams a live video feed to the website. I personally can't imagine broadcasting my life, but I have to respect the social commentary. In a world where we're always connected and available through cellphones and IM, this guy is really always connected and available... to the entire world.

These three examples are part of a broader trend: I think that XMPP will become a critical piece of infrastructure for a large number of next-generation web efforts, including Google. Recent protocol advances like BOSH (XMPP for web pages), Jingle (voice and video) and PEP (advanced presence features) will only further help drive adoption.

On the Release Train

matt — Sun, 22 Apr 2007 20:13:35 GMT

The engineering team at Jive Software is growing fast (btw, we're hiring!). I thought it might be interesting to talk about some of the engineering process changes we're making to cope with that growth, especially since they directly affect product releases. First, a bit of history. We've always had a fairly agile development process -- lots of iterations, tools like unit tests and continuous integration, etc. But, we've consistently had a lot of pain around our current process:

* Not enough time for QA* . It's always scheduled, but gets squeezed due to lack of time. For example, the "official" QA time period for the Spark 2.5.0 release got crunched down to almost nothing. * Stress when we don't need it* . The stress is caused by having to cram a ton of work into a short period of time to make internally set product release dates. It's also caused by scheduling extra features into releases at the last minute and by having to do emergency patch releases due to bugs we missed. We like working really hard, but there should be a way to do that without excessive stress. * Unpredictable release dates* . The bigger a release is, the harder it is to make accurate work effort estimates. That means that dates slip and that it's hard to communicate to the outside world exactly when a release will ship.

So, there are some clear problems that we want to fix. The trick for us was to come up with a process that fixes those problems but that's still light-weight enough to avoid a huge administrative burden. We're now experimenting with a strict release train model for Spark and Smack. Assuming it goes well, the same model will be applied across all our products, including Clearspace. A visual summary of the model is below (click the image for a larger version):

[Clearspace|http://www.igniterealtime.org/blog/wp-content/uploads/2007/04/train.png|train_sm all.png]

The key points to this model:

We put out a new release every three weeks (although each release will have gone through a nine week process total).
Three weeks before each development cycle is reserved for product management; three weeks after each development cycle is reserved for QA. But notice that all three processes are all happening at the same time.

There are all sorts of subtleties to the system like how to deal with new features that take more than three weeks of engineering time, but so far everything is going quite well.

The start of the three week cycle is always on a Monday, and we have a big meeting to determine exactly which features will go into the release. Product management has already done the work of prioritizing the features so we just need to choose exactly what we're going to work on and come up with the plan about how to get it done over the next three weeks. On the second and third Mondays during each cycle we go through more detailed updates than in our SCRUM meetings to see what's on track and what's not.

At the end of the cycle, we:

Do the product release coming out of QA on Thursday.
On Friday, finish all development and create a branch ("spark_2_5_2_branch" for example).
Switch the builds in our continuous integration environment, TeamCity. The new branch becomes the "stable" release of the product and trunk becomes "development".
Release a beta version out of the new branch. In other words, the world generally always sees a stable official product release and a beta release that will become official in three weeks.

There's way more that I could talk about with this process (including some of the potential drawbacks), but I'll save that for future blog entries. Things we're jazzed about so far include solid release dates, having a better process in place to deal with new feature requests and bug reports, more stable quality in each release, and always having a place to check in new feature development (trunk). We're still early on in the new system, but we'll all know how exactly how well this works in the next two months. Of course, we'd love your feedback as time goes on about how well you think we're doing.

Flash-based Audio and Video in Openfire

dele — Tue, 17 Apr 2007 11:59:06 GMT

As the lead developer of the web-based version of a voice trading system used world-wide by some of the largest financial organizations, I have learned one or two things about web-based VoIP applications. When we designed our trading application, web-based VoIP was not ready for the 99% reliability and quality demanded for financial transactions. We therefore decided to keep the voice on a regular telephone (POTS) and used the web browser to handle the real-time signaling and associated data using a push method called pushlet.

My work on the Red5 plugin for Openfire is a personal attempt to investigate alternative communication technologies and get involved in the open source community. I could not have picked a better place than the Igniterealtime community. It will be interesting to see what happens when Adobe release their own VoIP enabled Flash client, but until then Flash and Red5 in my opinion is still the best way to do web-based VoIP applications.

The telephone landscape is changing. It's less about exclusive voice communication and more about real-time collaboration where voice is just one of the many communication channels used to do effective collaboration. I believe presence is a very important ingredient that brings intelligence to finding the right channel to communicate with one or many contacts in real time. This is a vision that I share with Jive Software and that why I endorse and work with Openfire on a personal basis. The plugin approach on both server and client is a very smart idea and has enabled me to integrate Flash and Red5 with Spark and Openfire with little or no effort. Implementing HTTP-binding in Openfire was icing on the cake and enabled me to add audio and video to a web-based client like JWChat with relative ease.

This brings me to the purpose of this blog.

At the moment, Spark users get a choice of SIP, Jingle and Red5 for voice. These features enable Spark to make calls to SIP phones, XMPP users and web-based XMPP users. SparkWeb does not do VOIP yet, but that will change when Jingle supports a web-compatible transport.

Web-based XMPP users could not call SIP phones until now.

I have designed a gateway (audio bridge) which converts red5 calls to and from SIP. I have now implemented a version for Windows XP/2k3 using Asteriskwin32. The basic engine is an ActiveX component written in Visual Basic and the source code is open. To solve the problem of transcoding the proprietary Nellymoser codec to SIP, I have used 8 pairs of virtual soundcards provided by a commercial product called VAC (Virtual Audio cable). It is very popular and has been used on other projects to bridge Skype for example.

I now have tight integration between Openfire, Red5 and Asterisk and this has lead to a new version of the Red5 plugin which can control the red5Gateway and implement some interesting ideas. For example, the gateway makes it possible to make every user and group JID become a public SIP address. The gateway will route and convert all SIP calls for user@domain.com to a red5 call. In reverse, if you make a Red5 call to an extension nnnnnn, the gateway will convert it to SIP and pass it on to your configured SIP service provider to route. In the case of group@example, Asterisk is used to hold the caller in a queue while every group member is called until someone answers.

Red5 gateway support is disabled by default on the Red5 plugin, as it is an optional component and is not part of the red5.war file. I am in the process of creating a hosted server to demo it over the next few weeks. If your Openfire runs on Windows and you are interested in trying it before then, send me an email and I will send you the link and documentation as soon as I finish writing it.

In the meantime, enjoy the latest version of the red5 plugin 0.0.7 which is now compatible with Openfire 3.3.0 and Spark 2.5.1. It has the latest Red5 server code: 0.6RC3. I tried to downgrade it to java5, but it was incompatible with Openfire, so I gave up trying :(. I have added support for viewing vcard avatars. I will add the ability to upload images in the next release so you will need Spark to upload the photographs/images for now. As I use jwchat5 myself, I have made the chat UI much like Pandion style, which I like. I only got one bug report from the previous release and have fixed it.

A big thanks to everyone at Igniterealtime, especially the folks at Jive who have added Red5 support duties to their already busy day jobs.

As always, all feedback positive or negative helps to motivate an open-source development and influence product design. The current version 0.0.7 is still far from 1.0.0, so please keep the comments coming.

-dele

Looking Through The Fiery Gate

jadestorm — Mon, 02 Apr 2007 17:15:28 GMT

As many of you may have noticed. the IM Gateway plugin version 1.0 was released alongside Wildfire 3.2.3. (sorry guys, I can't call it Openfire until the official release is 3.3.0 ) The release was quite a step for me as I've never actually released a 1.0 of anything! Typically I'll take the attitude of having to get everything perfect before I can release a 1.0. One of the things that Jive helped me realize is that I could label some of the problem children (Yahoo, ICQ) support as experimental and basically release a 1.0 with good solid support for the rest of the protocols. That hadn't occured to me before and I'm quite glad to see 1.0 out. Amazingly enough, there haven't been a lot of bug reports. I hope that everyone who is using it is enjoying it.

I must admit that pushing Yahoo and ICQ support to the side "hurt" a little. However the library I'm using for Yahoo isn't particularly stable and the ICQ support in joscar isn't entirely golden either, and I didn't want to see the release drawn out way into the future. I am pleased to say that the Yahoo library is getting some love as we speak and there's been improvements to the ICQ support submitted to the joscar team and I'm working on a couple of other improvements. I would consider myself highly familiar with the OSCAR (AIM and ICQ) protocol itself, but not at all for Yahoo. (I'm referring to the low level details of how the protocol itself works, in theory the libraries I use ought to hide the details from me) As it turns out, I'm getting more familiar with the MSN protocol as I ended up taking over as lead developer of Java-JML. Thankfully, I've got some help on that front as well. It's always nice to see help coming in from many angles!

So what's coming next? Well, my highest priority is to get ICQ fixed up. Once the Yahoo library improvements come about, I'll be aiming to solidify that as well. Of overall new features, I'm waffling between groupchat and file transfer. Groupchat is now the most voted for feature for the plugin. File transfer, on the other hand, is something I've never had an opportunity to get involved in, so generally I'm interested in learning how it works by diving into it. Buddy icons are also highly voted for. Such a seemingly pointless/unimportant feature, but I'm with you all; it's one of the features that gives me the biggest joy to see in a chat client. They're just so cute. ;D

Why did I title this Looking Throught The Fiery Gate? Well.. it's a gateway plugin.. it's in Openfire/Wildfire... yeah.

Openfire Name Launched

matt — Thu, 15 Mar 2007 17:25:30 GMT

Today we re-launched Wildfire using the Openfire name. Along with the name change, we also have a great new logo. I wanted to provide some details about how the name change is being implemented to decrease confusion.

First, you'll notice that all website content has been updated to Openfire. There may be a few broken links still, so if you notice anything please let us know. We're also reaching out to external sites that still use the old Wildfire name and asking them to make updates.

The product releases will work as follows:

3.2.x series: these releases will continue to use the Wildfire name so that critical bug fixes can be delivered with minimal hassle to existing users. In fact, we're releasing 3.2.3 today.
3.3.x series: starting with 3.3.0, the server is called Openfire. There will be a small amount of upgrade pain associated with this release since configuration files like wildfire.xml will now be openfire.xml, etc. We'll be releasing 3.3.0 beta early next week, with a final release following as soon as we can ensure the upgrade process is as smooth as possible.

Again, my deep thanks to everyone for all the support you've given us during the name change process. A lot of the hard work is already out of the way at this point, so now it's time to spread the word as much as possible!

A new name: Openfire

matt — Wed, 28 Feb 2007 17:05:01 GMT

I'm pleased to announce that we've chosen a new name for the Wildfire server: Openfire. There's several things I really like about the new name:

We preserve the fire theme, which makes for an easier transitition. In fact, we're working on some snazzy updated logo ideas right now.
The new name better communicates two of the strongest aspects of the project: open protocol and Open Source.
The suggestion originally came from a community member.
Things look good on the trademark front.

For those looking for more context about the name change, see my original blog entry and Greg's update. Making the full transition to the new name is going to be a long and painful process. First we'll tag everything as "Openfire, formerly Wildfire" and then will gradually transition to just using the Openfire name. We truly appreciate everyone's help, especially with the huge list of name ideas. I'm eager to hear what you all think of the name name!

Unexpected XML parsing learnings

dombiak_gaston — Tue, 27 Feb 2007 19:43:48 GMT

Smack uses an XML Pull Parser (XPP) to parse XMPP stanzas and build custom Packet objects. Wildfire represents XMPP stanzas as DOM objects wrapped by Packet objects. Each approach has its own pros and cons. Moreover, in Wildfire we can use different parsers to generate DOM objects.

For Wildfire 3.2 we needed to change the way we were parsing XML to work with the new, more scalable, networking layer built using MINA. We wanted to keep building DOM objects wrapped by Packet objects so our parsers were still useful. However, we needed a way to parse stanzas received in an asynchronous way. We ended up reusing a contribution made by a community member. We know that it is a custom solution that we might need to replace at some point since the parsing is not 100% efficient and some cases may potentially break the parser as was seen in Wildfire 3.2.1.

Anyway, while doing some heavy load testing on Wildfire and measuring performance we noticed that the parser we were using to generate DOM objects was becoming a serious bottleneck. At that point we were using a SAX parser (SAXReader) so we decided to try with other parsers and compare results. The other parser that we had in Wildfire was XMPPPacketReader that uses an XML Pull Parser (XPP). To my surprise the performance improvement was substantial with the new parser: around 30% faster with the XPP parser compared to the SAX parser.

Architecturally it is obvious that an XPP parser will be much faster than a SAX parser, but since both parsers were being used to create DOM objects I initially thought it wouldn't make much of a difference which one was used since building DOM objects was the expensive operation. Well, I was simply flat wrong and I learned it the hard way. I'm happy, though, that we found this bottleneck during our performance tests prior to release. It is yet another proof of how important it is to include QA as part of your development cycle.

More on the Renaming of Wildfire

greg — Tue, 27 Feb 2007 19:35:05 GMT

Thanks to everyone for the feedback on our need to move to a new name for Wildfire. We are still working on a new name to take the place of Wildfire. Also, based on the blog posts we received, we realized that we should provide more clarification on how the trademark issue arose, and our need to move to a new name. Here's the story -

It started in December 2005, when we adopted the Wildfire name, believing it to be available for use with our IM server. In 2006, a company by the name of HBN, Inc. became aware of our usage, and HBN noted that there was possible overlap with their usage of the Wildfire name. They pointed out that they were first to use the Wildfire name and requested that we rename our product to avoid any potential confusion in the future. We agreed on it after some discussions around the issue. When we first announced this situation, we did not mention HBN but that is part of the story and our reason for changing names.

We'll be moving forward with the new name shortly....