Issue Details (XML | Word | Printable)

Key: JM-379
Type: Improvement Improvement
Status: Open Open
Priority: Critical Critical
Assignee: Gaston Dombiak
Reporter: Matt Tucker
Votes: 54
Watchers: 31
Operations

If you were logged in you would be able to see more operations.
Openfire (ARCHIVED)

Change caching strategy for shared groups

Created: 08/20/05 02:27 AM   Updated: 09/25/08 11:27 PM
Component/s: Core
Affects Version/s: 2.2.0
Fix Version/s: 3.x

Time Tracking:
Not Specified

Issue Links:
Related to
 


 Description  « Hide
Many users are complaining that changes to LDAP groups are not being reflected in rosters quickly enough. There are two layers to this:

The roster code retrieves group information by calling GroupManagergetGroups(User). At the moment, there is no caching in this method although it should be added. The entire roster is then cached, included shared group information.

Because of the roster caching logic in place, it makes it very frustrating for people to see group changes take effect. First, because they'll only see changes when logging out and then back in. Second, because the roster is cached, it can take 6 hours to see changes. People generally resort to just shutting down the server.

The first step could be to change how roster caching works. I'd like to see if we could cache the db roster data but then dynamically build out the shared roster information. I'm not sure how expensive the shared roster group calculations are, which would dictate the feasability of this. Assuming it's not too expensive, if the shared group data was dynamic, then it would be GroupManager.getGroups(User) caching that would dictate how quickly changes in the back-end groups were reflected in rosters.

Second, we should try to figure out if it's possible to push roster changes to a user. An idea: have a flag in the roster code that says whether to check for roster changes on a periodic basis. Any time an external group system is used, the flag would get set by that provider. So, every X hours that a user is logged in, recalculate the shared group portion of their roster. If it's changed, send those changes to them dynamically. This would mean that if someone changes a group in LDAP, after X hours everyone from that group that continues to be logged in will see that change.



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
alexander wenckus added a comment - 08/24/05 11:46 PM
Shouldn't clients be able to recognize roster changes when they are just pushed out to them as regular roster items. When several clients are connected with different resources on the same node aren't roster updates pushed out this way?

Matt Tucker added a comment - 08/24/05 11:49 PM
> Shouldn't clients be able to recognize roster changes when they are just pushed out to them as
> regular roster items. When several clients are connected with different resources on the same
> node aren't roster updates pushed out this way?

Yep, that's correct. It's not the actual pushing that's hard. It's figuring out when to do the pushing. Basically, we need to re-calculate someone's roster in order to figure out if there are changes, which is what the X hours calculations are about above.


Dimitar Dimitrov added a comment - 02/20/06 07:48 AM
What could be implemented is:

1. When the user logs in, you can send send the cached roster
2. Push a LoginRosterRebuildTask in a blocking priority queue
3. Have a scheduled thread pool executor which runs the pushed rebuild tasks with certain frequency as to not overload the db server
3.1. when a task runs, it rebuilds the roster and compares it with the cached roster
3.2. if changes were detected - push them to the user
3.3. push a RefreshRosterRebuildTask to the queue
4. When the user logs out, remove the corresponding RefreshRosterRebuildTask from the queue


Patrick Siu added a comment - 03/03/06 10:53 AM
We may bootstrap user*name*s from LDAP and store them in the DB (If we go for space-optimization we can just store username and status), as well as storing a group-user mapping and group sharing mapping, making 3 tables in total. We can then bump out rosters by (dynamic?) DB queries.

If we queue the roster recalculation schedule user by user, the connection setup delay can be an issue, so I'd prefer unified synchronization, by gathering all online users and get their rosters when time has come. Moreover, batch-processing grouped users can also speed up the process.

Seems I am getting past caching but if we just dynamically build up changes for the cached, it may cause hinderance...


Brian Biales added a comment - 10/26/07 11:51 PM
I see this issue has the most votes of any open high priority (critical) issue in the Jira DB, and it has been there since 2005. Forgotten but not gone?Since the squeaky wheel sometimes gets the grease, I just want to say that the priority is accurate. Anyone integrated with LDAP would like to at least be able to push a button when LDAP properties change (users are added/changed/deleted, groups are added/changed/deleted). I personally don't require anything fancy. Even if it were an API call that would be fired by me to tell the server to please go refresh its list of users, groups, rosters, etc. My server is not busy with lots of messages, but I do need rosters to be correct. Anything short of restarting the server would be a huge improvement. Please advise if/when this issue will be addressed, thanks.

SVigano added a comment - 09/25/08 11:27 PM
To answer Brians question about not having to restart the server, there is a work around. You can force a refresh of the cache in question by using the "Cache Summary" page in "Server Manager". Check the box for the "Roster" cache and flush it. Unfortunately it does not update clients that are already logged on, but next time they logout and back in, the new users will appear in their roster. Without having to restart the server. (wonder if I can script this from the command line somehow??? any ideas??)

Even with this workaround, I too would like to see a real time, or near real time, update happen with LDAP Group additions/deletions. Our LDAP directory is on the small side so I'm not too concerned with occasional refreshes. Maybe the timing of the automatic refresh could be Admin configurable. That way some admins could schedule a refresh overnight if needed and other could go more frequently.

Big bonus points for pushing any changes out to existing logged in accounts.

Thanks
-S