Ignite Realtime: Lag in Multi User Chat ...

Up to Discussions in Openfire Support

14 Replies Last post: Dec 13, 2007 7:32 AM by notz

Clemens

5 posts since
Sep 23, 2007

Currently Being Moderated

Oct 11, 2007 11:01 AM

Lag in Multi User Chat

Greetings!

I would like to know if anybody else experienced this:

I'm participating in a MUC with about 7-10 users. It sometimes occurs that there is a huge lag for (seemingly) all participants. This includes both users that connect from another server as well as local users. Messages geht queued and every (exactly) 20 seconds one message is spit out. During a recent "lag period" it even happened to me that one of my messages only appeared after seven minutes.

It seems to me that this is more or less unrelated to the level of activity, because it does not happen daily and not necessarily when we are engaged into a verbose debate

I wonder if that is some problem with karma/internal traffic shaping which gets applied to the MUC component.

I have not seen any messages related to this in the Warn, Error or Info log.

Running Openfire 3.3.3 on Linux.

Tags: openfire, muc, chat, lag

Gaston Dombiak

3,771 posts since
Sep 26, 2001

Currently Being Moderated

Oct 11, 2007 12:23 PM

in response to: Clemens

Re: Lag in Multi User Chat

Hey Clemens,

Openfire does not apply karma or any throttling mechanism to the network traffic. This could be a network problem (e.g. a lazy client not reading quickly incoming traffic thus letting his TCP buffer fill up) or it may be a server problem. For the former you can use the # plugin and monitor the MINA queues (i.e. the last 2 columns) and for the latter you can get a few thread dumps of the server's JVM to see what the server is doing. To get a thread dump in linux just execute kill -3 \[process id\].

Regards,

-- Gato

Attachments:

loadStats.jar (8.2 K)

Report Abuse

Clemens

5 posts since
Sep 23, 2007

Currently Being Moderated

Oct 11, 2007 4:28 PM

in response to: Gaston Dombiak

Re: Lag in Multi User Chat

Thanks a lot, Gato!

A very helpful reply indeed; although I have a question regarding the output of loadStats.

I receive output like:

1192137389325,5,15,6,0,17,0,0,493310,21,1493,7325,0,-7311

...

1192137544325,5,15,6,0,17,0,0,493374,21,1495,7336,0,-7322

...

1192137694324,5,15,6,0,17,0,0,493447,21,1508,7350,0,-7336

I wonder why column 14 (Queued NIO writes) does have a negative value that decreases.

Thanks in advance!

Report Abuse

Gaston Dombiak

3,771 posts since
Sep 26, 2001

Currently Being Moderated

Oct 11, 2007 4:48 PM

in response to: Clemens

Re: Lag in Multi User Chat

Hey Clemens,

Today we were discussing this problem in this other thread: Re: 3.3.3 - loadStats broken?.

Regards,

-- Gato

Report Abuse

Clemens

5 posts since
Sep 23, 2007

Currently Being Moderated

Oct 24, 2007 12:33 PM

in response to: Gaston Dombiak

Re: Lag in Multi User Chat

I just caught an increasing number in Queued NIO events "on tape". I noticed that there again was a lag, but this time also when sending messages to another server ? receiving messages was just fine.

Log sample, lines with same 'Queued NIO events' count removed

1193246244321,5,15,6,0,17,0,0,788074,15,24475,75182,0,-15566

1193246249321,5,15,6,0,17,1,0,788076,15,24476,75184,1,-15568

1193246274321,5,15,6,0,17,1,0,788081,15,24476,75184,3,-15568

1193246279321,5,15,6,0,17,1,0,788082,15,24476,75184,4,-15568

1193246314321,5,15,6,0,17,1,0,788091,15,24477,75187,5,-15571

1193246319321,5,15,6,0,17,1,0,788092,15,24477,75187,5,-15571

1193246329321,5,15,6,0,17,1,0,788093,15,24477,75187,6,-15571

1193246334321,5,15,6,0,17,1,0,788095,15,24477,75188,7,-15572

1193246339321,5,15,6,0,17,1,0,788095,15,24477,75188,8,-15572

1193246344321,5,15,6,0,17,1,0,788095,15,24477,75188,9,-15572

1193246384321,5,15,6,0,17,1,0,788102,15,24477,75188,10,-15572

1193246439320,5,15,6,0,17,1,0,788112,15,24477,75189,11,-15573

1193246444320,5,15,6,0,17,1,0,788113,15,24477,75191,12,-15575

1193246449320,5,15,6,0,17,1,0,788113,15,24477,75192,13,-15576

1193246459324,5,15,6,0,17,1,0,788116,15,24477,75193,14,-15577

1193246464320,5,15,6,0,17,1,0,788116,15,24478,75193,16,-15577

1193246489320,5,15,6,0,17,1,0,788119,15,24478,75193,17,-15577

1193246509320,5,15,6,0,17,1,0,788124,15,24478,75194,18,-15578

1193246519320,5,15,6,0,17,1,0,788126,15,24478,75196,19,-15580

1193246549320,5,15,6,0,17,1,0,788129,15,24478,75196,20,-15580

1193246604320,5,15,6,0,17,1,0,788140,15,24478,75198,21,-15582

1193246659320,5,15,6,0,17,1,0,788150,15,24478,75198,22,-15582

1193246664320,5,15,6,0,17,1,0,788151,15,24479,75201,21,-15585

1193246669319,5,15,6,0,17,1,0,788152,15,24479,75201,22,-15585

1193246684327,5,15,6,0,17,1,0,788158,15,24480,75205,21,-15589

1193246704319,5,15,6,0,17,1,0,788162,15,24486,75208,2,-15592

1193246709323,5,15,6,0,17,1,0,788163,15,24486,75209,4,-15593

1193246714323,5,15,6,0,17,1,0,788165,15,24486,75209,4,-15593

1193246719323,5,15,6,0,17,0,0,788170,14,24486,75213,5,-15597

1193246724323,5,15,6,0,17,0,0,788171,14,24486,75214,0,-15598

Not sure if that is helpful in any way, but maybe some evidence is better than none

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 7, 2007 8:50 AM

in response to: Clemens

Re: Lag in Multi User Chat

i see the same problem (especially under 3.4.x) - running with connection manager:

i have attached an extra log of each of my 5 connection manager sessions to the server. only one is hanging. there a no read messages, but read bytes and queued nio events...

perhaps some developer can explain, where these messages are queued? (ProtocolDecoderOutput ?)

i really would like to find issue, but i need some help.

sometimes the queue is cleaned after some time, but also sometimes it never gets cleaned .... i always have to restart the server & connection manager -> that's bad....

please help,

thanks

Attachments:

queued_nio_events.log (5.2 K)

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 8, 2007 8:01 AM

in response to: notz

Re: Lag in Multi User Chat

that what seeing from a jstack dump if the error occured the last time. seems to be a dead lock:

"connectionManager-9" daemon prio=10 tid=0x0865c000 nid=0x340d runnable http://0x700c7000..0x700c8020

java.lang.Thread.State: RUNNABLE

at java.lang.Throwable.fillInStackTrace(Native Method)

- locked (a org.jivesoftware.openfire.user.UserNotFoundException) at java.lang.Throwable.(Throwable.java:196) at java.lang.Exception.(Exception.java:41) at org.jivesoftware.openfire.user.UserNotFoundException.]]>(UserNotFoundException.java:31)

at org.jivesoftware.openfire.roster.Roster.getRosterItem(Roster.java:223)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.directedPresenceSent(Pr esenceUpdateHandler.java:330)

at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:161)

at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.broadcastUnavailableFor DirectedPresences(PresenceUpdateHandler.java:447)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:136)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:110)

at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:174)

at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:133)

at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:75)

at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:11 1)

at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:66 )

at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.route(MultiplexerP acketHandler.java:164)

at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processRoute(Multiplexer StanzaHandler.java:89)

at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processUnknowPacket(Mult iplexerStanzaHandler.java:96)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:271)

at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167)

at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132)

at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)

at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)

at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)

at java.lang.Thread.run(Thread.java:619)

{/code}

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 8, 2007 8:11 AM

in response to: notz

Re: Lag in Multi User Chat

that's what i got after another jstack dump:


"connectionManager-9" daemon prio=10 tid=0x0865c000 nid=0x340d runnable [http://0x700c7000..0x700c8020|http://0x700c7000..0x700c8020]
   java.lang.Thread.State: RUNNABLE
        at java.lang.String.intern(Native Method)
        at org.jivesoftware.openfire.muc.spi.MultiUserChatServerImpl.getChatUser(MultiUser ChatServerImpl.java:602)
        at org.jivesoftware.openfire.muc.spi.MultiUserChatServerImpl.processPacket(MultiUs erChatServerImpl.java:247)
        at org.jivesoftware.openfire.muc.spi.MultiUserChatServerImpl.process(MultiUserChat ServerImpl.java:227)
        at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)
        at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:163)
        at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.broadcastUnavailableFor DirectedPresences(PresenceUpdateHandler.java:447)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:136)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:110)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:174)
        at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:133)
        at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)
        at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:75)
        at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:11 1)
        at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:66 )
        at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.route(MultiplexerP acketHandler.java:164)
        at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processRoute(Multiplexer StanzaHandler.java:89)
        at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processUnknowPacket(Mult iplexerStanzaHandler.java:96)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:271)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167)
        at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132)
        at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)
        at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
        at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
        at java.lang.Thread.run(Thread.java:619)

Report Abuse

Gaston Dombiak

3,771 posts since
Sep 26, 2001

Currently Being Moderated

Dec 8, 2007 10:55 AM

in response to: notz

Re: Lag in Multi User Chat

Try upgrading to Openfire 3.4.2 that solves a problem with the outgoing MINA queue. notz, you can use the load statistics debugger to find out the size of the MINA queues and other Openfire internals. Another tool that you can use to analyze these problems is jmap to generate heap dumps and then use jhat to analyze them.

Regards,

-- Gato

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 9, 2007 12:29 PM

in response to: Gaston Dombiak

Re: Lag in Multi User Chat

i already use 3.4.2 - so the logs are all taken with 3.4.2. also i use the loadStats plugin. i only modified the plugin that i can see the output per connection manager session.

i also know jmap, but don't think that it would help on this sort of problem. on every increase (nio events) there was one thread hanging/looping/waiting. so jstack will help on debugging this. my last 2 posts was on a deadlock (endless loop - it took 100% cpu).

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 10, 2007 2:15 AM

in response to: notz

Re: Lag in Multi User Chat

i don't know if this is the problem but seems that MucServer is creating a new local user upon that direct unaivailable presence:


"connectionManager-6" daemon prio=10 tid=0x0836dc00 nid=0x77ed runnable [http://0x703ac000..0x703ad120|http://0x703ac000..0x703ad120]
   java.lang.Thread.State: RUNNABLE
        at org.dom4j.tree.AbstractElement.attributeValue(AbstractElement.java:562)
        at org.xmpp.packet.Packet.getTo(Packet.java:133)
        at org.jivesoftware.openfire.muc.spi.LocalMUCUser.process(LocalMUCUser.java:417)
        at org.jivesoftware.openfire.muc.spi.LocalMUCUser.process(LocalMUCUser.java:163)
        at org.jivesoftware.openfire.muc.spi.MultiUserChatServerImpl.processPacket(MultiUs erChatServerImpl.java:247)
        at org.jivesoftware.openfire.muc.spi.MultiUserChatServerImpl.process(MultiUserChat ServerImpl.java:227)
        at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)
        at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:163)
        at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.broadcastUnavailableFor DirectedPresences(PresenceUpdateHandler.java:447)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:136)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:110)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.process(PresenceUpdateH andler.java:174)
        at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:133)
        at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)
        at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:75)
        at org.jivesoftware.openfire.SessionManager$ClientSessionListener.onConnectionClos e(SessionManager.java:1115)
        at org.jivesoftware.openfire.net.VirtualConnection.notifyCloseListeners(VirtualCon nection.java:190)
        - locked <0x7fbdbba8> (a java.util.HashMap)
        at org.jivesoftware.openfire.net.VirtualConnection.close(VirtualConnection.java:16 6)
        at org.jivesoftware.openfire.session.LocalSession.close(LocalSession.java:303)
        at org.jivesoftware.openfire.multiplex.ConnectionMultiplexerManager.closeClientSes sion(ConnectionMultiplexerManager.java:160)
        at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.handle(Multiplexer PacketHandler.java:89)
        at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processIQ(MultiplexerSta nzaHandler.java:57)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:268)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167)
        at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132)
        at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)
        at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
        at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)

it's always looping on this position:

org.jivesoftware.openfire.handler.PresenceUpdateHandler.broadcastUnavailableForD irectedPresences

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 10, 2007 3:53 AM

in response to: notz

Re: Lag in Multi User Chat

the next endless loop:


"connectionManager-8" daemon prio=10 tid=0x082c2c00 nid=0x2b18 runnable [http://0x6f600000..0x6f600ea0|http://0x6f600000..0x6f600ea0]
   java.lang.Thread.State: RUNNABLE
        at java.lang.String.indexOf(String.java:1767)
        at java.lang.String.indexOf(String.java:1734)
        at java.lang.String.indexOf(String.java:1716)
        at java.lang.String.contains(String.java:2104)
        at org.jivesoftware.openfire.handler.PresenceUpdateHandler.hasDirectPresence(Prese nceUpdateHandler.java:461)
        at org.jivesoftware.openfire.spi.PresenceManagerImpl.sendUnavailableFromSessions(P resenceManagerImpl.java:435)
        at org.jivesoftware.openfire.handler.PresenceSubscribeHandler.process(PresenceSubs cribeHandler.java:179)
        at org.jivesoftware.openfire.PresenceRouter.handle(PresenceRouter.java:173)
        at org.jivesoftware.openfire.PresenceRouter.route(PresenceRouter.java:69)
        at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:75)
        at org.jivesoftware.openfire.roster.Roster.deleteRosterItem(Roster.java:443)
        at org.jivesoftware.openfire.handler.IQRosterHandler.removeItem(IQRosterHandler.ja va:253)
        at org.jivesoftware.openfire.handler.IQRosterHandler.manageRoster(IQRosterHandler. java:215)
        at org.jivesoftware.openfire.handler.IQRosterHandler.handleIQ(IQRosterHandler.java :105)
        at org.jivesoftware.openfire.handler.IQHandler.process(IQHandler.java:48)
        at org.jivesoftware.openfire.IQRouter.handle(IQRouter.java:348)
        at org.jivesoftware.openfire.IQRouter.route(IQRouter.java:100)
        at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:67)
        at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:99 )
        at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:60 )
        at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.route(MultiplexerP acketHandler.java:164)
        at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processRoute(Multiplexer StanzaHandler.java:89)
        at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processUnknowPacket(Mult iplexerStanzaHandler.java:96)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:271)
        at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167)
        at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132)
        at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58)
        at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173)
        at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
        at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
        at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
        at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
        at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 10, 2007 5:41 AM

in response to: notz

Re: Lag in Multi User Chat

i think the presence update handler in 3.4.x is not really thread-safe. one thing that changed from 3.3.x to 3.4.x is that you use in your new class DirectedPresence a HashSet (wihich is not thread safe) before you used an ConcurrentHashSet which should be thread safe.

i will give it a try on next lockup, if this change something.

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 10, 2007 6:59 PM

in response to: notz

Re: Lag in Multi User Chat

the next lock, now there are 2 threads waiting:

"connectionManager-17" daemon prio=10 tid=0x716e4000 nid=0x4a27 in Object.wait() [http://0x6f62a000..0x6f62afa0|http://0x6f62a000..0x6f62afa0] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.jivesoftware.openfire.user.UserManager.isRegisteredUser(UserManager.java:34 9) - locked <0x747218f0> (a java.lang.String) at org.jivesoftware.openfire.OfflineMessageStore.addMessage(OfflineMessageStore.ja va:112) at org.jivesoftware.openfire.OfflineMessageStrategy.store(OfflineMessageStrategy.j ava:138) at org.jivesoftware.openfire.OfflineMessageStrategy.storeOffline(OfflineMessageStr ategy.java:106) at org.jivesoftware.openfire.MessageRouter.routingFailed(MessageRouter.java:188) at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:305) at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:96) at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:71) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:10 5) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:63 ) at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.route(MultiplexerP acketHandler.java:164) at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processRoute(Multiplexer StanzaHandler.java:89) at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processUnknowPacket(Mult iplexerStanzaHandler.java:96) at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:271) at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167) at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132) at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570) at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299) at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53) at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648) at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173) at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299) at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53) at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648) at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239) at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51) at java.lang.Thread.run(Thread.java:619) . . . "connectionManager-15" daemon prio=10 tid=0x0869b400 nid=0x4a1f in Object.wait() [http://0x700e1000..0x700e1ea0|http://0x700e1000..0x700e1ea0] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.jivesoftware.openfire.user.UserManager.isRegisteredUser(UserManager.java:34 9) - locked <0x7471f780> (a java.lang.String) at org.jivesoftware.openfire.OfflineMessageStore.addMessage(OfflineMessageStore.ja va:112) at org.jivesoftware.openfire.OfflineMessageStrategy.store(OfflineMessageStrategy.j ava:138) at org.jivesoftware.openfire.OfflineMessageStrategy.storeOffline(OfflineMessageStr ategy.java:106) at org.jivesoftware.openfire.MessageRouter.routingFailed(MessageRouter.java:188) at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:305) at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:96) at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:71) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:10 5) at org.jivesoftware.openfire.SessionPacketRouter.route(SessionPacketRouter.java:63 ) at org.jivesoftware.openfire.multiplex.MultiplexerPacketHandler.route(MultiplexerP acketHandler.java:164) at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processRoute(Multiplexer StanzaHandler.java:89) at org.jivesoftware.openfire.net.MultiplexerStanzaHandler.processUnknowPacket(Mult iplexerStanzaHandler.java:96) at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:271) at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:167) at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandl er.java:132) at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived (AbstractIoFilterChain.java:570) at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299) at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53) at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648) at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimplePr otocolDecoderOutput.java:58) at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecF ilter.java:173) at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299) at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53) at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648) at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239) at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51) at java.lang.Thread.run(Thread.java:619)

Report Abuse

notz

81 posts since
May 22, 2007

Currently Being Moderated

Dec 13, 2007 7:33 AM

in response to: