Good afternoon!
While using Openfire we had some problems in messages with cyrillics. Some symbols (cyrillic) displayed incorrectly. It looks as one entered cyrillic symbol, after the sending, replaced with two rectangle symbols. It happens not more than for two symbols per message. Bad symbol each time different. And if you repeat your message - this symbol will display correct, but possible another symbol will play this joke again.
The same results for gentoo(Java Version: 1.6.0_03 Sun Microsystems Inc. -- Java HotSpot(TM) Server VM) and for debian testing (1.6.0 Sun Microsystems Inc. -- Java HotSpot(TM) Server VM)
We have tryed to use different client programs (psi, pidgin, miranda) on the different
platforms (Linux, Windows).
Hi,
this sounds a lot like JM-1175. Or did you already downgrade the MINA version?
LG
>> this sounds a lot like
yes
>> Or did you already downgrade the MINA version?
no ![]()
svn co http://svn.igniterealtime.org/svn/repos/openfire/trunk openfire
copy mina files, rename:
mv mina-core-1.1.0.jar mina-core.jar; mv mina-filter-compression-1.1.0.jar mina-filter-compression.jar; mv mina-filter-ssl-1.1.0.jar mina-filter-ssl.jar
"Change dir to $ /build, and execute : ant openfire"
ant openfire
Buildfile: build.xml
init:
compile:
javac Compiling 556 source files to /opt/svn/openfire/work/classes
javac /opt/svn/openfire/src/java/org/jivesoftware/openfire/sasl/SaslServerFactoryImpl .java:19: warning: com.sun.security.sasl.util.PolicyUtils is Sun proprietary API and may be removed in a future release
javac import com.sun.security.sasl.util.PolicyUtils;
javac &n bsp; &nbs p; ^
javac /opt/svn/openfire/src/java/org/apache/mina/management/MINAStatCollector.java:30 9: cannot find symbol
javac symbol : method getEventQueueSize(org.apache.mina.common.IoSession)
javac location: class org.apache.mina.filter.executor.ExecutorFilter
javac &n bsp; tmpQueuevedEvents += executorFilter.getEventQueueSize(session);
javac &n bsp; &nbs p; &n bsp; ^
javac /opt/svn/openfire/src/java/org/apache/mina/management/MINAStatCollector.java:30 9: inconvertible types
javac found : <nulltype>
javac required: long
javac &n bsp; tmpQueuevedEvents += executorFilter.getEventQueueSize(session);
javac &n bsp; &nbs p; &n bsp; &nbs p; ^
javac /opt/svn/openfire/src/java/org/jivesoftware/openfire/sasl/SaslServerFactoryImpl .java:30: warning: com.sun.security.sasl.util.PolicyUtils is Sun proprietary API and may be removed in a future release
javac private static final int mechPolicies[] = { PolicyUtils.NOANONYMOUS };
javac &n bsp; &nbs p; ^
javac /opt/svn/openfire/src/java/org/jivesoftware/openfire/sasl/SaslServerFactoryImpl .java:49: warning: com.sun.security.sasl.util.PolicyUtils is Sun proprietary API and may be removed in a future release
javac if (mechanism.equals(myMechs[PLAIN]) && PolicyUtils.checkPolicy(mechPolicies[PLAIN], props)) {
javac &n bsp; &nbs p; ^
javac /opt/svn/openfire/src/java/org/jivesoftware/openfire/sasl/SaslServerFactoryImpl .java:65: warning: com.sun.security.sasl.util.PolicyUtils is Sun proprietary API and may be removed in a future release
javac return PolicyUtils.filterMechs(myMechs, mechPolicies, props);
javac &n bsp; ^
javac Note: Some input files use or override a deprecated API.
javac Note: Recompile with -Xlint:deprecation for details.
javac Note: Some input files use unchecked or unsafe operations.
javac Note: Recompile with -Xlint:unchecked for details.
javac 2 errors
javac 4 warnings
BUILD FAILED
/opt/svn/openfire/build/build.xml:278: Compile failed; see the compiler error output for details.
Total time: 8 seconds
done
can not be compilled bacause files, placed in /org/apache/mina/filter/executor/* (mina-core-1.1.*)
=)
Can we hope for you fix this bug in mina?
Hey vovans,
I would like to confirm that this is a MINA problem so I can report it to MINA developers. If you no longer see this problem when using an old version of MINA then let us know and we will talk with MINA devs.
Thanks,
-- Gato
>> If you no longer see this problem when using an old version of MINA then let us know and we will talk with MINA devs.
Yes, the problem has disappeared (fixed) (mina 1.1.0)
thanks
Confirm.
Built openfire with mina 1.1.0 and everything is clear (tested with 2 clients only sending each other large messages in russian).
Built with mina 1.1.1 and got this bug again.
Hello everyone!
I've enabled additional debugging in read method of XMLLightWeightParser:
/*
Main reading method
*/
public void read(ByteBuffer byteBuffer) throws Exception {
invalidateBuffer();
// Check that the buffer is not bigger than 1 Megabyte. For security reasons
// we will abort parsing when 1 Mega of queued chars was found.
if (buffer.length() > 1048576) {
throw new Exception("Stopped parsing never ending stanza");
}
Log.debug("XMLLightWeightParser: " + byteBuffer.toString());
CharBuffer charBuffer = encoder.decode(byteBuffer.buf());
Log.debug("XMLLightWeightParser: " + charBuffer.toString());
char[] buf = charBuffer.array();
int readByte = charBuffer.remaining();
to see what happens.
So, Openfire reads messages by parts. Cyrillic letters are double-byte. If first byte at the end of one part and second byte at the beginning of next one, each byte decodes in CharBuffer charBuffer = encoder.decode(byteBuffer.buf()) separetaly and replaces with Unicode replacement symbol 0xFFFD. Users see two 0xFFFD (square or question mark) symbols instead of one cyrillic (and other double-byte i think) letter.
It's happens with mina 1.1.0 in my case too, but more rare. It's uncommon for secure connections (i could reproduce this behaviour once or twice in all my tests).
I can send my debug logs, if you need them.
Hey Kinyapin,
Excellent analysis. I can now totally see why this problem is happening. I'm now trying to figure out the correct way to fix it but the problem has been identified now.
Just as you mentioned this problem happens when each byte of a double-byte char arrives in 2 different iterations but what we are storing in XMLLightWeightParser is a corrupted/incomplete conversion of the byte into a character.
Thanks a lot for this information and the analysis. I will try to find a fix for this tonight or tomorrow.
Regards,
-- Gato
Hey Kinyapin,
I'm no longer able to reproduce it locally so I was not able to test the fix. If you or anyone else is willing to test it I can send you a modified version of Openfire 3.4.4 or 3.4.5 with the patch so you can try it out.
Thanks,
-- Gato
Hi Gato!
Yes, i can test it this weekend and send results on monday or tuesday.
Thank you for quick fix.
Hey Kinyapin,
Drop me an email with the Openfire version that you are using. Tomorrow morning (PST) I will send you back a modified version that you can try.
Thanks,
-- Gato
Hey Gato,
I'm also interested in that patch. We're still using 3.4.2 though. Could you send me the patch itself (instead of a completely modified server)? We already applied other (unrelated) changes that I'd rather not override. Thanks!
Guus so you will not upgrate your openfire? It's not good...
if your changes are interesting for others send your patchs to developers of thr openfire
else make them as plugins
Thanks Gato, I've applied the patch to our source. I'll try to deploy it to the one machine that gives us the best chances of reproducing the problem.
Vanaf, I've actually sent in patches for a lot of the non-plugin modifications that I have. I believe serveral of them are scheduled for the upcoming 3.5.0 release. We're currently running four domains with Openfire, three of them in a clustered setup. We have been integrating a lot of custom systems (through 20+ plugins, client connections and external component implementations. Each time we update Openfire, we need to do regression tests on all of them, to check for compatibility issues. This entire process (which has to be repeated in several test environments) just eats up to much of our time to do it with ever minor Openfire release - that's why I'm still running 3.4.2