[rt-users] Mail notification MIME encoding problems (cont...)

[Alan Premselaar]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I posted to the list a couple of weeks back now I think about having
problems with multi-byte characters in email subjects (specifically
Japanese)

I've found a few problems.
I think I've (at least temporarily) fixed one problem.

the first problem was with the line folding of the Subject: header with
a mix of single and multi-byte characters.  Ultimately I installed Jcode
from CPAN and added the following lines to the MIMEEncodeString
subroutine in /opt/rt3/libs/RT/Actions/SendEmail.pm

        use Jcode;
        my $jcode = Jcode->new($str);
        my @chunks = $jcode->jfold($max,'\n');

and commented out the following lines:

#      my @chunks = unpack("a$max" x int(length($str)/$max
#                                  + ((length($str) % $max) ? 1:0)), $str);

now my mail subjects aren't getting trashed by line folding.
(for what it's worth, if i just returned an unfolded encoded Subject: i
didn'T have an problems, so it was obviously an issue where it was
folding mid-2byte character in some cases)

the second problem is a little more complicated and I haven't been able
to determine exactly *why* it's happening.  I've added debug statements
in the code to track what's happening but it's still a little uncertain
as to why it's happening.  Here's the debug info (it's kind of long):

[Thu Aug 17 08:07:09 2006] [warning]: Use of uninitialized value in
substitution (s///) at /opt/rt3/lib/RT/Interface/Web.pm line 617.
(/opt/rt3/lib/RT/Interface/Web.pm:617)
[Thu Aug 17 08:07:09 2006] [debug]: About to think about scrips for
transaction #5081 (/opt/rt3/lib/RT/Transaction_Overlay.pm:165)
[Thu Aug 17 08:07:09 2006] [debug]: About to prepare scrips for
transaction #5081 (/opt/rt3/lib/RT/Transaction_Overlay.pm:169)
[Thu Aug 17 08:07:09 2006] [debug]: Found 4 scrips
(/opt/rt3/lib/RT/Scrips_Overlay.pm:363)
[Thu Aug 17 08:07:11 2006] [debug]: sub before: [Comment] testr用な結構
長い件名しなければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:752)
[Thu Aug 17 08:07:11 2006] [debug]: sub after : [Comment] testr用な結構
長い件名しなければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:755)
[Thu Aug 17 08:07:11 2006] [debug]: new subject: [sysrequest #371]
[Comment] testr用な結構長い件名しなければtextならないのでこれは結構長い
件名です。  (/opt/rt3/lib/RT/Action/SendEmail.pm:758)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: field = Subject, enc = utf-8
(/opt/rt3/lib/RT/Action/SendEmail.pm:863)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: value = [sysrequest #371]
[Comment] testr用な結構長い件名しなければtextならないのでこれは結構長い
件名です。  (/opt/rt3/lib/RT/Action/SendEmail.pm:870)
[Thu Aug 17 08:07:11 2006] [debug]: value before utf encoding:
[sysrequest #371] [Comment] testr用な結構長い件名しなければtextならない
のでこれは結構長い件名です。 (/opt/rt3/lib/RT/Action/SendEmail.pm:916)
[Thu Aug 17 08:07:11 2006] [debug]: value = [sysrequest #371] [Comment]
testr?¨?????§?????¶???????°text?????????§???????§?????¶??§??
(/opt/rt3/lib/RT/Action/SendEmail.pm:921)
[Thu Aug 17 08:07:11 2006] [debug]: chunks = [sysrequest #371] [Comment]
testr?¨?? ???§?????¶??? ????°text????? ????§??????? §?????¶??§??
(/opt/rt3/lib/RT/Action/SendEmail.pm:936)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: after encoding value =
=?UTF-8?B?W3N5c3JlcXVlc3QgIzM3MV0gW0NvbW1lbnRdIHRlc3Ryw6fClMKow6PCgcKq?=
 =?UTF-8?B?w6fCtcKQw6bCp8KLw6nClcK3w6PCgcKEw6TCu8K2w6XCkMKNw6PCgcKXw6M=?=
 =?UTF-8?B?woHCqsOjwoHCkcOjwoLCjMOjwoHCsHRleHTDo8KBwqrDo8KCwonDo8KBwqo=?=
 =?UTF-8?B?w6PCgcKEw6PCgcKuw6PCgcKnw6PCgcKTw6PCgsKMw6PCgcKvw6fCtcKQw6Y=?=
 =?UTF-8?B?wqfCi8OpwpXCt8OjwoHChMOkwrvCtsOlwpDCjcOjwoHCp8OjwoHCmcOjwoA=?=
 =?UTF-8?B?woI=?= (/opt/rt3/lib/RT/Action/SendEmail.pm:878)
[Thu Aug 17 08:07:11 2006] [debug]: sub before: testr用な結構長い件名し
なければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:752)
[Thu Aug 17 08:07:11 2006] [debug]: sub after : testr用な結構長い件名し
なければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:755)
[Thu Aug 17 08:07:11 2006] [debug]: new subject: [sysrequest #371] testr
用な結構長い件名しなければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:758)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: field = Subject, enc = utf-8
(/opt/rt3/lib/RT/Action/SendEmail.pm:863)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: value = [sysrequest #371]
testr用な結構長い件名しなければtextならないのでこれは結構長い件名です。
 (/opt/rt3/lib/RT/Action/SendEmail.pm:870)
[Thu Aug 17 08:07:11 2006] [debug]: value before utf encoding:
[sysrequest #371] testr用な結構長い件名しなければtextならないのでこれは
結構長い件名です。 (/opt/rt3/lib/RT/Action/SendEmail.pm:916)
[Thu Aug 17 08:07:11 2006] [debug]: value = [sysrequest #371] testr用な
結構長い件名しなければtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:921)
[Thu Aug 17 08:07:11 2006] [debug]: chunks = [sysrequest #371] testr用な
結構長い件名しなけ ればtextならないのでこれは結構長い件名です。
(/opt/rt3/lib/RT/Action/SendEmail.pm:936)
[Thu Aug 17 08:07:11 2006] [debug]: SHAE: after encoding value =
=?UTF-8?B?W3N5c3JlcXVlc3QgIzM3MV0gdGVzdHLnlKjjgarntZDmp4vplbfjgYTku7blkI3jgZfjgarjgZE=?=
 =?UTF-8?B?44KM44GwdGV4dOOBquOCieOBquOBhOOBruOBp+OBk+OCjOOBr+e1kOani+mVt+OBhOS7tuWQjeOBp+OBmeOAgg==?=
(/opt/rt3/lib/RT/Action/SendEmail.pm:878)
[Thu Aug 17 08:07:19 2006] [debug]: RT::Date used date::parse to make
1970-01-01 -32400 (/opt/rt3/lib/RT/Date.pm:211)


specifically what seems to be happening is, if I add "Subject:
[something] {$Ticket->Subject}" to a template, and that template is used
in the email process then in MIMEEncodeString subroutine, the
Encode::_utf8_off($value) call is mangling the string.  oddly, it only
happens if the Subject: header is put into the template.

I'm running RT 3.6.1 on a RH FC 5 machine.

These issues are preventing me from being able to put RT into production
(which I'd really like to do) so any help would be *greatly* appreciated.

Thanks,

Alan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE5CaxE2gsBSKjZHQRAu+bAKCR2MmvPemfKor7CY93A/g3dTXftwCfb5Y8
DGy9e/7u4bT0uyrz7bnJul8=
=551B
-----END PGP SIGNATURE-----