[rt-users] attachment encoding causes internal server error

[Joosep Meus]

Hi everyone!

I'm using rt 3.8.8 with debian linux 5.0 and postgresql 8.3.12.

When sending e-mail with an attachment to RT, i receive following error 
message:
"Ticket could not be created due to an internal error"

After some investigation i discovered, that this error occurs only, when 
attachment name contains iso-8859-1 letters and mail is sent using 
iso-5589-1 encoding. Sending the same file with mail encoding utf8 works 
fine.

When error occurs, these messages will be logged into syslog:
Nov 15 22:40:56 rt RT: Converting 'ISO-8859-1' to 'utf-8' for text/plain 
- Subjectless message
Nov 15 22:40:56 rt RT: Guessed encoding: ascii
Nov 15 22:40:56 rt RT: Guessed encoding: ascii
Nov 15 22:40:56 rt RT: Converting 'ascii' to 'utf-8' for text/plain - 
Subjectless message
Nov 15 22:40:56 rt RT: Guessed encoding: ascii
Nov 15 22:40:56 rt RT: Guessed encoding: ascii
Nov 15 22:40:56 rt RT: Mail from user #23 (some at mail.address)
Nov 15 22:40:56 rt RT: About to think about scrips for transaction #349
Nov 15 22:40:56 rt RT: About to think about scrips for transaction #350
Nov 15 22:40:56 rt RT: About to think about scrips for transaction #351
Nov 15 22:40:56 rt RT: About to think about scrips for transaction #352
Nov 15 22:40:56 rt RT: Attachment insert failed: ERROR:  invalid byte 
sequence for encoding "UTF8": 0xf6fcf5e4#012HINT:  This error can also 
happen if the byte sequence does not match the encoding expected by the 
server, which is controlled by "client_encoding". 
(/opt/rt3/bin/../lib/RT/Attachment_Overlay.pm:192)
Nov 15 22:40:56 rt RT: Attachment insert failed: ERROR:  invalid byte 
sequence for encoding "UTF8": 0xf6fcf5e4#012HINT:  This error can also 
happen if the byte sequence does not match the encoding expected by the 
server, which is controlled by "client_encoding". 
(/opt/rt3/bin/../lib/RT/Attachment_Overlay.pm:165)
Nov 15 22:40:56 rt RT: About to think about scrips for transaction #353
Nov 15 22:40:56 rt RT: About to prepare scrips for transaction #353
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x50005f0) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: RT::Scrips=HASH(0x500d580) couldn't load ticket 
34 (/opt/rt3/bin/../lib/RT/Scrips_Overlay.pm:291)
Nov 15 22:40:56 rt RT: Found 5 scrips for TransactionCreate stage with 
applicable type(s) Create
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4fc5008) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66bc0) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4be7000) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66bd8) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x5004930) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x5014810) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66770) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: About to commit scrips for transaction #353
Nov 15 22:40:56 rt RT: Committing scrip #11 on txn #353 of ticket #
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x501c8a8) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: HasRight called with no valid object 
(/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
Nov 15 22:40:56 rt RT: Tried to load a bogus ticket id: ''
Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4ff5840) tried to load a bogus 
ticket: 34
Nov 15 22:40:56 rt RT: Ticket couldn't be created: 
(/opt/rt3/bin/../lib/RT/Ticket_Overlay.pm:679)
Nov 15 22:40:56 rt RT: Ticket could not be created due to an internal 
error (/opt/rt3/bin/../lib/RT/Interface/Email.pm:244)

There are also the "invalid byte sequence for encoding "UTF8": 
0xf6fcf5e4" messages in the Postgresql log.

I understand(correct me if i'm wrong) that the cause of the problem is 
that RT is trying to insert into UTF8 database something that is not 
UTF8. And the reason of that is that for some reason RT does guess' the 
attachments encoding wrong. Allthough the log says "Subjectless 
message", the subject line of the message is not empty.
Does anyone has any idea why is RT guessing the encoding(wrong), 
allthough it sees that encoding is ISO-8859-1? Is there any way i can 
change this behavior?
I tried to fiddle with the
Set(@EmailInputEncodings, qw(utf-8 iso-8859-1 us-ascii)); option, but so 
far no luck.


Searching google did'nt give anything useful so far.

Any help would be much appreciated.


all the best,
Joosep Meus