[rt-users] attachment encoding causes internal server error

[Joosep Meus]

Hi again!

I should probably clarify that problem is related to iso-8859-1 special 
characters. Exapmle filename i used for testing was "öüõä.txt". 
Attachments, that do not contain those characters will not cause the 
problem.

All the best,
Joosep MEus

On 11/15/2010 11:32 PM, Joosep Meus wrote:
> Hi everyone!
>
> I'm using rt 3.8.8 with debian linux 5.0 and postgresql 8.3.12.
>
> When sending e-mail with an attachment to RT, i receive following error
> message:
> "Ticket could not be created due to an internal error"
>
> After some investigation i discovered, that this error occurs only, when
> attachment name contains iso-8859-1 letters and mail is sent using
> iso-5589-1 encoding. Sending the same file with mail encoding utf8 works
> fine.
>
> When error occurs, these messages will be logged into syslog:
> Nov 15 22:40:56 rt RT: Converting 'ISO-8859-1' to 'utf-8' for text/plain
> - Subjectless message
> Nov 15 22:40:56 rt RT: Guessed encoding: ascii
> Nov 15 22:40:56 rt RT: Guessed encoding: ascii
> Nov 15 22:40:56 rt RT: Converting 'ascii' to 'utf-8' for text/plain -
> Subjectless message
> Nov 15 22:40:56 rt RT: Guessed encoding: ascii
> Nov 15 22:40:56 rt RT: Guessed encoding: ascii
> Nov 15 22:40:56 rt RT: Mail from user #23 (some at mail.address)
> Nov 15 22:40:56 rt RT: About to think about scrips for transaction #349
> Nov 15 22:40:56 rt RT: About to think about scrips for transaction #350
> Nov 15 22:40:56 rt RT: About to think about scrips for transaction #351
> Nov 15 22:40:56 rt RT: About to think about scrips for transaction #352
> Nov 15 22:40:56 rt RT: Attachment insert failed: ERROR: invalid byte
> sequence for encoding "UTF8": 0xf6fcf5e4#012HINT: This error can also
> happen if the byte sequence does not match the encoding expected by the
> server, which is controlled by "client_encoding".
> (/opt/rt3/bin/../lib/RT/Attachment_Overlay.pm:192)
> Nov 15 22:40:56 rt RT: Attachment insert failed: ERROR: invalid byte
> sequence for encoding "UTF8": 0xf6fcf5e4#012HINT: This error can also
> happen if the byte sequence does not match the encoding expected by the
> server, which is controlled by "client_encoding".
> (/opt/rt3/bin/../lib/RT/Attachment_Overlay.pm:165)
> Nov 15 22:40:56 rt RT: About to think about scrips for transaction #353
> Nov 15 22:40:56 rt RT: About to prepare scrips for transaction #353
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x50005f0) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: RT::Scrips=HASH(0x500d580) couldn't load ticket
> 34 (/opt/rt3/bin/../lib/RT/Scrips_Overlay.pm:291)
> Nov 15 22:40:56 rt RT: Found 5 scrips for TransactionCreate stage with
> applicable type(s) Create
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4fc5008) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66bc0) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4be7000) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66bd8) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x5004930) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x5014810) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4f66770) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: About to commit scrips for transaction #353
> Nov 15 22:40:56 rt RT: Committing scrip #11 on txn #353 of ticket #
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x501c8a8) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: HasRight called with no valid object
> (/opt/rt3/bin/../lib/RT/Principal_Overlay.pm:322)
> Nov 15 22:40:56 rt RT: Tried to load a bogus ticket id: ''
> Nov 15 22:40:56 rt RT: RT::Ticket=HASH(0x4ff5840) tried to load a bogus
> ticket: 34
> Nov 15 22:40:56 rt RT: Ticket couldn't be created:
> (/opt/rt3/bin/../lib/RT/Ticket_Overlay.pm:679)
> Nov 15 22:40:56 rt RT: Ticket could not be created due to an internal
> error (/opt/rt3/bin/../lib/RT/Interface/Email.pm:244)
>
> There are also the "invalid byte sequence for encoding "UTF8":
> 0xf6fcf5e4" messages in the Postgresql log.
>
> I understand(correct me if i'm wrong) that the cause of the problem is
> that RT is trying to insert into UTF8 database something that is not
> UTF8. And the reason of that is that for some reason RT does guess' the
> attachments encoding wrong. Allthough the log says "Subjectless
> message", the subject line of the message is not empty.
> Does anyone has any idea why is RT guessing the encoding(wrong),
> allthough it sees that encoding is ISO-8859-1? Is there any way i can
> change this behavior?
> I tried to fiddle with the
> Set(@EmailInputEncodings, qw(utf-8 iso-8859-1 us-ascii)); option, but so
> far no luck.
>
>
> Searching google did'nt give anything useful so far.
>
> Any help would be much appreciated.
>
>
> all the best,
> Joosep Meus
>