[rt-users] rt-mailgate breaks when charset isn't UTF-8

[Jamie Wilkinson]

Hi,

RT version 3.0.1

We've got some tickets being created from mail that is created from a web
interface.  The web interface sends the mail with a charset of iso-8859-1.

What's happening is that the mail gets to rt-mailgate, and the ticket gets
created, but the message body doesn't get inserted into the database.  The
template that gets sent to the queue admincc watchers shows all the ticket
data is there, but there is no body and no subject.  This only occurs for a
couple of mails, where the hyphen appears as 

So, we tracked down the problem to rt-mailgate; it looks like rt-mailgate is
assuming everything is UTF-8 encoding, and breaking when it sees a character
that shouldn't exist.

We've worked around the problem by creating a shell script wrapper that
assumes the input encoding is iso-8859-1 and converts to UTF-8, also
changing the LANG to C.

Here's the script:
#!/bin/sh
#
# rt-mailgate expects input in UTF-8 and bombs out if input is not valid

LANG=C
MAILGATE=/opt/rt3/bin/rt-mailgate
ICONV=/usr/bin/iconv

export LANG

exec $ICONV -f iso8859-1 -t UTF-8 | $MAILGATE "$@"

The problem is that again, non-iso-8859-1 emails will get corrupted, eg
UTF-8 mail using non-ASCII characters.

So, I'd suggest that rt-mailgate shouldn't assume anything about the
encoding, then look at the mail headers to decide.  I think you should
default to an octet-stream if nothing can be found, or US-ASCII (which I
believe is the de-facto standard encoding when not explicit).

By the way, I was also going to submit this to the rt3 bugs address, but I
couldn't find it listed on the rt website.

-- 
jaq at spacepants.org                           http://spacepants.org/jaq.gpg