[Rt-commit] rt branch, 4.2/less-qp, created. rt-4.2.10-134-g373db21

[Alex Vandiver]

The branch, 4.2/less-qp has been created
        at  373db219e41bdbe5a001694735392c1e51156564 (commit)

- Log -----------------------------------------------------------------
commit 373db219e41bdbe5a001694735392c1e51156564
Author: Alex Vandiver <alexmv at bestpractical.com>
Date:   Thu Mar 5 14:31:04 2015 -0500

    Only store text as quoted-printable when the bytes not invalid UTF-8
    
    The existing check for Encode::is_utf8() checks if the UTF8 flag is on
    _and_ that the internal representation is consistent.  Since the UTF8
    flag is almost certainly off, as _EncodeLOB takes bytes, not characters,
    this causes the quoted-printable encoding to be chosen even for byte
    sequences which are valid UTF-8.
    
    Switch to checking if the byte sequence is valid UTF-8; if so, upgrade
    it to characters and store it verbatim.  If it is not, then store it as
    quoted-printable as before.  This increases the set of content which is
    stored with encoding "none", which eases direct comprehension of the
    data.

diff --git a/lib/RT/Record.pm b/lib/RT/Record.pm
index d65ecab..b585d0c 100644
--- a/lib/RT/Record.pm
+++ b/lib/RT/Record.pm
@@ -787,7 +787,6 @@ sub _EncodeLOB {
 
     #if the current attachment contains nulls and the
     #database doesn't support embedded nulls
-
     if ( ( !$RT::Handle->BinarySafeBLOBs ) && ( $Body =~ /\x00/ ) ) {
 
         # set a flag telling us to mimencode the attachment
@@ -798,9 +797,16 @@ sub _EncodeLOB {
         $MaxSize = $MaxSize * 3 / 4;
     # Some databases (postgres) can't handle non-utf8 data
     } elsif (    !$RT::Handle->BinarySafeBLOBs
-              && $Body =~ /\P{ASCII}/
-              && !Encode::is_utf8( $Body, 1 ) ) {
-          $ContentEncoding = 'quoted-printable';
+              && $Body =~ /\P{ASCII}/ ) {
+        local $@;
+        # If it's valid UTF-8 data, store it as characters with no
+        # transfer-encoding; otherwise QP it
+        my $chars = eval { Encode::decode( "UTF-8", $Body, Encode::FB_CROAK | Encode::LEAVE_SRC ) };
+        if (not $@) {
+            $Body = $chars;
+        } else {
+            $ContentEncoding = 'quoted-printable';
+        }
     }
 
     #if the attachment is larger than the maximum size

-----------------------------------------------------------------------