[Rt-commit] rt branch, 4.4/fix-importer-encoding-for-pg, created. rt-4.4.3-17-g3b20d15fc

[? sunnavy]

The branch, 4.4/fix-importer-encoding-for-pg has been created
        at  3b20d15fc798d2b3fd9c5667a074aa7cb22e14d9 (commit)

- Log -----------------------------------------------------------------
commit 3b20d15fc798d2b3fd9c5667a074aa7cb22e14d9
Author: sunnavy <sunnavy at bestpractical.com>
Date:   Tue Jul 3 03:26:36 2018 +0800

    Always pass UTF-8 decoded data to Create method for rt-importer
    
    MySQL and Pg are different: serializer generates UTF-8 encoded bytes for
    MySQL and UTF-8 decoded string for Pg, respectively. This is not a
    problem if you stick to the same database, but when you try to migrate
    an RT db from MySQL to Pg, encoding issues could happen because Pg
    expects UTF-8 decoded values instead of encoded ones. MySQL on the other
    hand, isn't that picky.
    
    This commit fixes this particular issue by always passing UTF-8 decoded
    string to Create.

diff --git a/lib/RT/Migrate/Importer.pm b/lib/RT/Migrate/Importer.pm
index 6eef04532..7a04bffef 100644
--- a/lib/RT/Migrate/Importer.pm
+++ b/lib/RT/Migrate/Importer.pm
@@ -319,6 +319,15 @@ sub Create {
     my ($id, $msg) = eval {
         # catch and rethrow on the outside so we can provide more info
         local $SIG{__DIE__};
+
+        # Make sure data passed to Create are UTF-8 decoded. Without this,
+        # data could be be wrongly encoded on Pg.
+        for my $field ( keys %$data ) {
+            if ( $data->{$field} && !utf8::is_utf8( $data->{$field} ) ) {
+                utf8::decode( $data->{$field} );
+            }
+        }
+
         $obj->DBIx::SearchBuilder::Record::Create(
             %{$data}
         );

-----------------------------------------------------------------------