<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body wsmode="reply" bgcolor="#FFFFFF" text="#000000">
<p>Hi all,</p>
<p>I finally resolved the issue with support from RT engineers. So
big thanks to them.<br>
I'm posting the fix, if someone will be interested (maybe in the
future), so it can be found in list archive.</p>
<p>Here is answer from RT engineers:</p>
<pre wrap=""><i>We use Net::LDAP and there is an option called 'raw' that might properly convert the incoming content to utf8. That's the first thing to try since we pass parameters through to Net::LDAP and you can put it right in the config file.
</i><i><a class="moz-txt-link-freetext" href="https://metacpan.org/pod/distribution/perl-ldap/lib/Net/LDAP.pod">https://metacpan.org/pod/distribution/perl-ldap/lib/Net/LDAP.pod</a></i><i>
However, there is likely another bit of code we need to add to RT to be explicit about the incoming text and treat it as utf8 when told to do so.
We can file it as a bug, or provide some commercial assistance if you are interested.
</i></pre>
So I add <br>
<pre wrap="">raw => qr/(?i:^jpegPhoto|;binary)/
</pre>
as net_ldap_args parameter in RT_SiteConfig.pm.<br>
<br>
Now it is all working fine, the names are imported correctly from
LDAP (MS AD, LDAP protocol version 3).<br>
I also suggested to add information about raw option with example to
RT docs.<br>
<br>
Best regards<br>
Jan Burian<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 11.10.2016 11:51, Jan Burian wrote:<br>
</div>
<blockquote cite="mid:ac3fb7f4-56fa-b12c-f31e-ccf9c3e3a009@vsup.cz"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
Hi Bill,<br>
<br>
thank you for your response. Sry not to mention our database.<br>
We use PostreSQL.<br>
After I wrote first email a also checked encoding in database.<br>
<br>
The database was with following parameters:<br>
Name | Encoding | Collate | Ctype <br>
-------------+-------------+-----------------+------------------<br>
rt4 | UTF8 | en_US.UTF-8 | en_US.UTF-8<br>
<br>
1) I dump database with UTF-8 encoding parameter.<br>
2) Then I drop the databases.<br>
3) Create new database with following parameters:<br>
<br>
Name | Encoding | Collate | Ctype <br>
-------------+-------------+-----------------+------------------<br>
rt4 | UTF8 | cs_CZ.UTF-8 | cs_CZ.UTF-8<br>
<br>
4) And then import database from dump.<br>
<br>
But after that change names are loading from LDAP still with bad
characters :-/.<br>
<br>
When the user writes first email to queue, then is also
autocreated as unprivileged. If he/she was his/her name in From
header, then is used as RealName RT attribute. But in this case is
his/her name saved correctly.<br>
<br>
<b>Example from the log - autocreated from LDAP:</b><br>
[6937] [Tue Sep 27 15:59:25 2016] [info]:
RT::User::CanonicalizeUserInfoFromExternalAuth returning Disabled:
, EmailAddress: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated" href="mailto:novak@vsup.cz">novak@vsup.cz</a>,
Gecos: novak, Name: novak, Privileged: 1, RealName: Matouš
Novák, WorkPhone: (/opt/rt4/sbin/../lib/RT/User.pm:811)<br>
[6937] [Tue Sep 27 15:59:25 2016] [info]: Autocreated external
user novak ( 61 )
(/opt/rt4/sbin/../lib/RT/Authen/ExternalAuth.pm:356)<br>
[6937] [Tue Sep 27 15:59:25 2016] [info]:
RT::Authen::ExternalAuth::LDAP::GetAuth External Auth OK ( My_LDAP
): novak (/opt/rt4/sbin/../lib/RT/Authen/ExternalAuth/LDAP.pm:348)<br>
[6937] [Tue Sep 27 15:59:26 2016] [info]:
RT::User::CanonicalizeUserInfoFromExternalAuth returning
EmailAddress: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated" href="mailto:novak@vsup.cz">novak@vsup.cz</a>,
Name: novak, <b>RealName: Matouš Novák</b>, WorkPhone:
(/opt/rt4/sbin/../lib/RT/User.pm:811)<br>
<b><br>
</b><b>Example from the log - autocreated from email:</b><br>
[6026] [Mon Oct 10 06:26:02 2016] [info]:
RT::User::CanonicalizeUserInfoFromExternalAuth returning Comments:
Autocreated on ticket submission, Disabled: , EmailAddress: <a
moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:Tereza.Skvarova@seznam.cz">Tereza.Skvarova@seznam.cz</a>,
Name: <a moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:Tereza.Skvarova@seznam.cz">Tereza.Skvarova@seznam.cz</a>,
Privileged: , <b>RealName: Tereza Škvárová</b>
(/opt/rt4/sbin/../lib/RT/User.pm:811)<br>
<br>
Any other ideas?<br>
<br>
Best regards<br>
Jan Burian<br>
<br>
<div class="moz-cite-prefix">On 11.10.2016 05:41, Bill Cole wrote:<br>
</div>
<blockquote
cite="mid:9902590C-3265-4E60-B1FC-54957DB03129@billmail.scconsult.com"
type="cite">On 10 Oct 2016, at 16:26, Jan Burian wrote: <br>
<br>
<blockquote type="cite">Hi all, <br>
<br>
we have RT 4.4.0 on CentOS 7 and Perl v5.22.1. And we are
starting to <br>
use RT in production. <br>
<br>
We configured RT to authenticate users via LDAP <br>
(RT::Authen::ExternalAuth::LDAP). Our LDAP server is MS AD
(Win 2008 R2). <br>
</blockquote>
[...] <br>
<blockquote type="cite">Authentication is working fine. Users
can log in, if the user doesn't <br>
exist in RT the account is autocreated. All the configured
attributes <br>
are transferred. <br>
</blockquote>
<br>
This is a strong sign that the LDAP part is working correctly.
If the LDAP server (AD) and client (Perl's Net::LDAP module) are
using mismatched encodings, it is likely to show up in
authentication failures due to incompatible encodings of the
same (logical) characters that 8-bit encodings assign to byte
values 0x80-0xff. <br>
<br>
Fortunately, it is somewhere between arcane and impossible to
make Net::LDAP use anything other than UTF-8. There's *probably*
some way to make it do T.61 for ancient-history compatibility,
but that's mostly pointless. <br>
<br>
[...] <br>
<blockquote type="cite">We had similar problem with Moodle. When
we configured Moodle against <br>
Active Directory and set cp1250 encoding, then it was doing
exactly same <br>
thing. After we changed encoding for LDAP connector to utf-8
then the <br>
names was <br>
corrected. <br>
</blockquote>
<br>
Which makes sense: LDAP v3 by default uses UTF-8 and you have a
modern system with a mature LDAP client. I know of no way to
configure a CentOS 7/Perl 5.22 system such that the LDAP
interaction with an AD LDAP server talking UTF-8 would be the
source of this sort of encoding conflict. I'm mildly surprised
that anything talking LDAPv3 can be made to use cp1250 encoding,
but I suppose Microsoft makes their own rules to go along with
their own unique code pages. <br>
<br>
[...] <br>
<blockquote type="cite">Also I red thath MS AD in LDAP protocol
version 3 returns any string to <br>
LDAP client in utf-8 encoding. <br>
I really don't know where could be a problem. <br>
</blockquote>
<br>
The most likely place is in your database. I'm guessing that you
are using MySQL, which defaults to latin1 encoding. When you
store a UTF-8 string into a latin1 table, it breaks any
multi-byte characters into 2 or 3 characters, but the right bits
are still there. This issue has come up a few times on this list
over the past decade and I think Best Practical has documented
how to safely convert a RT database with that sort of problem
from latin1 to utf8. It is probably worth looking through their
docs (possibly one of the UPGRADING* files?) and the RT Wiki for
a solution. I expect it could be done with a binary dump of the
database, altering of any latin1 tables to use utf8, and a
re-import of the binary dump. I'm not enough of a MySQL expert
to detail that process (I generally use Postgres where
possible.) <br>
--------- <br>
RT 4.4 and RTIR training sessions, and a new workshop day! <a
moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://bestpractical.com/training">https://bestpractical.com/training</a>
<br>
* Boston - October 24-26 <br>
* Los Angeles - Q1 2017 <br>
</blockquote>
<br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">---------
RT 4.4 and RTIR training sessions, and a new workshop day! <a class="moz-txt-link-freetext" href="https://bestpractical.com/training">https://bestpractical.com/training</a>
* Boston - October 24-26
* Los Angeles - Q1 2017</pre>
</blockquote>
<br>
<br></br>
</body>
</html>