From jlong at messiah.edu Fri Nov 2 15:47:35 2007
From: jlong at messiah.edu (Jason Long)
Date: Fri Nov 2 15:49:58 2007
Subject: [Rt-devel] [PATCH 3.7-RTIR-RELENG 1/3] xhtml: tag in wrong
place
Message-ID: <20071102194735.22391.20100.stgit@carina.messiah.edu>
The opening tag came before the IF statement,
and the closing tag was inside the IF statement.
This patch moves the opening tag inside the IF statement,
to match the closing tag.
---
html/Search/Elements/EditSearches | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
(Patch is attached)
-------------- next part --------------
diff --git a/html/Search/Elements/EditSearches b/html/Search/Elements/EditSearches
index eda59c5..6e16844 100644
--- a/html/Search/Elements/EditSearches
+++ b/html/Search/Elements/EditSearches
@@ -60,8 +60,8 @@
<&|/l&>Description:
% if ($Id ne 'new') {
+
% if ($Dirty) {
That's two "width" attributes, one of which is getting unnecessarily
HTML-encoded. This patch is a possible fix.
---
html/Elements/MyRT | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
(Patch is attached)
-------------- next part --------------
diff --git a/html/Elements/MyRT b/html/Elements/MyRT
index 76dbc5d..91d24eb 100644
--- a/html/Elements/MyRT
+++ b/html/Elements/MyRT
@@ -48,7 +48,7 @@
->
+ >
% $show_cb->($_) foreach @$body;
From jesse at bestpractical.com Mon Nov 5 11:23:47 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Mon Nov 5 11:24:09 2007
Subject: [Rt-devel] [PATCH 3.7-RTIR-RELENG 3/3] xhtml: multiple width=
attributes in MyRT
In-Reply-To: <20071102194746.22391.38387.stgit@carina.messiah.edu>
References: <20071102194735.22391.20100.stgit@carina.messiah.edu>
<20071102194746.22391.38387.stgit@carina.messiah.edu>
Message-ID: <472F4393.2080100@bestpractical.com>
Thanks. Applied
From jesse at bestpractical.com Mon Nov 5 11:23:57 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Mon Nov 5 11:24:20 2007
Subject: [Rt-devel] [PATCH 3.7-RTIR-RELENG 2/3] xhtml: cannot have
closing tag
In-Reply-To: <20071102194740.22391.38037.stgit@carina.messiah.edu>
References: <20071102194735.22391.20100.stgit@carina.messiah.edu>
<20071102194740.22391.38037.stgit@carina.messiah.edu>
Message-ID: <472F439D.3080002@bestpractical.com>
Thanks. Applied
From sturner at MIT.EDU Tue Nov 6 13:55:29 2007
From: sturner at MIT.EDU (Stephen Turner)
Date: Tue Nov 6 13:55:45 2007
Subject: [Rt-devel] RT 3.6.5 RT::SearchBuilder bug report and patch
Message-ID: <6.2.3.4.2.20071106134736.049b13a8@po14.mit.edu>
Hello,
In tracking down an apparent difference between RT 3.4 and 3.6, I've
found what appears to be a bug, or at least an inconsistency in 3.6.
It's causing several of my perl scripts that use the RT API to fail.
The difference between 3.4 & 3.6 is that in 3.6, objects have to have
CurrentUser actually be a RT::CurrentUser object. 3.4 allowed
CurrentUser to be a RT::User or RT:CurrentUser.
The inconsistency (in 3.6) is that collection objects behave
differently from single-row objects:
With an RT::User object called $UserObj, if you do:
my $tkt = RT::Ticket->new ($UserObj);
then $tkt->CurrentUser will automatically be set to an
RT::CurrentUser object for you, and life is good.
However, if you do:
my $tkts = RT::Tickets->new ($UserObj);
you'll get this error:
[err]: RT::Tickets=HASH(0xaeaf1b4) was created without a CurrentUser.
Any RT object which is subclass of RT::Base must be created with a
RT::CurrentUser or a RT::User obejct as the first argument.
(/var/local/rt364/lib/RT/Base.pm:107)
[crit]: RT::Tickets=HASH(0xaeaf1b4) was created without a CurrentUser
at /var/local/rt364/lib/RT/SearchBuilder.pm line 85
As an aside, the first error message implies that using an RT::User
object is OK whereas it isn't.
I'd like to suggest a patch to RT/SearchBuilder.pm that will do the
right thing if you supply an RT::User object to "new" on a
collection. The patch is based on RT 3.6.5. (This patch also has the
effect of making the error message correct. Well, except for "obejct"! ).
Steve
--- SearchBuilder.pm.base 2007-11-06 13:40:03.000000000 -0500
+++ SearchBuilder.pm 2007-11-06 13:39:20.000000000 -0500
@@ -80,7 +80,7 @@
my $self = shift;
$self->{'user'} = shift;
- unless(defined($self->CurrentUser)) {
+ unless(defined($self->CurrentUser( $self->{'user'} ))) {
use Carp;
Carp::confess("$self was created without a CurrentUser");
$RT::Logger->err("$self was created without a CurrentUser");
Stephen Turner
Senior Programmer/Analyst - Client Support Services
MIT Information Services and Technology (IS&T)
From jesse at bestpractical.com Tue Nov 6 14:25:05 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Tue Nov 6 14:25:53 2007
Subject: [Rt-devel] RT 3.6.5 RT::SearchBuilder bug report and patch
In-Reply-To: <6.2.3.4.2.20071106134736.049b13a8@po14.mit.edu>
References: <6.2.3.4.2.20071106134736.049b13a8@po14.mit.edu>
Message-ID: <20071106192504.GB20484@bestpractical.com>
On Tue, Nov 06, 2007 at 01:55:29PM -0500, Stephen Turner wrote:
> Hello,
>
> In tracking down an apparent difference between RT 3.4 and 3.6, I've
> found what appears to be a bug, or at least an inconsistency in 3.6.
> It's causing several of my perl scripts that use the RT API to fail.
>
> The difference between 3.4 & 3.6 is that in 3.6, objects have to have
> CurrentUser actually be a RT::CurrentUser object. 3.4 allowed
> CurrentUser to be a RT::User or RT:CurrentUser.
It's actually something we tightened up. In RT 3.4, the loose behaviour
caused a lot of subtle bugs further down the chain.
>
> The inconsistency (in 3.6) is that collection objects behave
> differently from single-row objects:
That's not good.
>
> I'd like to suggest a patch to RT/SearchBuilder.pm that will do the
> right thing if you supply an RT::User object to "new" on a
> collection. The patch is based on RT 3.6.5. (This patch also has the
> effect of making the error message correct. Well, except for "obejct"! ).
That seems reasonable, but will break RT 3.7 as RT::CurrentUser now ISA
RT::User. Ruz, thoughts?
> Steve
>
>
> --- SearchBuilder.pm.base 2007-11-06 13:40:03.000000000 -0500
> +++ SearchBuilder.pm 2007-11-06 13:39:20.000000000 -0500
> @@ -80,7 +80,7 @@
> my $self = shift;
>
> $self->{'user'} = shift;
> - unless(defined($self->CurrentUser)) {
> + unless(defined($self->CurrentUser( $self->{'user'} ))) {
> use Carp;
> Carp::confess("$self was created without a CurrentUser");
> $RT::Logger->err("$self was created without a CurrentUser");
>
>
>
>
> Stephen Turner
> Senior Programmer/Analyst - Client Support Services
> MIT Information Services and Technology (IS&T)
>
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
>
> SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:
>
> If you sign up for a new RT support contract before December 31, we'll take
> up to 20 percent off the price. This sale won't last long, so get in touch
> today. Email us at sales@bestpractical.com or call us at +1 617 812 0745.
>
--
From lekefasola at gmail.com Thu Nov 8 05:27:51 2007
From: lekefasola at gmail.com (Leke Fasola)
Date: Thu Nov 8 05:29:51 2007
Subject: [Rt-devel] Enable communication btw RT-Mailgate and MS Exchange
Message-ID: <90055900711080227y5378b553i46afb0e8f73dad78@mail.gmail.com>
Hello,
I use MS exchange in my Organisation. I just install RT and its working.
can somebpdy show me step-step on how to configure communication betwwen
RT-MAILGATE and MS Exchange
Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071108/2f6f3bdc/attachment.htm
From nicolas.chuche at barna.be Thu Nov 8 16:58:40 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Thu Nov 8 16:58:46 2007
Subject: [Rt-devel] RTx::EmailCompletion LDAP configuration
Message-ID: <742e684f0711081358i4fa1ada1y74dbcf02fd8c140f@mail.gmail.com>
Hello,
I'm working on adding completion against an LDAP directory in
RTx::EmailCompletion.
I need to add some configuration like the ldap server, ldap filter,
ldap base, ...
The LDAP authentification overlay already use RT::LdapServer, RT::LdapBase, ...
I wonder if I should use the same configuration or use my own vars
like RT::EmailCompletionLdapServer.
An Idea to help me ?
From ruz at bestpractical.com Thu Nov 8 18:05:27 2007
From: ruz at bestpractical.com (Ruslan Zakirov)
Date: Thu Nov 8 18:05:35 2007
Subject: [Rt-devel] [PATCH 3.7-RTIR-RELENG 1/3] xhtml: tag in wrong
place
In-Reply-To: <20071102194735.22391.20100.stgit@carina.messiah.edu>
References: <20071102194735.22391.20100.stgit@carina.messiah.edu>
Message-ID: <589c94400711081505t663e29c2v1d087fad161660b2@mail.gmail.com>
thanks, applied to 3.6, will merge into 3.7.
On Nov 2, 2007 10:47 PM, Jason Long wrote:
> The opening tag came before the IF statement,
> and the closing tag was inside the IF statement.
>
> This patch moves the opening tag inside the IF statement,
> to match the closing tag.
> ---
>
> html/Search/Elements/EditSearches | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> (Patch is attached)
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
>
> SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:
>
> If you sign up for a new RT support contract before December 31, we'll take
> up to 20 percent off the price. This sale won't last long, so get in touch today.
> Email us at sales@bestpractical.com or call us at +1 617 812 0745.
>
--
Best regards, Ruslan.
From ruz at bestpractical.com Thu Nov 8 18:08:51 2007
From: ruz at bestpractical.com (Ruslan Zakirov)
Date: Thu Nov 8 18:08:59 2007
Subject: [Rt-devel] [PATCH 3.7-RTIR-RELENG] xhtml: extra closing tag
in Admin/Groups/Members.html
In-Reply-To: <20071102162308.15237.25640.stgit@carina.messiah.edu>
References: <20071102162308.15237.25640.stgit@carina.messiah.edu>
Message-ID: <589c94400711081508y7816cf9cu280f1cec31854f6b@mail.gmail.com>
Thanks, applied to 3.6
On Nov 2, 2007 7:23 PM, Jason Long wrote:
> In this case the was opened inside an IF block,
> but closed outside the IF block.
>
> Then I saw that User/Groups/Members.html had the same problem.
> This patch fixes both.
> ---
>
> html/Admin/Groups/Members.html | 2 +-
> html/User/Groups/Members.html | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> (Patch is attached)
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
>
> SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:
>
> If you sign up for a new RT support contract before December 31, we'll take
> up to 20 percent off the price. This sale won't last long, so get in touch today.
> Email us at sales@bestpractical.com or call us at +1 617 812 0745.
>
--
Best regards, Ruslan.
From spiller at spring.de Fri Nov 9 09:31:59 2007
From: spiller at spring.de (Marianne Spiller)
Date: Fri Nov 9 09:32:16 2007
Subject: [Rt-devel] Problem with dumpfile-to-rt-3.0
Message-ID: <47346F5F.8030306@spring.de>
Hi there,
I have to migrate a RT2 (webmux version 2.0.1) to an actual revision,
but I'm running into serious problems using dumpfile-to-rt-3.0 (version
RT-Extension-RT2toRT3-1.26_01).
The old host: SuSE Linux 6.1, mysql 3.23.40, apache 1.3.20, perl 5.6.1
The new host: Fedora Core 6, mysql 5.0.27, apache 2.2.6, perl 5.8.8
I got a dump from the old rt2 mysql database (via mysqldump) and
transferred it to the new host; then I got a tarball from the old
'/opt/rt2' and transferred it, too.
After I unpacked on the new host, I made a 'perl rt-2.0-to-dumpfile
/opt/RTmigration' whereas my rt2.sql database dump was stored in
'/opt/RTmigration'. It seemed to work fine; these are the contents of
this directory:
-rw-r--r-- 1 root root 244497 Nov 9 09:14 metadata
-rw-r--r-- 1 mspiller users 63809998 Nov 8 12:25 rt2.sql
drwxr-x--x 2 root root 28672 Nov 9 09:15 tickets-0
drwxr-x--x 2 root root 28672 Nov 9 09:16 tickets-1000
drwxr-x--x 2 root root 28672 Nov 9 09:18 tickets-2000
drwxr-x--x 2 root root 28672 Nov 9 09:19 tickets-3000
drwxr-x--x 2 root root 24576 Nov 9 09:21 tickets-4000
drwxr-x--x 2 root root 12288 Nov 9 09:21 tickets-5000
Then I installed several versions of RT3 and tried the 'dumpfile'
command, but without any success; the installation process goes well,
but importing data fails (see [1]).
Can you please give me a hint how to get it work?
What am I doing wrong?
Thanks in advance,
Marianne
[1] Executing this results in the following output:
$ perl dumpfile-to-rt-3.0 /opt/RTmigration.WEG/
"my" variable $msg masks earlier declaration in same scope at
/opt/rt3/lib/RT/Ticket_Overlay.pm line 3139.
Name "RT::LogStackTraces" used only once: possible typo at
dumpfile-to-rt-3.0 line 64.
Importing scrips
Importing users
Loading user Unnamed user 1
Creating new user for Unnamed user 1
[Fri Nov 9 13:59:11 2007] [error]: Could not create a new user -
(/opt/rt3/lib/RT/User_Overlay.pm:282)
Failed to create user for$VAR1 = {
'CryptedPassword' => undef,
'Name' => 'Unnamed user 1'
};
[... for about 300 users and so on...]
Importing groups
g
Couldn't find user with RT2 userid 10 - not adding to
g
Couldn't find user with RT2 userid 7 - not adding to
Couldn't find user with RT2 userid 12 - not adding to
Couldn't find user with RT2 userid 15 - not adding to
Couldn't find user with RT2 userid 276 - not adding to
g
Couldn't find user with RT2 userid 9 - not adding to
Couldn't find user with RT2 userid 11 - not adding to
Couldn't find user with RT2 userid 276 - not adding to
g
Couldn't find user with RT2 userid 4 - not adding to
Couldn't find user with RT2 userid 8 - not adding to
g
Couldn't find user with RT2 userid 4 - not adding to
Couldn't find user with RT2 userid 6 - not adding to
Couldn't find user with RT2 userid 8 - not adding to
Couldn't find user with RT2 userid 11 - not adding to
Couldn't find user with RT2 userid 101 - not adding to
Importing rights
Couldn't load principal to grant them Watch on queue
$VAR1 = {
'PrincipalType' => 'User',
'PrincipalId' => '4',
'Name' => 'Watch'
};
Couldn't load principal to grant them WatchAsAdminCc on queue
$VAR1 = {
'PrincipalType' => 'User',
'PrincipalId' => '4',
'Name' => 'WatchAsAdminCc'
};
Couldn't create TICKET: Could not create ticket. Queue not set$VAR1 = {
'Requestor' => [
'daniel.haha@blah.de'
],
'Owner' => undef,
'LastUpdatedBy' => undef,
'_RecordTransaction' => '0',
'id' => '14981',
'Queue' => 'IVWBOX',
'Creator' => undef
};
[Fri Nov 9 14:00:33 2007] [crit]: Died at dumpfile-to-rt-3.0 line 697.
(/opt/rt3/lib/RT.pm:254)
From MarkRoedel at letu.edu Fri Nov 9 15:43:49 2007
From: MarkRoedel at letu.edu (Roedel, Mark)
Date: Fri Nov 9 15:56:02 2007
Subject: [Rt-devel] Ticket "Owner" data duplication?
Message-ID: <0DD6BC9ABC0D2747AF8589AEA9722A4001431DC0@rachael-reborn.letnet.net>
In poking around with the database, it looks like a ticket's "Owner" is
stored in two places:
(1) in the "Tickets" table
(2) in a role-based group membership in the "Groups" and
"GroupMembers" tables
So...just out of curiosity, is it correct that the same data is being
stored in both locations (If so, is there a specific reason?), or is
there a chance they could differ? (Under what circumstances?)
--
Mark Roedel
Senior Programmer / Analyst
LeTourneau University
From nicolas.chuche at barna.be Sun Nov 11 13:57:12 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Sun Nov 11 13:57:20 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
Message-ID: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
Hello,
I've added LDAP autocompletion in RTx::EmailCompletion. As I won't use
this feature myself, I need some testers before releasing it next
week.
You can find it here : http://gaspard.mine.nu/RTx-EmailCompletion-0.03.tar.gz
Thanks
From MarkRoedel at letu.edu Mon Nov 12 11:47:05 2007
From: MarkRoedel at letu.edu (Roedel, Mark)
Date: Mon Nov 12 11:47:19 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
Message-ID: <0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
Seems to be working all right here so far -- I did make a couple of
minor tweaks (diff is attached):
* Added EmailCompletionLdapUser and EmailCompletionLdapPass
configuration variables to support LDAP servers that don't allow
anonymous binding
* Sorted the results of the LDAP search
One possible weakness I can see with my changes is that I don't think
they'll allow for a case where the ldap server used for email
autocompletion allows anonymous binds but the server used for
authentication doesn't. I'm not sure what the most graceful way would
be to handle that scenario while still supporting fallthrough to an
underlying LDAP configuration.
Also...I'm not sure how hard this would be, but for sites that use both
LDAP and RDBM lookups for email completion, I can see it being useful
for the dropdown to indicate where each item originated.
--
Mark Roedel
Senior Programmer / Analyst
LeTourneau University
Longview, Texas USA
-----Original Message-----
From: rt-devel-bounces@lists.bestpractical.com
[mailto:rt-devel-bounces@lists.bestpractical.com] On Behalf Of Nicolas
Chuche
Sent: Sunday, November 11, 2007 12:57 PM
To: rt-devel@lists.bestpractical.com
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
Hello,
I've added LDAP autocompletion in RTx::EmailCompletion. As I won't use
this feature myself, I need some testers before releasing it next
week.
You can find it here :
http://gaspard.mine.nu/RTx-EmailCompletion-0.03.tar.gz
Thanks
-------------- next part --------------
diff -r -u RTx-EmailCompletion-0.03/html/SelfService/Ajax/EmailCompletion RTx-EmailCompletion-0.03-letu/html/SelfService/Ajax/EmailCompletion
--- RTx-EmailCompletion-0.03/html/SelfService/Ajax/EmailCompletion Sun Nov 11 12:42:37 2007
+++ RTx-EmailCompletion-0.03-letu/html/SelfService/Ajax/EmailCompletion Mon Nov 12 10:00:06 2007
@@ -7,6 +7,8 @@
$RT::EmailCompletionSearch ||= 'LIKE';
$RT::EmailCompletionLdapServer ||= $RT::LdapServer;
+$RT::EmailCompletionLdapUser ||= $RT::LdapUser;
+$RT::EmailCompletionLdapPass ||= $RT::LdapPass;
$RT::EmailCompletionLdapBase ||= $RT::LdapBase;
$RT::EmailCompletionLdapFilter ||= $RT::LdapFilter;
$RT::EmailCompletionLdapAttrShow ||= "mail";
diff -r -u RTx-EmailCompletion-0.03/lib/RTx/EmailCompletion/Ldap.pm RTx-EmailCompletion-0.03-letu/lib/RTx/EmailCompletion/Ldap.pm
--- RTx-EmailCompletion-0.03/lib/RTx/EmailCompletion/Ldap.pm Sun Nov 11 10:49:54 2007
+++ RTx-EmailCompletion-0.03-letu/lib/RTx/EmailCompletion/Ldap.pm Mon Nov 12 10:33:33 2007
@@ -16,7 +16,14 @@
my $ldap = new Net::LDAP($RT::EmailCompletionLdapServer);
- my $mesg = $ldap->bind();
+ my $mesg;
+
+ if ($RT::EmailCompletionLdapUser && $RT::EmailCompletionLdapPass) {
+ $mesg = $ldap->bind($RT::EmailCompletionLdapUser, password=>$RT::EmailCompletionLdapPass);
+ } else {
+ $mesg = $ldap->bind();
+ }
+
if ($mesg->code != LDAP_SUCCESS) {
$RT::Logger->crit("Unable to bind to $RT::EmailCompletionLdapServer: ", ldap_error_name($mesg->code), "\n");
return;
@@ -35,7 +42,7 @@
$RT::Logger->crit("Unable to search in LDAP: ", ldap_error_name($mesg->code), "\n");
}
- my @emails = map { $_->get_attribute( $RT::EmailCompletionLdapAttrShow ) } $mesg->entries;
+ my @emails = sort map { $_->get_attribute( $RT::EmailCompletionLdapAttrShow ) } $mesg->entries;
@emails = grep { m/$RT::EmailCompletionUnprivileged/ } @emails
if ref($RT::EmailCompletionUnprivileged) eq 'Regexp' and not $CurrentUser->Privileged();
From nicolas.chuche at barna.be Mon Nov 12 16:39:26 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Mon Nov 12 16:39:36 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
Message-ID: <742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
Hello,
New rc available http://gaspard.mine.nu/RTx-EmailCompletion-0.03rc2.tar.gz
> * Added EmailCompletionLdapUser and EmailCompletionLdapPass
> configuration variables to support LDAP servers that don't allow
> anonymous binding
>
> * Sorted the results of the LDAP search
Ok got it. I've change it a bit and applied it.
> One possible weakness I can see with my changes is that I don't think
> they'll allow for a case where the ldap server used for email
> autocompletion allows anonymous binds but the server used for
> authentication doesn't. I'm not sure what the most graceful way would
> be to handle that scenario while still supporting fallthrough to an
> underlying LDAP configuration.
The best idea I can think of (is it english ?) is to verify that
LdapServer and EmailCompletionLdapServer are the same and if not,
don't fallback to LdapUser/LdapPass. I will do that way for the
moment. If you have a better idea, let me know.
I will keep the fallback for EmailCompletionBase and
EmailCompletionFilter because it seems good for those two.
> Also...I'm not sure how hard this would be, but for sites that use both
> LDAP and RDBM lookups for email completion, I can see it being useful
> for the dropdown to indicate where each item originated.
Quite easy, LDAP emails now have light grey background. If you have a
better idea or a better color, let me know.
By the way, thanks for the patch and for your help.
From MarkRoedel at letu.edu Mon Nov 12 17:09:40 2007
From: MarkRoedel at letu.edu (Roedel, Mark)
Date: Mon Nov 12 17:09:51 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
Message-ID: <0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
One correction:
In lib/RTx/EmailCompletion/Ldap.pm, line 20 currently reads
$ldap->bind($RT::EmailCompletionLdapUser,
$RT::EmailCompletionLdapPass)
And should be
$ldap->bind($RT::EmailCompletionLdapUser,
password=>$RT::EmailCompletionLdapPass)
I'm also not seeing the grey backgrounds, but haven't had time to look
at whether that's a problem with the plugin, or somehow something in our
local setup. I'll hopefully have time to work with it some more
tomorrow.
--
Mark Roedel
Senior Programmer / Analyst
LeTourneau University
-----Original Message-----
From: nchuche@gmail.com [mailto:nchuche@gmail.com] On Behalf Of Nicolas
Chuche
Sent: Monday, November 12, 2007 3:39 PM
To: Roedel, Mark
Cc: rt-devel@lists.bestpractical.com
Subject: Re: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
Hello,
New rc available
http://gaspard.mine.nu/RTx-EmailCompletion-0.03rc2.tar.gz
> * Added EmailCompletionLdapUser and EmailCompletionLdapPass
> configuration variables to support LDAP servers that don't allow
> anonymous binding
>
> * Sorted the results of the LDAP search
Ok got it. I've change it a bit and applied it.
> One possible weakness I can see with my changes is that I don't think
> they'll allow for a case where the ldap server used for email
> autocompletion allows anonymous binds but the server used for
> authentication doesn't. I'm not sure what the most graceful way would
> be to handle that scenario while still supporting fallthrough to an
> underlying LDAP configuration.
The best idea I can think of (is it english ?) is to verify that
LdapServer and EmailCompletionLdapServer are the same and if not,
don't fallback to LdapUser/LdapPass. I will do that way for the
moment. If you have a better idea, let me know.
I will keep the fallback for EmailCompletionBase and
EmailCompletionFilter because it seems good for those two.
> Also...I'm not sure how hard this would be, but for sites that use
both
> LDAP and RDBM lookups for email completion, I can see it being useful
> for the dropdown to indicate where each item originated.
Quite easy, LDAP emails now have light grey background. If you have a
better idea or a better color, let me know.
By the way, thanks for the patch and for your help.
From nicolas.chuche at barna.be Mon Nov 12 17:49:19 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Mon Nov 12 17:49:28 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
Message-ID: <742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
> $ldap->bind($RT::EmailCompletionLdapUser,
> password=>$RT::EmailCompletionLdapPass)
Oups, should have read your patch more carefully !
rc3 available here : http://gaspard.mine.nu/rtx/
> I'm also not seeing the grey backgrounds, but haven't had time to look
> at whether that's a problem with the plugin, or somehow something in our
> local setup. I'll hopefully have time to work with it some more
> tomorrow.
I must admit, I'm not really the css kind of guy, I'm more sysadmin.
It works for me but perhaps by mistake :).
From otto.hirr at olabinc.com Mon Nov 12 17:19:56 2007
From: otto.hirr at olabinc.com (Otto Hirr)
Date: Mon Nov 12 18:04:12 2007
Subject: [Rt-devel] constraints.mysql ... in use by anyone?
Message-ID: <5D1275B9B7994B5B9A4FE7F55420FF5A@olabinc.com>
Is there a reason that the schema creation scripts
do not spec constraints?
Most of the db's seem to have the foreign key constraint
ability, and those that do not, eg sqlite, take it as a no-op,
at least in my understanding.
Would it not be better to move forward with having constraints
defined and be able to drop them for those who don't/can't have them?
There is a file, constraints.mysql, however it is not valid,
ie there are columns that are not defined in all the schemas,
eg EffectiveTicket is defined in schema.Sybase and is used
in constraints.mysql, but is not defined in any of the other
schemas.
If schema.Sybase is old, then it should be dropped or updated.
Is there someone that can "push a button" and generate a
new constraints file?
Regards,
..Otto
From andrew at etc.gen.nz Mon Nov 12 21:48:31 2007
From: andrew at etc.gen.nz (Andrew Ruthven)
Date: Mon Nov 12 22:13:32 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
Message-ID: <1194922111.11583.13.camel@dirk.catalyst.net.nz>
Hi,
On Mon, 2007-11-12 at 23:49 +0100, Nicolas Chuche wrote:
> rc3 available here : http://gaspard.mine.nu/rtx/
Just a minor bug fix to the documentation, EmailCompletionLdapServer
must be set. The documentation says "If not set, RTx::EmailCompletion
will search for LdapServer parameter...", this isn't the case. If
EmailCompletionLdapServer isn't set then LDAP isn't used for
EmailCompletion.
Cheers!
--
Andrew Ruthven, Wellington, New Zealand
At home: andrew@etc.gen.nz | This space intentionally
| left blank.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071113/27f25a2d/attachment.pgp
From nicolas.chuche at barna.be Tue Nov 13 06:07:48 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Tue Nov 13 06:08:01 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <1194922111.11583.13.camel@dirk.catalyst.net.nz>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
<1194922111.11583.13.camel@dirk.catalyst.net.nz>
Message-ID: <742e684f0711130307k6f38c6f5r583fecd5eb2c5cb@mail.gmail.com>
> Just a minor bug fix to the documentation, EmailCompletionLdapServer
> must be set. The documentation says "If not set, RTx::EmailCompletion
> will search for LdapServer parameter...", this isn't the case. If
> EmailCompletionLdapServer isn't set then LDAP isn't used for
> EmailCompletion.
It's a bug, it should work. I will search for this bug.
From jesse at bestpractical.com Tue Nov 13 16:59:14 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Tue Nov 13 16:59:21 2007
Subject: [Rt-devel] constraints.mysql ... in use by anyone?
In-Reply-To: <5D1275B9B7994B5B9A4FE7F55420FF5A@olabinc.com>
References: <5D1275B9B7994B5B9A4FE7F55420FF5A@olabinc.com>
Message-ID: <20071113215914.GM27094@bestpractical.com>
On Mon, Nov 12, 2007 at 02:19:56PM -0800, Otto Hirr wrote:
> Is there a reason that the schema creation scripts
> do not spec constraints?
Mostly because RT does a whole much of app-level referential integrity, both for things that can be represented in SQL and things which can't (There are many items in RT which can refer to any other RT table/record type).
I'd love to get a contribution of updated referential constraints for
any database anyone would care to provide them for.
>
> Most of the db's seem to have the foreign key constraint
> ability, and those that do not, eg sqlite, take it as a no-op,
> at least in my understanding.
>
> Would it not be better to move forward with having constraints
> defined and be able to drop them for those who don't/can't have them?
>
> There is a file, constraints.mysql, however it is not valid,
> ie there are columns that are not defined in all the schemas,
> eg EffectiveTicket is defined in schema.Sybase and is used
> in constraints.mysql, but is not defined in any of the other
> schemas.
Yes. It's out of date.
> If schema.Sybase is old, then it should be dropped or updated.
It certainly should be updated. (The folks who contributed the sybase
port never finished the work).
> Is there someone that can "push a button" and generate a
> new constraints file?
Sadly no, it's a hand-generated affair.
Best,
Jesse
> Regards,
>
> ..Otto
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-devel
>
> SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:
>
> If you sign up for a new RT support contract before December 31, we'll take
> up to 20 percent off the price. This sale won't last long, so get in touch today.
> Email us at sales@bestpractical.com or call us at +1 617 812 0745.
>
--
From vivek at khera.org Tue Nov 13 17:06:47 2007
From: vivek at khera.org (Vivek Khera)
Date: Tue Nov 13 17:13:56 2007
Subject: [Rt-devel] constraints.mysql ... in use by anyone?
In-Reply-To: <20071113215914.GM27094@bestpractical.com>
References: <5D1275B9B7994B5B9A4FE7F55420FF5A@olabinc.com>
<20071113215914.GM27094@bestpractical.com>
Message-ID: <8CE6106A-B50F-4587-BEDB-67480DC6F47D@khera.org>
On Nov 13, 2007, at 4:59 PM, Jesse Vincent wrote:
> I'd love to get a contribution of updated referential constraints for
> any database anyone would care to provide them for.
I started doing them for postgres after I migrated to 3.x from 2.x,
but couldn't do it because of what you wrote in your original message.
From nicolas.chuche at barna.be Wed Nov 14 07:37:55 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Wed Nov 14 07:39:05 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <1194922111.11583.13.camel@dirk.catalyst.net.nz>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
<1194922111.11583.13.camel@dirk.catalyst.net.nz>
Message-ID: <742e684f0711140437j29026a3mb4e55227e16bf2ab@mail.gmail.com>
Hello,
> Just a minor bug fix to the documentation, EmailCompletionLdapServer
> must be set. The documentation says "If not set, RTx::EmailCompletion
> will search for LdapServer parameter...", this isn't the case. If
> EmailCompletionLdapServer isn't set then LDAP isn't used for
> EmailCompletion.
Can't reproduce this bug. With this /opt/rt3/etc/RT_SiteConfig.pm :
Set($LdapServer, "localhost");
Set($LdapBase, "dc=me,dc=org");
It works perfectly here.
Is there someone else with this problem in rc3 (http://gaspard.mine.nu/rtx/) ?
From nicolas.chuche at barna.be Wed Nov 14 07:41:08 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Wed Nov 14 07:41:17 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
Message-ID: <742e684f0711140441u62a0e2b6pe06ecf07a100819a@mail.gmail.com>
Hello,
> I'm also not seeing the grey backgrounds, but haven't had time to look
> at whether that's a problem with the plugin, or somehow something in our
> local setup. I'll hopefully have time to work with it some more
> tomorrow.
Could you try this page : http://gaspard.mine.nu/nc/. Do you see gray
background for the last three lines ? What is your browser ?
From MarkRoedel at letu.edu Wed Nov 14 11:45:49 2007
From: MarkRoedel at letu.edu (Roedel, Mark)
Date: Wed Nov 14 11:46:05 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <742e684f0711140441u62a0e2b6pe06ecf07a100819a@mail.gmail.com>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711140441u62a0e2b6pe06ecf07a100819a@mail.gmail.com>
Message-ID: <0DD6BC9ABC0D2747AF8589AEA9722A400148162E@rachael-reborn.letnet.net>
I found the problem -- there was an old emailcompletion.css from a
previous release in my rt3/local hierarchy that was overriding the new
one.
Thanks for your help...
--
Mark Roedel
Senior Programmer / Analyst
LeTourneau University
-----Original Message-----
From: nchuche@gmail.com [mailto:nchuche@gmail.com] On Behalf Of Nicolas
Chuche
Sent: Wednesday, November 14, 2007 6:41 AM
To: Roedel, Mark
Cc: rt-devel@lists.bestpractical.com
Subject: Re: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
Hello,
> I'm also not seeing the grey backgrounds, but haven't had time to look
> at whether that's a problem with the plugin, or somehow something in
our
> local setup. I'll hopefully have time to work with it some more
> tomorrow.
Could you try this page : http://gaspard.mine.nu/nc/. Do you see gray
background for the last three lines ? What is your browser ?
-----Original Message-----
From: nchuche@gmail.com [mailto:nchuche@gmail.com] On Behalf Of Nicolas
Chuche
Sent: Wednesday, November 14, 2007 6:41 AM
To: Roedel, Mark
Cc: rt-devel@lists.bestpractical.com
Subject: Re: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
Hello,
> I'm also not seeing the grey backgrounds, but haven't had time to look
> at whether that's a problem with the plugin, or somehow something in
our
> local setup. I'll hopefully have time to work with it some more
> tomorrow.
Could you try this page : http://gaspard.mine.nu/nc/. Do you see gray
background for the last three lines ? What is your browser ?
--
BEGIN-ANTISPAM-VOTING-LINKS
------------------------------------------------------
Teach CanIt if this mail (ID 9199988) is spam:
Spam:
http://antispam.letu.edu/canit/b.php?i=9199988&m=69c857f574af&c=s
Not spam:
http://antispam.letu.edu/canit/b.php?i=9199988&m=69c857f574af&c=n
Forget vote:
http://antispam.letu.edu/canit/b.php?i=9199988&m=69c857f574af&c=f
------------------------------------------------------
END-ANTISPAM-VOTING-LINKS
From andrew at etc.gen.nz Wed Nov 14 21:35:47 2007
From: andrew at etc.gen.nz (Andrew Ruthven)
Date: Wed Nov 14 21:36:05 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <742e684f0711140437j29026a3mb4e55227e16bf2ab@mail.gmail.com>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
<1194922111.11583.13.camel@dirk.catalyst.net.nz>
<742e684f0711140437j29026a3mb4e55227e16bf2ab@mail.gmail.com>
Message-ID: <1195094147.3690.5.camel@dirk.catalyst.net.nz>
On Wed, 2007-11-14 at 13:37 +0100, Nicolas Chuche wrote:
> Hello,
>
> > Just a minor bug fix to the documentation, EmailCompletionLdapServer
> > must be set. The documentation says "If not set, RTx::EmailCompletion
> > will search for LdapServer parameter...", this isn't the case. If
> > EmailCompletionLdapServer isn't set then LDAP isn't used for
> > EmailCompletion.
>
> Can't reproduce this bug. With this /opt/rt3/etc/RT_SiteConfig.pm :
>
> Set($LdapServer, "localhost");
> Set($LdapBase, "dc=me,dc=org");
>
> It works perfectly here.
Ah, it does actually work. I think I had another problem (which I've
already fixed). I was looking in lib/RTx/EmailCompletion.pm which made
no mention of LdapServer, I've since found where it uses LdapServer.
Apologies for the false alarm!
--
Andrew Ruthven, Wellington, New Zealand
At home: andrew@etc.gen.nz | This space intentionally
| left blank.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071115/23d748eb/attachment.pgp
From nicolas.chuche at barna.be Thu Nov 15 05:51:47 2007
From: nicolas.chuche at barna.be (Nicolas Chuche)
Date: Thu Nov 15 05:52:30 2007
Subject: [Rt-devel] RTx::EmailCompletion with LDAP : testers needed
In-Reply-To: <1195094147.3690.5.camel@dirk.catalyst.net.nz>
References: <742e684f0711111057u77edcba4oed74db0a4499acef@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A4001431F38@rachael-reborn.letnet.net>
<742e684f0711121339s314a8dc7g7586bbdf23838096@mail.gmail.com>
<0DD6BC9ABC0D2747AF8589AEA9722A400143208C@rachael-reborn.letnet.net>
<742e684f0711121449oecfcc41gb74c2b32ddfb32ed@mail.gmail.com>
<1194922111.11583.13.camel@dirk.catalyst.net.nz>
<742e684f0711140437j29026a3mb4e55227e16bf2ab@mail.gmail.com>
<1195094147.3690.5.camel@dirk.catalyst.net.nz>
Message-ID: <742e684f0711150251j354bc03dy5b0faff0cdcf501@mail.gmail.com>
> Apologies for the false alarm!
No worries and thanks for your help.
From falcone at bestpractical.com Thu Nov 15 12:27:34 2007
From: falcone at bestpractical.com (Kevin Falcone)
Date: Thu Nov 15 12:48:06 2007
Subject: [Rt-devel] Problem with dumpfile-to-rt-3.0
In-Reply-To: <47346F5F.8030306@spring.de>
References: <47346F5F.8030306@spring.de>
Message-ID:
On Nov 9, 2007, at 9:31 AM, Marianne Spiller wrote:
Hi Marianne
> After I unpacked on the new host, I made a 'perl rt-2.0-to-dumpfile
> /opt/RTmigration' whereas my rt2.sql database dump was stored in
> '/opt/RTmigration'. It seemed to work fine; these are the contents of
> this directory:
>
> -rw-r--r-- 1 root root 244497 Nov 9 09:14 metadata
> -rw-r--r-- 1 mspiller users 63809998 Nov 8 12:25 rt2.sql
> drwxr-x--x 2 root root 28672 Nov 9 09:15 tickets-0
> drwxr-x--x 2 root root 28672 Nov 9 09:16 tickets-1000
> drwxr-x--x 2 root root 28672 Nov 9 09:18 tickets-2000
> drwxr-x--x 2 root root 28672 Nov 9 09:19 tickets-3000
> drwxr-x--x 2 root root 24576 Nov 9 09:21 tickets-4000
> drwxr-x--x 2 root root 12288 Nov 9 09:21 tickets-5000
Can you confirm that your metadata file is valid
perl -c metadata should suffice.
Also, just to be clear, you installed a running copy of RT2 on the new
host in order to perform the dump. RT2toRT3 expects to be able
to query a live system.
> Then I installed several versions of RT3 and tried the 'dumpfile'
> command, but without any success; the installation process goes well,
> but importing data fails (see [1]).
Is RT3 running properly before the import?
Have you made any test users/tickets/queues prior to the import?
Also, what version of RT3 are you trying.
I've run this several times against 3.6.3 and 3.6.5 with success.
There is an updated version in subversion, it may have some more
debugging for the user failure. You can find a copy of it at
http://code.bestpractical.com/bps-public/rt2-to-rt3/trunk/
Usually when we see catastrophic failures like this, its because the
metadata
file was corrupt and meant queues and users failed to be created
properly.
-kevin
>
> [1] Executing this results in the following output:
>
> $ perl dumpfile-to-rt-3.0 /opt/RTmigration.WEG/
> "my" variable $msg masks earlier declaration in same scope at
> /opt/rt3/lib/RT/Ticket_Overlay.pm line 3139.
> Name "RT::LogStackTraces" used only once: possible typo at
> dumpfile-to-rt-3.0 line 64.
> Importing scrips
>
> Importing users
> Loading user Unnamed user 1
> Creating new user for Unnamed user 1
> [Fri Nov 9 13:59:11 2007] [error]: Could not create a new user -
> (/opt/rt3/lib/RT/User_Overlay.pm:282)
> Failed to create user for$VAR1 = {
> 'CryptedPassword' => undef,
> 'Name' => 'Unnamed user 1'
> };
>
> [... for about 300 users and so on...]
>
> Importing groups
> g
> Couldn't find user with RT2 userid 10 - not adding to
> g
> Couldn't find user with RT2 userid 7 - not adding to
>
> Couldn't find user with RT2 userid 12 - not adding to
>
> Couldn't find user with RT2 userid 15 - not adding to
>
> Couldn't find user with RT2 userid 276 - not adding to
> g
> Couldn't find user with RT2 userid 9 - not adding to
>
> Couldn't find user with RT2 userid 11 - not adding to
>
> Couldn't find user with RT2 userid 276 - not adding to
> g
> Couldn't find user with RT2 userid 4 - not adding to
>
> Couldn't find user with RT2 userid 8 - not adding to
> g
> Couldn't find user with RT2 userid 4 - not adding to
>
> Couldn't find user with RT2 userid 6 - not adding to
>
> Couldn't find user with RT2 userid 8 - not adding to
>
> Couldn't find user with RT2 userid 11 - not adding to
>
> Couldn't find user with RT2 userid 101 - not adding to
>
> Importing rights
>
> Couldn't load principal to grant them Watch on queue
> $VAR1 = {
> 'PrincipalType' => 'User',
> 'PrincipalId' => '4',
> 'Name' => 'Watch'
> };
>
> Couldn't load principal to grant them WatchAsAdminCc on queue
> $VAR1 = {
> 'PrincipalType' => 'User',
> 'PrincipalId' => '4',
> 'Name' => 'WatchAsAdminCc'
> };
>
> Couldn't create TICKET: Could not create ticket. Queue not set$VAR1
> = {
> 'Requestor' => [
> 'daniel.haha@blah.de'
> ],
> 'Owner' => undef,
> 'LastUpdatedBy' => undef,
> '_RecordTransaction' => '0',
> 'id' => '14981',
> 'Queue' => 'IVWBOX',
> 'Creator' => undef
> };
> [Fri Nov 9 14:00:33 2007] [crit]: Died at dumpfile-to-rt-3.0 line
> 697.
> (/opt/rt3/lib/RT.pm:254)
>
>
> _______________________________________________
> List info: http://lists.bestpractical.com/cgi-bin/mailman/listinfo/
> rt-devel
>
> SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:
>
> If you sign up for a new RT support contract before December 31,
> we'll take
> up to 20 percent off the price. This sale won't last long, so get
> in touch today. Email us at sales@bestpractical.com or call us
> at +1 617 812 0745.
>
From sturner at MIT.EDU Thu Nov 15 16:39:08 2007
From: sturner at MIT.EDU (Stephen Turner)
Date: Thu Nov 15 16:39:29 2007
Subject: [Rt-devel] RT 3.6.5 Bug - Queue-list caching
Message-ID: <6.2.3.4.2.20071115161326.04563598@po14.mit.edu>
Hello,
In RT 3.6.5, /Admin/Queues/Modify.html, there are still references to
the session cache of queues with key "create_in_queues". This is a
legacy of the older (i.e. 3.4) , simpler, single queue cache from
/Elements/SelectNewTicketQueue. This cache scheme has been expanded
in 3.6 to cover multiple queue caches with cache keys that look like:
"SelectQueue---" . $session{'CurrentUser'}->Id .
"---$CheckQueueRight---$ShowAllQueues"
Anyway, the code in Admin/Queues/Modify.html is still trying to
delete the older-style cache and has no effect. This can be verified
by creating a new queue and checking the "create" queue drop-downs on
the home page - the new queue does not appear.
I haven't supplied a patch because I wanted to suggest alternative
solutions. When a queue is created or modified you could
a) Look through all session keys and delete the ones that match the
new queue-cache format ( starts with "SelectQueue---" ). The problem
with this is a performance hit while the cache is then reconstructed
- a lot of queues means a delay. The deletion & recreation of the
caches seems to be a brute-force way of getting the queue listing
right. Another way (we use this at MIT) is:
b) Instead of deleting the queue caches & forcing the slow reload,
just modify them:
- New queue: add the new queue to the appropriate caches
- Disabled queue: remove the queue from the caches
- Enabled queue: add the new queue to the appropriate caches
If someone lets me know a preferred way, I'll be happy to send a patch.
Thanks,
Steve
From pgsql at j-davis.com Sun Nov 18 17:10:09 2007
From: pgsql at j-davis.com (Jeff Davis)
Date: Sun Nov 18 17:32:29 2007
Subject: [Rt-devel] Huge number of database queries
Message-ID: <1195423809.24251.234.camel@jdavis>
RT seems to be generating a huge amount of database traffic. After
investigating, we found that most of the queries were because the static
content (e.g. .css files, image files, .js files) was also generating
database traffic, about 19 queries each. The total number was about 700
queries for one page load!
So, of course, we reconfigured so that static content is not handled by
RT. However, a single dynamic page (e.g. ticket view) still generates
119 queries, which still seems like a huge number to me.
Here is the breakdown for one dynamic page load (displaying a ticket):
119 queries total
067 statements
049 select
005 ping test
044 other
016 deallocate
002 set
052 parse/bind/execute
Is this expected? Is there a way to bring this to a more reasonable
number? It's hard for me to imagine that we need 44 normal SELECT
statements to view a ticket. The 16 additional prepared statements,
which are then immediately DEALLOCATEd are probably excessive as well.
Keep in mind this is a page _reload_ so the page has already been loaded
once.
Regards,
Jeff Davis
From jesse at bestpractical.com Sun Nov 18 17:58:15 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Sun Nov 18 17:58:22 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195423809.24251.234.camel@jdavis>
References: <1195423809.24251.234.camel@jdavis>
Message-ID: <20071118225815.GA25399@bestpractical.com>
On Sun, Nov 18, 2007 at 02:10:09PM -0800, Jeff Davis wrote:
> RT seems to be generating a huge amount of database traffic. After
> investigating, we found that most of the queries were because the static
> content (e.g. .css files, image files, .js files) was also generating
> database traffic, about 19 queries each.
So, RT 3.6.6 should significantly improve the state of the static
content. We've finally deployed CSS::Squish to force only a single CSS
file for your RT instance and we've improved the browser hinting that
should force caching so you only load it once unless you're forcing the
browser's cache to wipe itself on reload. But do note that even 3.6.5
should be caching those files.
> RT. However, a single dynamic page (e.g. ticket view) still generates
> 119 queries, which still seems like a huge number to me.
When I saw your blog post about this, I thought you were talking about
_SQL_ queries.
> Here is the breakdown for one dynamic page load (displaying a ticket):
How many custom fields and transactions on the ticket?
>
> 119 queries total
> 067 statements
> 049 select
> 005 ping test
> 044 other
> 016 deallocate
> 002 set
> 052 parse/bind/execute
>
> Is this expected? Is there a way to bring this to a more reasonable
> number?
Do you just find it "not-right" or are you seeing poor performance?
Either way, I'd like to see what the queries are, so as to be able to
give you a decent answer there.
> It's hard for me to imagine that we need 44 normal SELECT
> statements to view a ticket.
Can you pull out what they are?
The 16 additional prepared statements,
> which are then immediately DEALLOCATEd are probably excessive as well.
I suspect that the issue here stems from the fact that until _very_
recently, DBD::mysql and DBD::Pg didn't support server side prepared
statements - everything was emulated on the client. Putting together a
cache at the ORM layer should be doable. But I've never seen those sorts
of queries actually generate measurable overhead.
Best,
Jesse
From ruz at bestpractical.com Sun Nov 18 18:24:00 2007
From: ruz at bestpractical.com (Ruslan Zakirov)
Date: Sun Nov 18 18:24:05 2007
Subject: [Rt-devel] Description of the latest improvement in ACL checks
Message-ID: <589c94400711181524l3fec2586ia9a200110a710260@mail.gmail.com>
Hey, guys.
I've committed quite major change to ACL check Principals.pm that:
1) split queries, but makes them more effective
2) adds additional caching of positive results
3) get rid of one join in a query
1. about splitting queries
If you'll compare explains of the two queries below:
SELECT id FROM grp WHERE
type = 'Owner'
AND (
( domain = 'RT::Ticket-Role' AND instance = 10 )
OR ( domain = 'RT::Queue-Role' AND instance = 3 )
OR ( domain = 'RT::System-Role' AND instance = 1 )
);
SELECT STRAIGHT_JOIN g.id FROM acl a, grp g WHERE
g.type = a.type
AND (
( g.domain = 'RT::Ticket-Role' AND g.instance = 10 )
OR ( g.domain = 'RT::Queue-Role' AND g.instance = 3 )
OR ( g.domain = 'RT::System-Role' AND g.instance = 1 )
);
then you'll see that in the first mysql uses "range" scan and in the
second one "ref", usually ref is better than range, but if you'll look
closer then you'll see that ref scan uses only first part
('group.type') part of an index. That's bug of mysql and it's been
fixed in 5.0.45 only. Why it's so bad for RT? That's bad as we have
one group per ticket and type combination, so if you have 10k tickets
then you have 10k groups of 'Owner' type. So for each record in ACL
table mysql finds 10k groups using index, for each group it fetches
other fields from the table data and applies other conditions. In
mysql 5.0.45 this bug has been fixed and mysql uses range scan and
gets much less records and filter them using an index only.
Now we split this query into group:
SELECT STRAIGHT_JOIN g.id FROM acl a, grp g WHERE
g.type = a.type
AND ( g.domain = 'RT::Ticket-Role' AND g.instance = 10 )
it's even more effective as mysql and other DBs too can use ref access
type and get set of records using an index only even not applying
where conditions, however we have to run more queries in the worst
case so it should be quite balanced.
This check we do quite often and people should see improvement when
they grant many rights via roles.
Anyway there is less used query that get all users who has right two
own tickets in all queues, we run it when we build the query builder
and it's not affected by this improvement. I think we should change
this as well, but I have no patch.
2. about additional caching of positive results
I've added new type of caching of positive results what can reduce
number of request really dramatically, especially for privileged users
who really have rights. Before we very caching positive and negative
answers using uid, right and whole list of acl equiv objects. However
if user is member of a group that has the right on one of those
objects directly (not via roles) then really user has that right one
any object which list the same acl equiv object. For example if we
check a right on a ticket X then we list System, Queue-Y and Ticket-X.
Imagine that user has that right via group or personal on Queue-Y what
means that he really has the right to all tickets in Y. In this case
we also cache result with key: uid, right, object. Then we'll fetch
results from cache for any request where queue Y is listed as ACL
equiv object. We don't cache negative answers so the old cache is
still effective and left in its place.
--
Best regards, Ruslan.
From pgsql at j-davis.com Sun Nov 18 19:34:36 2007
From: pgsql at j-davis.com (Jeff Davis)
Date: Sun Nov 18 19:30:09 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <20071118225815.GA25399@bestpractical.com>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
Message-ID: <1195432476.24251.292.camel@jdavis>
On Sun, 2007-11-18 at 17:58 -0500, Jesse Vincent wrote:
> So, RT 3.6.6 should significantly improve the state of the static
> content. We've finally deployed CSS::Squish to force only a single CSS
> file for your RT instance and we've improved the browser hinting that
> should force caching so you only load it once unless you're forcing the
> browser's cache to wipe itself on reload. But do note that even 3.6.5
> should be caching those files.
>
Hmm... perhaps Ctrl-R forced it to re-request everything. I'm not quite
as concerned about this, because there's a workaround, but I'm glad to
hear it's being improved.
>
> > RT. However, a single dynamic page (e.g. ticket view) still generates
> > 119 queries, which still seems like a huge number to me.
>
> When I saw your blog post about this, I thought you were talking about
> _SQL_ queries.
What I mean, very specifically, is that when I turn
"log_min_duration_statement = 0" in postgresql.conf, it generates 119
lines that contain "duration:" in the log. These are primarily SQL
queries, although I suppose that depends on your definition.
> > Here is the breakdown for one dynamic page load (displaying a ticket):
>
> How many custom fields and transactions on the ticket?
No custom fields and 3 transactions, as far as I can tell. We have an
almost empty database.
> >
> > 119 queries total
> > 067 statements
> > 049 select
> > 005 ping test
> > 044 other
> > 016 deallocate
> > 002 set
> > 052 parse/bind/execute
> >
> > Is this expected? Is there a way to bring this to a more reasonable
> > number?
>
> Do you just find it "not-right" or are you seeing poor performance?
> Either way, I'd like to see what the queries are, so as to be able to
> give you a decent answer there.
>
I would say that the problem is more that it's "not right" than any
serious performance problem now.
We haven't deployed it to production yet, but we will have a substantial
number of users though, and most of those users will be using RT
constantly through the workday. People will avoid using the ticketing
system if every minor update or action interrupts them with a delay
(even a small delay), so I'd like the system to be essentially
instantaneous.
I'm more worried about the stability of performance than anything else.
With so many network round-trips to the database, I'm worried that any
minor slowdown would make RT unavailable. We don't notice any serious
problem right now, because our network ping time is ~0.2 ms, and our
processor usage on the database server is also low. Also, we have
essentially an empty database now, so what will the performance be like
as it grows? Will the number of queries per page load change as the
dataset changes?
It also just makes it harder to administer. If I turn on query logging
to diagnose a problem, the signal to noise ratio makes it almost
useless.
>
> > It's hard for me to imagine that we need 44 normal SELECT
> > statements to view a ticket.
>
> Can you pull out what they are?
I have to scrub the data before putting it in a public place (yeah, I
know there's nothing sensitive, but I still have to look to be sure), so
this is just a little more detail to the summary:
016 parse
001 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
002 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
001 "SELECT * FROM Tickets WHERE id = $1"
002 "SELECT * FROM Queues WHERE id = $1"
001 "SELECT * FROM Transactions WHERE id = $1"
009 other
018 bind
018 execute
016 deallocate
049 select
005 ping test
011 "SELECT ... from ACL, Groups, Principals, CachedGroupMembers ..."
004 "SELECT main.* FROM ( SELECT main.id FROM CustomFields ..."
004 "SELECT DISTINCT main.* FROM Users main JOIN Principals ..."
006 "SELECT main.* FROM Tickets main WHERE ..."
006 "SELECT main.* FROM Links main WHERE ..."
002 "...Users main CROSS JOIN ACL..."
011 other
>
> The 16 additional prepared statements,
> > which are then immediately DEALLOCATEd are probably excessive as well.
>
> I suspect that the issue here stems from the fact that until _very_
> recently, DBD::mysql and DBD::Pg didn't support server side prepared
> statements - everything was emulated on the client. Putting together a
> cache at the ORM layer should be doable. But I've never seen those sorts
> of queries actually generate measurable overhead.
I'm not worried about optimizing away a few protocol round-trips that
are only generated by DBD::Pg. I'm worried about the big numbers. If you
count just the executes, and ignore parse/bind/deallocate (which is fine
by me), that's still 18+49 = 67 queries.
So I guess the numbers are: 119 round trips to the DB server, and 67
queries. It would be nice if we could get both of those numbers down to
something reasonable.
Regards,
Jeff Davis
From jesse at bestpractical.com Mon Nov 19 09:22:02 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Mon Nov 19 09:22:06 2007
Subject: [Rt-devel] RT 3.6.5 Bug - Queue-list caching
In-Reply-To: <6.2.3.4.2.20071115161326.04563598@po14.mit.edu>
References: <6.2.3.4.2.20071115161326.04563598@po14.mit.edu>
Message-ID: <20071119142202.GE25399@bestpractical.com>
On Thu, Nov 15, 2007 at 04:39:08PM -0500, Stephen Turner wrote:
> Hello,
>
> In RT 3.6.5, /Admin/Queues/Modify.html, there are still references to
> the session cache of queues with key "create_in_queues". This is a
> legacy of the older (i.e. 3.4) , simpler, single queue cache from
> /Elements/SelectNewTicketQueue. This cache scheme has been expanded
> in 3.6 to cover multiple queue caches with cache keys that look like:
>
> "SelectQueue---" . $session{'CurrentUser'}->Id .
> "---$CheckQueueRight---$ShowAllQueues"
>
> Anyway, the code in Admin/Queues/Modify.html is still trying to
> delete the older-style cache and has no effect. This can be verified
> by creating a new queue and checking the "create" queue drop-downs on
> the home page - the new queue does not appear.
>
> I haven't supplied a patch because I wanted to suggest alternative
> solutions. When a queue is created or modified you could
>
> a) Look through all session keys and delete the ones that match the
> new queue-cache format ( starts with "SelectQueue---" ). The problem
> with this is a performance hit while the cache is then reconstructed
> - a lot of queues means a delay. The deletion & recreation of the
> caches seems to be a brute-force way of getting the queue listing
> right. Another way (we use this at MIT) is:
>
> b) Instead of deleting the queue caches & forcing the slow reload,
> just modify them:
> - New queue: add the new queue to the appropriate caches
> - Disabled queue: remove the queue from the caches
> - Enabled queue: add the new queue to the appropriate caches
>
> If someone lets me know a preferred way, I'll be happy to send a patch.
b) sounds the most sane. I'd lvoe a patch.
From jesse at bestpractical.com Mon Nov 19 09:23:46 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Mon Nov 19 09:23:51 2007
Subject: [Rt-devel] RT 3.6.5 Bug - Queue-list caching
In-Reply-To: <20071119142202.GE25399@bestpractical.com>
References: <6.2.3.4.2.20071115161326.04563598@po14.mit.edu>
<20071119142202.GE25399@bestpractical.com>
Message-ID: <20071119142346.GF25399@bestpractical.com>
> > a) Look through all session keys and delete the ones that match the
> > new queue-cache format ( starts with "SelectQueue---" ). The problem
> > with this is a performance hit while the cache is then reconstructed
> > - a lot of queues means a delay. The deletion & recreation of the
> > caches seems to be a brute-force way of getting the queue listing
> > right. Another way (we use this at MIT) is:
> >
> > b) Instead of deleting the queue caches & forcing the slow reload,
> > just modify them:
> > - New queue: add the new queue to the appropriate caches
> > - Disabled queue: remove the queue from the caches
> > - Enabled queue: add the new queue to the appropriate caches
> >
> > If someone lets me know a preferred way, I'll be happy to send a patch.
>
>
> b) sounds the most sane. I'd lvoe a patch.
Though hm. This will only fix it for the current session. I wonder if we
could just be clever and do a join on the ACL table these days. I think
we have a lot more of the infrastructure we'd need to do a sane query
than we sued to.
From G.E.Fowler at lboro.ac.uk Mon Nov 19 11:44:20 2007
From: G.E.Fowler at lboro.ac.uk (Graeme Fowler)
Date: Mon Nov 19 12:00:07 2007
Subject: [Rt-devel] PATCHES: Allow RT to authenticate against PAM
Message-ID: <1195490660.2700.92.camel@squonk.lboro.ac.uk>
Hi all
I've just coded up an extension to RT to allow it to authenticate using
PAM (on a Linux system or other system where PAM is available, at
least). For those whom the term PAM is meaningless:
http://www.kernel.org/pub/linux/libs/pam/
PAM can authenticate against a multitude of sources (backends) - an LDAP
system, Active Directory, Kerberos, local passwd/shadow files, a
separate MySQL instance... you name it, it's probably got a module for
it somewhere.
This will allow RT admins to use a flexible authentication source
*without* having to modify RT beyond a couple of configuration options.
The changes made are principally to the "IsPassword" function in
lib/RT/CurrentUser.pm such that it can use PAM to check a password, can
use the RT database to check a password, or fall through from PAM to
database if PAM fails.
Additionally, if PAM succeeds and the RT database differs from that
password, there's the option to set the RT password (thus keeping them
in sync).
In combination, if the PAM source goes away, the system can still
authenticate against the RT database using the most recently successful
PAM password. It's possible to turn off the password change and fallback
options, too.
There are both changes to CurrentUser.pm and some additional sections to
RT_SiteConfig.pm (see patches inline). Some of the logging statements
could be changed (it's a bit verbose at the moment) but otherwise it's
all as functional as it can be.
The only requirement is that Perl Authen::PAM module is installed on the
system. It's only loaded as and when required. The array defining
modules (see below) currently contains two module names - rt-krb5-auth
and rt-dummy-module. Any module name passed in this way containing the
word "dummy" is skipped - it's there purely for reference, to see that
the code looped as I wanted it to.
I've been testing it against our local AD (using Kerberos underneath
PAM) and it works as expected; I haven't found any boogs yet, but that
doesn't mean there aren't any :)
I've tested the fallback code by briefly firewalling out our domain
controllers; after a brief pause the code logged the appropriate message
("no Kerberos KDC is reachable", or similar) and fell through to the
database authentication. That then let me in.
Please give it a try (if it's relevant) and see what you think.
Graeme
--- etc/RT_SiteConfig.pm.pre-PAM 2007-11-19 10:41:00.000000000
+0000
+++ etc/RT_SiteConfig.pm 2007-11-19 14:55:30.000000000 +0000
@@ -316,6 +316,30 @@
Set($WebExternalAuto , undef);
+# --- BEGIN MODIFICATION ---
+# If $PamAuth is defined, RT will attempt to use a PAM module on this
+# server to authenticate against, for example, Active Directory or
+# an LDAP server. The config of the PAM module is outside the scope
+# of this documentation. The name(s) of the PAM modules are listed
+# in the array $PamAuthModules
+
+Set($PamAuth , 1 );
+Set($PamAuthModules, [ 'rt-krb5-auth', 'rt-dummy-module' ]);
+
+# If $PamFallbackToInternal is defined, the password will be checked
+# against the RT internal database upon PAM auth failure. This permits
+# the use of RT when the external authentication source has "gone
away".
+
+Set($PamFallbackToInternal , 1 );
+
+# If $PamUpdateRTPassword is defined, successful authentication against
+# the configured PAM module(s) will update the local RT user's password
+# in the database. This permits the use of RT with the same passwords
+# when the external authentication source has "gone away".
+
+Set($PamUpdateRTPassword, 1 );
+# --- END MODIFICATION ---
+
# $WebSessionClass is the class you wish to use for managing Sessions.
# It defaults to use your SQL database, but if you are using MySQL 3.x
and
# plans to use non-ascii Queue names, uncomment and add this line to
--- lib/RT/CurrentUser.pm 2007-04-24 18:21:42.000000000 +0100
+++ local/lib/RT/CurrentUser.pm 2007-11-19 14:57:13.000000000 +0000
@@ -294,9 +294,13 @@
=head2 IsPassword
-Takes a password as a string. Passes it off to IsPassword in this
-user's UserObj. If it is the user's password and the user isn't
-disabled, returns 1.
+Takes a password as a string. Either:
+ 1. Checks it against PAM if $RT::PamAuth is set.
+ - if $RT::PamFallbackToInternal is set and PAM fails, falls to step
2
+ - sets RT password if $RT::PamUpdateRTPassword is set
+ 2. Passes it off to IsPassword in this user's UserObj.
+
+If the password is validated and the user isn't disabled, returns 1.
Otherwise, returns undef.
@@ -306,6 +310,47 @@
my $self = shift;
my $value = shift;
+ if ($RT::PamAuth) {
+ my $name = $self->UserObj->Name;
+
+ # First check that the user isn't disabled.
+ # There is no point authenticating if they are!
+
+ if ($self->PrincipalObj->Disabled) {
+ $RT::Logger->warning("User $name attempted PAM login but is
disabled in RT database.");
+ }
+
+ use Authen::PAM;
+ foreach my $pam_module (@$RT::PamAuthModules) {
+ $RT::Logger->debug("MODULE: " . $pam_module);
+ next if ($pam_module =~ /dummy/);
+ my $pam_conv_func = sub { my @res = (PAM_SUCCESS); unshift @res,
$value and unshift @res, 0 while( shift @_ && shift @_ ); @res };
+ my( $res, $pamh, );
+ if( ( $res = pam_start( $pam_module, $name, $pam_conv_func,
$pamh ) ) == PAM_SUCCESS
+ && ( $res = pam_authenticate( $pamh, 0 ) )
== PAM_SUCCESS
+ && ( $res = pam_end( $pamh, 0 ) )
== PAM_SUCCESS
+ ) {
+ # If $PamUpdateRTPassword is defined we need to set the RT
password
+ # but only if the old RT password differs
+ if ($RT::PamUpdateRTPassword && (!
$self->UserObj->IsPassword($value))) {
+ $RT::Logger->warning("Attempting to change RT password for
$name");
+ my ( $val, $msg ) = $self->UserObj->SetPassword($value);
+ if ($val) {
+ $RT::Logger->warning("User $name RT password changed to PAM
password on successful PAM login");
+ }
+ }
+ return ( 1 );
+ } else {
+ $RT::Logger->warning("User $name failed PAM authentication");
+ unless ($RT::PamFallbackToInternal) {
+ return ( 0 );
+ } else {
+ $RT::Logger->warning("User $name falling back to RT database
authentication");
+ }
+ }
+ }
+ }
+
return ($self->UserObj->IsPassword($value));
}
-----------------------
Our /etc/pam.d/rt-krb5-auth file contains:
#%PAM-1.0
auth required pam_krb5.so debug no_user_check
...and that's it.
From pgsql at j-davis.com Tue Nov 20 15:13:45 2007
From: pgsql at j-davis.com (Jeff Davis)
Date: Tue Nov 20 15:14:07 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195432476.24251.292.camel@jdavis>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
Message-ID: <1195589625.22428.268.camel@dogma.ljc.laika.com>
On Sun, 2007-11-18 at 16:34 -0800, Jeff Davis wrote:
> So I guess the numbers are: 119 round trips to the DB server, and 67
> queries. It would be nice if we could get both of those numbers down to
> something reasonable.
Is this considered a bug, or expected behavior?
Regards,
Jeff Davis
From jesse at bestpractical.com Tue Nov 20 15:16:40 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Tue Nov 20 15:16:55 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195589625.22428.268.camel@dogma.ljc.laika.com>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
<1195589625.22428.268.camel@dogma.ljc.laika.com>
Message-ID: <20071120201640.GX1301@bestpractical.com>
On Tue, Nov 20, 2007 at 12:13:45PM -0800, Jeff Davis wrote:
> On Sun, 2007-11-18 at 16:34 -0800, Jeff Davis wrote:
> > So I guess the numbers are: 119 round trips to the DB server, and 67
> > queries. It would be nice if we could get both of those numbers down to
> > something reasonable.
>
> Is this considered a bug, or expected behavior?
You got my message from yesterday, where I asked for some more detail
about the exact queries, right? I can't _tell_ exactly what's biting you
until I see what's going on.
>
> Regards,
> Jeff Davis
>
--
From jesse at bestpractical.com Mon Nov 19 08:18:31 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Tue Nov 20 15:57:47 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195432476.24251.292.camel@jdavis>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
Message-ID: <47418D27.3060900@bestpractical.com>
Jeff Davis wrote:
> Hmm... perhaps Ctrl-R forced it to re-request everything. I'm not quite
> as concerned about this, because there's a workaround, but I'm glad to
> hear it's being improved.
Yeah, it generally does.
>
>>> RT. However, a single dynamic page (e.g. ticket view) still generates
>>> 119 queries, which still seems like a huge number to me.
>> When I saw your blog post about this, I thought you were talking about
>> _SQL_ queries.
>
> What I mean, very specifically, is that when I turn
> "log_min_duration_statement = 0" in postgresql.conf, it generates 119
> lines that contain "duration:" in the log. These are primarily SQL
> queries, although I suppose that depends on your definition.
*nod* FWIW, enabling "log_statement" on 8.1 didn't do that for me.
>
>>> Here is the breakdown for one dynamic page load (displaying a ticket):
>> How many custom fields and transactions on the ticket?
>
> No custom fields and 3 transactions, as far as I can tell. We have an
> almost empty database.
> We haven't deployed it to production yet, but we will have a substantial
> number of users though, and most of those users will be using RT
> constantly through the workday. People will avoid using the ticketing
> system if every minor update or action interrupts them with a delay
> (even a small delay), so I'd like the system to be essentially
> instantaneous.
As would we.
>
> I'm more worried about the stability of performance than anything else.
> With so many network round-trips to the database, I'm worried that any
> minor slowdown would make RT unavailable. We don't notice any serious
> problem right now, because our network ping time is ~0.2 ms, and our
> processor usage on the database server is also low. Also, we have
> essentially an empty database now, so what will the performance be like
> as it grows? Will the number of queries per page load change as the
> dataset changes?
It shouldn't change significantly, but if you want to stress-test, the
results can help us to improve RT.
> It also just makes it harder to administer. If I turn on query logging
> to diagnose a problem, the signal to noise ratio makes it almost
> useless.
I think that a little experience about what to grep out will go a long
way toward that. We've had a fair amount of experience and luck tuning
RT on Postgres with those logs.
>>> It's hard for me to imagine that we need 44 normal SELECT
>>> statements to view a ticket.
>> Can you pull out what they are?
>
> I have to scrub the data before putting it in a public place (yeah, I
> know there's nothing sensitive, but I still have to look to be sure), so
> this is just a little more detail to the summary:
>
> 016 parse
> 001 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
> 002 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
LOWER(Gecos)? Really? Can you pull out the values there? I wouldn't
expect to see that in normal operation unless you're playing external
authenntication tricks.
> 001 "SELECT * FROM Tickets WHERE id = $1"
> 002 "SELECT * FROM Queues WHERE id = $1"
All thse seem pretty
> 001 "SELECT * FROM Transactions WHERE id = $1"
This seems a little surprsiing. That should get pulled in from a bigger
prefetch of all txns related to that ticket
> 009 other
> 018 bind
> 018 execute
> 016 deallocate
I talked about those in my last mail. I'd love to see a patch to
SearchBuilder to reuse prepared statements, but I'm not sure it's a
high-priority target.
> 049 select
> 005 ping test
> 011 "SELECT ... from ACL, Groups, Principals, CachedGroupMembers ..."
11 ACL checks seems slightly high but not insanely high, since RT is
checking queue, ticket and custom field ACLs. Though you should pay
attention to Ruslan's recent mail about refactoring the ACL code to be a
bit smarter and cut down on the number of queries.
> 004 "SELECT main.* FROM ( SELECT main.id FROM CustomFields ..."
Your custom fields.
> 004 "SELECT DISTINCT main.* FROM Users main JOIN Principals ..."
Requestors, Ccs, AdminCcs and Owners.
> 006 "SELECT main.* FROM Tickets main WHERE ..."
Unsure
> 006 "SELECT main.* FROM Links main WHERE ..."
Six different kinds of links on tickets
> 002 "...Users main CROSS JOIN ACL..."
> 011 other
What falls into "other"?
> So I guess the numbers are: 119 round trips to the DB server, and 67
> queries. It would be nice if we could get both of those numbers down to
> something reasonable.
*nod* Improving query count has generally taken a back-seat to improving
overall query _performance_, since quite often there are one or two
queries in that set that dwarf the run time of all the rest. That said,
I'd love to make fewer queries and get the same results ;) Would you
maybe be up for helping to improve things? Setting up a global
prepared-query cache seems like it might be a good, well-contained
project that would have a pretty direct impact on what's made your
spidey-sense tingle.
There's also interesting work to be done with caching and invalidating
caches of collection search results, possibly using a tool like MemCache
that I'd be happy to ramble about if somebody is interested in hacking
on it.
Best,
Jesse
>
> Regards,
> Jeff Davis
>
>
From pgsql at j-davis.com Tue Nov 20 19:39:07 2007
From: pgsql at j-davis.com (Jeff Davis)
Date: Tue Nov 20 19:39:19 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <47418D27.3060900@bestpractical.com>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
<47418D27.3060900@bestpractical.com>
Message-ID: <1195605547.22428.297.camel@dogma.ljc.laika.com>
On Mon, 2007-11-19 at 07:18 -0600, Jesse Vincent wrote:
> *nod* FWIW, enabling "log_statement" on 8.1 didn't do that for me.
>
PostgreSQL logging has many options, and many of them are
interdependent. I mentioned log_min_duration_statement=0 because that's
an easy way to make sure that the statements are logged.
> >
> > I'm more worried about the stability of performance than anything else.
> > With so many network round-trips to the database, I'm worried that any
> > minor slowdown would make RT unavailable. We don't notice any serious
> > problem right now, because our network ping time is ~0.2 ms, and our
> > processor usage on the database server is also low. Also, we have
> > essentially an empty database now, so what will the performance be like
> > as it grows? Will the number of queries per page load change as the
> > dataset changes?
>
>
> It shouldn't change significantly, but if you want to stress-test, the
> results can help us to improve RT.
We'll certainly provide our results.
> > It also just makes it harder to administer. If I turn on query logging
> > to diagnose a problem, the signal to noise ratio makes it almost
> > useless.
>
> I think that a little experience about what to grep out will go a long
> way toward that. We've had a fair amount of experience and luck tuning
> RT on Postgres with those logs.
>
Ok.
> >>> It's hard for me to imagine that we need 44 normal SELECT
> >>> statements to view a ticket.
> >> Can you pull out what they are?
> >
> > I have to scrub the data before putting it in a public place (yeah, I
> > know there's nothing sensitive, but I still have to look to be sure), so
> > this is just a little more detail to the summary:
> >
> > 016 parse
> > 001 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
> > 002 "SELECT * FROM Users WHERE LOWER(Gecos) = LOWER($1)"
>
> LOWER(Gecos)? Really? Can you pull out the values there? I wouldn't
> expect to see that in normal operation unless you're playing external
> authenntication tricks.
We set WebExternalAuth and we unset WebFallbackToInternalAuth. I don't
think we do any tricks aside from that. We use a .htaccess file to
authenticate through LDAP.
> > 001 "SELECT * FROM Transactions WHERE id = $1"
>
> This seems a little surprsiing. That should get pulled in from a bigger
> prefetch of all txns related to that ticket
>
> > 009 other
> > 018 bind
> > 018 execute
> > 016 deallocate
>
> I talked about those in my last mail. I'd love to see a patch to
> SearchBuilder to reuse prepared statements, but I'm not sure it's a
> high-priority target.
>
I'll have to look into it to see how many statements can be effectively
reused.
> > 049 select
> > 005 ping test
> > 011 "SELECT ... from ACL, Groups, Principals, CachedGroupMembers ..."
>
> 11 ACL checks seems slightly high but not insanely high, since RT is
> checking queue, ticket and custom field ACLs. Though you should pay
> attention to Ruslan's recent mail about refactoring the ACL code to be a
> bit smarter and cut down on the number of queries.
Ok.
> > 004 "SELECT main.* FROM ( SELECT main.id FROM CustomFields ..."
>
> Your custom fields.
>
"customfield" and "customfieldvalues" are both empty tables
> > 004 "SELECT DISTINCT main.* FROM Users main JOIN Principals ..."
>
> Requestors, Ccs, AdminCcs and Owners.
>
> > 006 "SELECT main.* FROM Tickets main WHERE ..."
>
> Unsure
>
> > 006 "SELECT main.* FROM Links main WHERE ..."
>
> Six different kinds of links on tickets
>
> > 002 "...Users main CROSS JOIN ACL..."
> > 011 other
>
> What falls into "other"?
>
I'll send the logs to you off-list.
> *nod* Improving query count has generally taken a back-seat to improving
> overall query _performance_, since quite often there are one or two
> queries in that set that dwarf the run time of all the rest. That said,
Good performance and also stable/scalable performance that doesn't
drastically change when the conditions change are the main goals for me.
I'm not implying that RT does that currently, but that's where I'm
coming from.
> I'd love to make fewer queries and get the same results ;) Would you
> maybe be up for helping to improve things? Setting up a global
> prepared-query cache seems like it might be a good, well-contained
> project that would have a pretty direct impact on what's made your
> spidey-sense tingle.
I've been looking into it as I learn RT. The deadline for deploying RT
is approaching, but afterward when RT is in maintenance I may be able to
contribute (employer permitting).
> There's also interesting work to be done with caching and invalidating
> caches of collection search results, possibly using a tool like MemCache
> that I'd be happy to ramble about if somebody is interested in hacking
> on it.
That's always an interesting subject, but I'm cautious about introducing
more caching because it's a challenge to get it right. I'd hate to give
up real stability for hypothetical performance ;)
Regards,
Jeff Davis
From jesse at bestpractical.com Fri Nov 23 12:58:58 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Fri Nov 23 12:59:12 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195605547.22428.297.camel@dogma.ljc.laika.com>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
<47418D27.3060900@bestpractical.com>
<1195605547.22428.297.camel@dogma.ljc.laika.com>
Message-ID: <20071123175858.GI12099@bestpractical.com>
>
> > *nod* Improving query count has generally taken a back-seat to improving
> > overall query _performance_, since quite often there are one or two
> > queries in that set that dwarf the run time of all the rest. That said,
>
> Good performance and also stable/scalable performance that doesn't
> drastically change when the conditions change are the main goals for me.
> I'm not implying that RT does that currently, but that's where I'm
> coming from.
Understood. Seeing a significant rise in query count per page load as the system scales...shouldn't happen. And postgres with a bit of tuning is very good at keeping the performance scale nicely. But there's no substitute for testing to make me eat my words ;)
>
> That's always an interesting subject, but I'm cautious about introducing
> more caching because it's a challenge to get it right. I'd hate to give
> up real stability for hypothetical performance ;)
:)
From jesse at bestpractical.com Fri Nov 23 13:20:19 2007
From: jesse at bestpractical.com (Jesse Vincent)
Date: Fri Nov 23 13:20:23 2007
Subject: [Rt-devel] Huge number of database queries
In-Reply-To: <1195700118.24251.331.camel@jdavis>
References: <1195423809.24251.234.camel@jdavis>
<20071118225815.GA25399@bestpractical.com>
<1195432476.24251.292.camel@jdavis>
<47418D27.3060900@bestpractical.com>
<1195700118.24251.331.camel@jdavis>
Message-ID: <20071123182019.GK12099@bestpractical.com>
On Wed, Nov 21, 2007 at 06:55:18PM -0800, Jeff Davis wrote:
>
> Here is the log of the SQL queries. I meant to send this earlier today.
>
> Note that I am sending this off-list. I have redacted some of the
> information, even though it may seem innocuous, but I don't think it
> will get in your way. I didn't actually delete any of the entries
> though.
[re-cced to rt-devel now that I've removed the SQL log]
There is one thing I notice in your log that's "not right" and is likely
RT's fault. of the 51 queries to build a ticket history page, a large
number of them are for "Transaction custom fields" Those could all be
done with a single query and pre-loaded. That may be worth looking at if
you see a performance issue there. But it's well-indexed and shouldn't
be _much_ of a performance it. (Famous last words, I know).
>
> Thanks for your help so far.
>
> Regards,
> Jeff Davis
From G.E.Fowler at lboro.ac.uk Thu Nov 29 12:47:41 2007
From: G.E.Fowler at lboro.ac.uk (Graeme Fowler)
Date: Thu Nov 29 12:48:02 2007
Subject: [Rt-devel] PATCHES: Allow RT to authenticate against PAM (updated)
Message-ID: <1196358461.22639.41.camel@squonk.lboro.ac.uk>
Hi all
(For those on -devel, this is essentially the same post as late last
week but with minor changes)
PAM - http://www.kernel.org/pub/linux/libs/pam/ - can authenticate
against a multitude of sources (backends) - an LDAP system, Active
Directory, Kerberos, local passwd/shadow files, a separate MySQL
instance... you name it, it's probably got a module for it somewhere.
This will allow RT admins to use a flexible authentication source
*without* having to modify RT beyond a couple of configuration options.
The changes made are principally to the "IsPassword" function in
lib/RT/CurrentUser.pm such that it can use PAM to check a password, can
use the RT database to check a password, or fall through from PAM to
database if PAM fails.
Additionally, if PAM succeeds and the RT database differs from that
password, there's the option to set the RT password (thus keeping them
in sync).
In combination, if the PAM source goes away, the system can still
authenticate against the RT database using the most recently successful
PAM password. It's possible to turn off the password change and fallback
options, too.
There are both changes to CurrentUser.pm and some additional sections to
RT_SiteConfig.pm. Some of the logging statements could be changed (it's
a bit verbose at the moment) but otherwise it's all as functional as it
can be.
The array defining modules (see below) currently contains one module
name - rt-krb5-auth - but can contain several in order to authenticate
against multiple backends.
NOTE: Any module name passed in this way containing the word "dummy" is
skipped - it's there purely for reference, to see that the code looped
as I wanted it to.
We now have this in production for our support staff within the
department and several other University departments without error.
The example pam.d file included here is for authentication against an
AD. You could just as easily test against the local system, or something
else, given the right config.
Please feel free to give this a try (where relevant). If anyone needs a
hand making PAM fire off against an AD, please feel free to contact me
off list - there's a bit of extra config to make Kerberos work properly.
Graeme
--
Graeme Fowler
Team Manager (ISSS), Computing Services
Loughborough University - http://www.lboro.ac.uk/
T: +44 1509 228226
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rt-krb5-auth.patch
Type: text/x-patch
Size: 197 bytes
Desc: not available
Url : http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071129/2d416fbc/rt-krb5-auth.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CurrentUser.pm.patch
Type: text/x-patch
Size: 2583 bytes
Desc: not available
Url : http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071129/2d416fbc/CurrentUser.pm.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RT_SiteConfig.pm.patch
Type: text/x-patch
Size: 1421 bytes
Desc: not available
Url : http://lists.bestpractical.com/pipermail/rt-devel/attachments/20071129/2d416fbc/RT_SiteConfig.pm.bin