<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Hi Everyone,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>First, some background information. Since we upgraded to RT
3.4.4, we have been seeing the “MySQL server has gone away”
message, which results in tickets not being written to the database. All told,
this has happened to us about 50 times since December, 2005. While the failure
rate is less than .1%, this still posed problems for the parties involved. That
is, the one submitting a ticket thought the ticket made it in the queue; they
get an e-mail. Also, the watchers of the queue also get an e-mail message, but
in the end no ticket makes it in the database. Finally, the only time that this
problem has occurred is when a user submits a ticket via the web interface.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>In researching the MySQL web site, I found that the “MySQL
server has gone away message” could be the results of several things,
i.e. incorrect parameters set in my.cnf, incorrect host name resolution (bad
entry in /etc/resolv.conf), closing the MySQL connection prematurely or not
opening up the connection at all. After investigating the first three
possibilities over time, I turned my attention to the final possibility.
Research indicated that my.cnf was fine (timeouts really were set to large
numbers and the transaction failure was immediate), /etc/resolv.conf was
cleaned up because the third nameserver entry was no longer valid (not likely
this would have caused the problem and this issue was caught after about 20
occurrences of the problem)), premature disconnection (usually occurrs with a MySQL
timeout; again not likely).<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>With that said, I decided to turn on MySQL General Logging
to capture everything that is sent to MySQL, but in text format (this generates
about 1 gb/day). It took about a month, but I finally was able to capture the
elusive error. I also decided to capture a good web based transaction, as well.
After comparing the two, I determined that in the errant situation, the
transactions required to create a ticket, are sent to MySQL before the MySQL “Connect”
is issued. Thus, in a normal log one would see:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>060824 9:26:34
Connect rt_user@localhost on rt3<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>
Query set autocommit=1<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Some housekeeping is done and an Apache Mason session is
created<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Next, the transaction is processed, about 300 or so lines of
Query, Insert and Update MySQL commands.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Housekeeping to end the transaction.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>In the case of when the error occurs, the sequence is:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Some housekeeping is done and an Apache Mason session is
created<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>The transaction is processed, about 300 or so lines of
Query, Insert and Update MySQL commands.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>060823 12:38:00
Connect rt_user@localhost on rt3<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>
Query set autocommit=1<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Housekeeping to end the transaction.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>When one views syslog, each RT transaction to MySQL fails,
because it cannot write to the database. The MySQL logs do not show a problem
has occurred, nor does the end user see any problems, either. However, when a
user enters a ticket, a correctly created ticket is displayed back to the user;
if the ticket was not correctly created they are displayed an error message,
which is probably ignored. As they receive an e-mail, the user thinks the
ticket actually did make it in the database and ignores the error returned by
the web interface. Some user education is a good work around, but this problem
should not be happening in the first place.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Request Tracker is a rather complicated code base and this
problem looks like that under certain conditions the subroutine to connect to
the database is being executed out of proper order; possibly a flag not being
set properly. This could be in Request Tracker proper or in the DBIx::SearchBuilder
module, as this module is the main interface between Request Tracker and MySQL.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>While this message does not provide a solution, it at least
provides an explanation. Also, one hopes that this issue has been fixed in
RT3.6.1.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Take care!<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Nick<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:7.5pt;font-family:Arial;font-weight:bold'>---------------------------------------------------------------------------------</span></font></b><o:p></o:p></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'>Nick
Metrowsky</span></font></b><b><font size=1><span style='font-size:8.0pt;
font-weight:bold'><o:p></o:p></span></font></b></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'>Consulting
System Administrator</span></font></b><b><font size=1><span style='font-size:
8.0pt;font-weight:bold'><o:p></o:p></span></font></b></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'>303-684-4785
Office</span></font></b><b><font size=1><span style='font-size:8.0pt;
font-weight:bold'><o:p></o:p></span></font></b></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'>303-684-4100
Fax</span></font></b><b><font size=1><span style='font-size:8.0pt;font-weight:
bold'><o:p></o:p></span></font></b></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'><a
href="mailto:nmetrowsky@digitalglobe.com"><font size=1><span style='font-size:
7.5pt'>nmetrowsky@digitalglobe.com</span></font></a></span></font></b><b><font
size=1><span style='font-size:8.0pt;font-weight:bold'><o:p></o:p></span></font></b></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:7.5pt;font-family:Arial;font-weight:bold'>DigitalGlobe
®, An Imaging and Information Company</span></font></b><o:p></o:p></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:8.0pt;font-family:Arial;font-weight:bold'><a
href="http://www.digitalglobe.com"><font size=1><span style='font-size:7.5pt'>http://www.digitalglobe.com</span></font></a></span></font></b><o:p></o:p></p>
<p style='margin:0in;margin-bottom:.0001pt;text-indent:-7.5pt'><b><font size=1
face=Arial><span style='font-size:7.5pt;font-family:Arial;font-weight:bold'>---------------------------------------------------------------------------------</span></font></b><o:p></o:p></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
</div>
</body>
</html>