<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Florin,</p>
<p>this is really interesting. The exception of <br>
</p>
<p><tt>Caused by:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The
last packet successfully received from the server was 64,841,997
milliseconds ago. The last packet sent successfully to the
server was 64,841,998 milliseconds ago. is longer than the
server configured value of 'wait_timeout'. You should consider
either expiring and/or testing connection validity before use in
your application, increasing the server configured values for
client timeouts, or using the Connector/J connection property
'autoReconnect=true' to avoid this problem.</tt></p>
<p>occurs in the context of Quartz scheduler, where the data source
that checks connection validity is used!</p>
<p>When you start midPoint there should be messages like these:</p>
<p><tt>2016-08-30 20:14:31,388 [] [RMI TCP Connection(5)-127.0.0.1]
INFO (com.evolveum.midpoint.task.quartzimpl.Initializer): Task
Manager: Quartz Job Store: JDBC, NOT clustered. Threads: 10</tt><tt><br>
</tt><tt>2016-08-30 20:14:31,388 [] [RMI TCP
Connection(5)-127.0.0.1] INFO
(com.evolveum.midpoint.task.quartzimpl.TaskManagerConfiguration):
Quartz database is at jdbc:postgresql://localhost/midpoint (a
JDBC URL)</tt></p>
<p>In your case there should be a reference to the data source.
Could you please verify it is OK, and perhaps paste relevant lines
here?<br>
</p>
<pre class="moz-signature" cols="72">Pavol Mederly
Software developer
evolveum.com
</pre>
<div class="moz-cite-prefix">On 30.08.2016 19:21, Florin. Stingaciu
wrote:<br>
</div>
<blockquote
cite="mid:CAMQHPY1er5vPaE0aHRFSLKHM7cuKDHhmc_K6Lvo2oLJOsP35Mw@mail.gmail.com"
type="cite">
<div dir="ltr">Hey Pavol,
<div><br>
</div>
<div>Over night, I now have the same error messages I initially
had: <a moz-do-not-send="true"
href="http://pastebin.com/iWeSaswD">http://pastebin.com/iWeSaswD</a></div>
<div><br>
</div>
<div>Thank you for all your help so far. Any further
recommendations would be really appreciated. </div>
<div><br>
</div>
<div>Thanks, </div>
<div>-F </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Aug 29, 2016 at 3:54 PM,
Florin. Stingaciu <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:fstingaciu@mirantis.com" target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hey Pavol,
<div><br>
</div>
<div>I switched the repository to the configuration you
mentioned, and I could now access the shadow page.
However (minus the task manager config), this is the
same configuration as I previously had where I ran into
errors and then made this thread because of it. For now
I'm not seeing any errors in the log files. </div>
<div><br>
</div>
<div>I'll update this thread once I run into issues. As of
right now, I'm still definitely not comfortable moving
to active active considering all these issues. Have you
guys seen similar issues before? Or do you believe I've
just misconfigured something?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>-F </div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Aug 29, 2016 at 3:39
PM, Pavol Mederly <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello Florin,</p>
<p>I've tried to use your setup in my
environment (slightly modified), but I was not
able to replicate the exceptions. I've tried
in particular to play with the listing of
shadow objects.</p>
<p>So we perhaps have to continue diagnosing on
your side. <br>
</p>
<p>I suggest to try to isolate the problem. If
you could use the following config:</p>
<p><span><span> </span></span></p>
<div> <repository></div>
<div>
<embedded>false</embedded></div>
<div>
<repositoryServiceFactoryClass<wbr>>com.evolveum.midpoint.repo.sq<wbr>l.SqlRepositoryFactory</reposi<wbr>toryServiceFactoryClass></div>
<div>
<hibernateHbm2ddl>validate</hi<wbr>bernateHbm2ddl></div>
<div>
<database>mysql</database><br>
</div>
<div>
<jdbcUsername>...</jdbcUsernam<wbr>e><br>
<jdbcPassword>...</jdbcPasswor<wbr>d><br>
<jdbcUrl>jdbc:mysql://servder:<wbr>3306/midpoint_dev</jdbcUrl><br>
</repository></div>
<span>
<div> <taskManager></div>
<div>
<clustered>true</clustered></div>
<div>
<jmxUsername>midpoint</jmxUser<wbr>name></div>
</span>
<div>
<jmxPassword>password</jmxPass<wbr>word><br>
<dataSource><span>java:comp/env/jdbc<wbr>/mysql</span></dataSource><br>
</div>
<div> </taskManager><br>
<workflow><br>
<enabled>false</enabled><br>
<span></span>
<div> </workflow><br>
</div>
<div><br>
</div>
...and try the testing. The configuration
simply makes midPoint use its own connection
mechanism (c3p0) and Quartz use the tomcat's
connection pool.<br>
</div>
<span>
<div>Best regards,<br>
</div>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On 29.08.2016 20:17, Florin. Stingaciu
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">As an update, after having
the instance up for half an hour or so
with the above config, I got the
following errors again: <a
moz-do-not-send="true"
href="http://pastebin.com/tJJVFVsu"
target="_blank">http://pastebin.com/tJJ<wbr>VFVsu</a>
<div><br>
</div>
<div>Also, in the deployment, there's no
workflows at all and very little user
load as it's just a dev env.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>-F </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Aug 29,
2016 at 10:43 AM, Florin. Stingaciu <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:fstingaciu@mirantis.com"
target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">Hey Pavol,
<div><br>
</div>
<div>Currently, for Quartz and
miPoint, I'm using a tomcat7
defined datasource with the
following config:<span><br>
<br>
<div> <Resource
name="jdbc/mysql"
auth="Container"
type="javax.sql.DataSource"</div>
</span>
<div> username="midpoint"
password="password"</div>
<div>
url="jdbc:mysql://servder:3306<wbr>/midpoint_dev"</div>
<span>
<div>
driverClassName="com.mysql.jdb<wbr>c.Driver"</div>
<div>
accessToUnderlyingConnectionAl<wbr>lowed="true"</div>
<div> initialSize="5"
maxWait="5000"</div>
<div> maxActive="30"
maxIdle="5"</div>
<div>
validationQuery="select 1"</div>
</span>
<div> readonly="false"
</div>
<div>
poolPreparedStatements="true"/<wbr>><br>
<br>
And here's my relevant
config.xml:<span><br>
<br>
<div>
<repository></div>
<div>
<embedded>false</embedded></div>
<div>
<repositoryServiceFactoryClass<wbr>>com.evolveum.midpoint.repo.sq<wbr>l.SqlRepositoryFactory</reposi<wbr>toryServiceFactoryClass></div>
<div>
<hibernateHbm2ddl>validate</hi<wbr>bernateHbm2ddl></div>
<div>
<hibernateDialect>com.evolveum<wbr>.midpoint.repo.sql.util.MidPoi<wbr>ntMySQLDialect</hibernateDiale<wbr>ct></div>
<div>
<dataSource>java:comp/env/jdbc<wbr>/mysql</dataSource></div>
<div>
</repository></div>
<div>
<taskManager></div>
<div>
<clustered>true</clustered></div>
<div>
<jmxUsername>midpoint</jmxUser<wbr>name></div>
<div>
<jmxPassword>password</jmxPass<wbr>word></div>
<div>
</taskManager></div>
</span></div>
<div><br>
</div>
<div>With this configuration,
when I was trying to access
the Shadows page, I got the
errors mentioned in my
previous message. </div>
<div><br>
</div>
<div>Following your last email,
I added the following
<dataSource>java:comp/env/jdbc<wbr>/mysql</dataSource>
under <taskManager>. Now
when I try to access the
Shadows Page I get the
following error: <a
moz-do-not-send="true"
href="http://imgur.com/a/391CU"
target="_blank">http://imgur.com/a/391C<wbr>U</a>.
Please note there are four
photos in that album. Also the
reason, I chose an image is
due to the fact that this
error does not appear anywhere
in the logs and it's in the
GUI only. </div>
<div><br>
</div>
<div>I would also like to add
that even though this error
occurs, midPoint still
continues to function. Before
I added the <dataSource>
in the taskManager section,
even though I was using a
datasource for the Repository,
when I tried to access the
shadows page, midPoint would
not work anymore (due to lack
of mysql connections) until I
restarted tomcat. </div>
<div><br>
</div>
<div>The last point I want to
make is that from what I've
seen so far, this issue only
happens when I try to access
the shadow page. I don't see
this issue while trying to
access any other type of
repository objects. </div>
<div><br>
</div>
<div>Thanks, </div>
<span><font color="#888888">
<div>-F</div>
<br>
</font></span></div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Fri, Aug 26, 2016 at 3:41
PM, Pavol Mederly <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000">
<p>Hello Florin,</p>
<p>it seems that either
midPoint (i.e.
hibernate), Quartz
scheduler, or Activiti
BPMS use connections
without properly
closing them. <br>
</p>
<p>First step to
diagnose it would be
perhaps to setup
things like this:</p>
<ol>
<li>Let midPoint and
Activiti use their
own data source (by
configuring them
with a JDBC URL):
see <a
moz-do-not-send="true"
href="https://wiki.evolveum.com/display/midPoint/Repository+Configuration"
target="_blank">Repository
configuration</a>
and <a
moz-do-not-send="true"
href="https://wiki.evolveum.com/display/midPoint/Workflow+configuration"
target="_blank">Workflow
configuration</a>
wiki pages. (If you
don't use approvals
you can disable
workflows altogether
by setting "enabled"
to "false".)</li>
<li>Keep Quartz
schedule using
tomcat data source:
something like</li>
</ol>
<div><code><</code><code>taskManager</code><code>></code></div>
<div><code> </code><code><</code><code>clustered</code><code>>true</</code><code>clustered</code><code>><br>
<dataSource>...your
data source
name...</dataSource><br>
</code></div>
...<br>
<div><code></</code><code>taskManager</code><code>><br>
</code><br>
And you'll see if the
problem would appear
again. (Maybe you
could even configure
three Tomcat data
sources; one for
midPoint, second for
Quartz, and the third
for Activiti, if
needed. But I have not
done this yet; these
are pure speculations
of mine...)<br>
<br>
Best regards,<br>
</div>
<span>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On 24.08.2016
22:41, Florin.
Stingaciu wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr"><font
face="Source
Sans Pro,
Helvetica
Neue,
Helvetica,
Arial,
sans-serif"
color="#333333"><span
style="font-size:14px;line-height:20px">I just rebuilt the latest
snapshot
(git-v3.5devel-221-g9744fa0)
and am
experiencing
the same
problem. </span></font>
<div><font
face="Source
Sans Pro,
Helvetica
Neue,
Helvetica,
Arial,
sans-serif"
color="#333333"><span
style="font-size:14px;line-height:20px"><br>
</span></font></div>
<div><font
face="Source
Sans Pro,
Helvetica
Neue,
Helvetica,
Arial,
sans-serif"
color="#333333"><span
style="font-size:14px;line-height:20px">Thanks,</span></font></div>
<div><font
face="Source
Sans Pro,
Helvetica
Neue,
Helvetica,
Arial,
sans-serif"
color="#333333"><span
style="font-size:14px;line-height:20px">-F </span></font></div>
<div>
<div><br>
</div>
</div>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, Aug 24,
2016 at 1:24
PM, Florin.
Stingaciu <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:fstingaciu@mirantis.com"
target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Upon
further
investigation
I found that
this error
only occurs
when I try to
access the
shadow
repository via
Configuration->Repository Objects->Shadows
<div><br>
</div>
<div>It seems
that something
triggers the
creation of DB
connections
until it runs
out of
available
connections.
For this
particular
instance, I'm
at <span>git-v3.5devel-170-g26b0644.<wbr> </span></div>
<div><span><br>
</span></div>
<div><span>Thanks, </span></div>
<span><font
color="#888888">
<div><span>-F </span></div>
</font></span></div>
<div>
<div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, Aug 24,
2016 at 1:13
PM, Florin.
Stingaciu <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:fstingaciu@mirantis.com"
target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hello
Pavol,
<div><br>
</div>
<div>After
running this
configuration
my dev
environment
with two
instances up
for a couple
of days, I
started to get
the following
error
messages: <a
moz-do-not-send="true"
href="http://pastebin.com/nkxE9LNA" target="_blank">http://pastebin.com/<wbr>nkxE9LNA</a></div>
<div><br>
</div>
<div>It seems
as though the
connection
pool is not
properly
cleaned up.
The database
logs indicate
no failure and
in fact my
production
environment is
hooked up to
the same DB
instance and
is working
fine. </div>
<div><br>
</div>
<div>Any
recommendations
would be very
appreciated. </div>
<div><br>
</div>
<div>Thanks, </div>
<span><font
color="#888888">
<div>-F</div>
</font></span></div>
<div>
<div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Aug 15,
2016 at 3:29
PM, Florin.
Stingaciu <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:fstingaciu@mirantis.com"
target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Awesome!
Thanks for all
your help
today Pavol. I
will try and
test in my
environment
and let you
know how that
goes.
<div><br>
</div>
<div>Thanks
again!</div>
<span><font
color="#888888">
<div>-F </div>
</font></span></div>
<div>
<div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Aug 15,
2016 at 3:00
PM, Pavol
Mederly <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<p>Florin,</p>
<p>I think
I've fixed it:
<a
moz-do-not-send="true"
href="https://github.com/Evolveum/midpoint/commit/c00f2979cea6f6ada0a1194b13bad51c0ecec2e3"
target="_blank">https://github.com/Evolveum/mi<wbr>dpoint/commit/c00f2979cea6f6ad<wbr>a0a1194b13bad51c0ecec2e3</a><br>
</p>
<p>Cannot
verify as the
issue isn't
visible on my
computer, but
I am quite
convinced it
should be
fixed now. The
suspected
cause is that
some
query-only
transactions
weren't closed
properly. C3P0
closes them
automatically
but <a
moz-do-not-send="true"
href="http://stackoverflow.com/questions/19396563/does-hibernates-session-close-automatically-rollback-uncommitted-transactions"
target="_blank">other connection providers probably not</a>.</p>
<p>You can
safely use the
master, or you
can apply the
above patch to
3.4 yourself.
The master is
something like
"3.4-support"
these days
(preparing for
3.4.1), so it
should be of
production-level
quality, or at
least near
it. <br>
(I'm going to
run full test
suite during
the night, but
I don't expect
any failures.
Or you can
check after
approx 3 hours
<a
moz-do-not-send="true"
href="https://travis-ci.org/Evolveum/midpoint" target="_blank">here</a>.)</p>
<span>
<p>Best
regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On
15.08.2016
22:57, Florin.
Stingaciu
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hey
Pavol,
<div><br>
</div>
<div>I
uploaded the
log file you
requested to
my gdrive (<a
moz-do-not-send="true"
href="https://drive.google.com/file/d/0B9f4-vWBsztoZy1zWThGYnpGeGs/view?usp=sharing"
target="_blank">https://drive.google.com/file<wbr>/d/0B9f4-vWBsztoZy1zWThGYnpGeG<wbr>s/view?usp=sharing</a>)
It's a trace
level debug so
the file is
quite large.
Let me know if
anything jumps
out at you. </div>
<div><br>
Thanks, </div>
<div>-F </div>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Aug 15,
2016 at 1:07
PM, Pavol
Mederly <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<p>Strange
enough. I've
replicated
your setup on
my computer
(using
PostgreSQL, as
I have no
MySQL
currently
installed),
and it works:
midPoint
starts
cleanly. I am
able to
add/delete
repository
objects.<br>
</p>
<p>Attached
are my
config.xml and
server.xml;
they are
pretty much
the same as
yours.</p>
<p>So, I have
no idea. You
could try
adding
<database>mysql</database>
into
<repository>
element in
your
config.xml,
but it would
most probably
make no
difference.</p>
<p>As last
resort, you
could turn on
the most
detailed
debugging for
the repository
on startup by
adding a line
like this</p>
<p><logger
name="<a
moz-do-not-send="true"
href="http://com.evolveum.midpoint.re" target="_blank">com.evolveum.midpoint.re</a><wbr>po"
level="TRACE"
/></p>
<p>into
webapps\midpoint\WEB-INF\class<wbr>es\logback.xml
file and start
midPoint. In
idm.log there
should be
quite a
detailed
record of
what's going
on.<br>
</p>
<span>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On
15.08.2016
21:03, Florin.
Stingaciu
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Here's
my datasource
config:<br>
<br>
<div>
<Resource
name="jdbc/mysql"
auth="Container" type="javax.sql.DataSource"</div>
<div>
username="midpoint"
password="pass"</div>
<div>
url="jdbc:mysql://SERVER:3306/<wbr>midpoint_dev"</div>
<div>
driverClassName="com.mysql.jdb<wbr>c.Driver"</div>
<div>
accessToUnderlyingConnectionAl<wbr>lowed="true"</div>
<div>
initialSize="5"
maxWait="5000"</div>
<div>
maxActive="30"
maxIdle="5"</div>
<div>
validationQuery="select
1"</div>
<div>
poolPreparedStatements="true"/<wbr>></div>
<div><br>
</div>
<div>And
here's my
config.xml:<br>
<br>
<div>
<repository></div>
<div>
<embedded>false</embedded></div>
<div>
<repositoryServiceFactoryClass<wbr>>com.evolveum.midpoint.repo.sq<wbr>l.SqlRepositoryFactory</reposi<wbr>toryServiceFactoryClass></div>
<div>
<hibernateHbm2ddl>validate</hi<wbr>bernateHbm2ddl></div>
<div>
<hibernateDialect>com.evolveum<wbr>.midpoint.repo.sql.util.MidPoi<wbr>ntMySQLDialect</hibernateDiale<wbr>ct></div>
<div>
<dataSource>java:comp/env/jdbc<wbr>/mysql</dataSource></div>
<div>
</repository></div>
<div>
<taskManager></div>
<div>
<clustered>true</clustered></div>
<div>
<jmxUsername>midpoint</jmxUser<wbr>name></div>
<div>
<jmxPassword>password</jmxPass<wbr>word></div>
<div>
</taskManager></div>
</div>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Aug 15,
2016 at 11:44
AM, Pavol
Mederly <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<p>Hello
Florin,</p>
<p>this is
really
interesting.
Please, could
you also share
your midPoint
config.xml, as
well as your
data source
configuration?
(except
credentials,
of course)</p>
<span>
<p>Best
regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On
15.08.2016
19:39, Florin.
Stingaciu
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hello
Pavol,
<div><br>
</div>
<div>Thanks
for your
detailed
response. I
tried setting
up the
datasource
with
validationQuery
set up
properly for
the mySQL
backed I have.
However, upon
service
restart I get
the following
errors: <a
moz-do-not-send="true"
href="http://pastebin.com/8dpGN0JC" target="_blank">http://pastebin.com/8d<wbr>pGN0JC</a><br>
<br>
To save you a
click, it
seems as
though the
connection is
set up in
readonly mode
or some other
strange things
happen. I've
tried setting <span>readonly</span><span>=</span><span>"false"</span><span> in
the resource
definition in
server.xml but
that didn't
help. I will
continue
researching
this, however
any guidance
would be quite
appreciated. </span></div>
<div><span><br>
</span></div>
<div><span>Thanks!</span></div>
<div><span>-F
</span></div>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Mon, Aug 15,
2016 at 2:29
AM, Pavol
Mederly <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:mederly@evolveum.com"
target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<p>Hello
Florin,</p>
<p>having
looked at your
logs, it seems
that maybe
explicit
setting of
validationQuery
in Quartz data
source setup
would help.
(See <a
moz-do-not-send="true"
href="http://www.quartz-scheduler.org/documentation/quartz-2.x/configuration/ConfigDataSources.html"
target="_blank">http://www.quartz-scheduler.or<wbr>g/documentation/quartz-2.x/con<wbr>figuration/ConfigDataSources.h<wbr>tml</a>.)</p>
<p>Unfortunately,
current
midPoint
implementation
does not allow
to configure
Quartz data
source
parameters.
So, there are
the following
three
possibilities:</p>
<ol>
<li>Take an
alternative
route, and use
application
server-defined
data source
(with
validationQuery
set up).<br>
</li>
<li>Patch Task
Manager
implementation
by adding
appropriate
lines to
Quartz
configuration
(see
LocalNodeManager.java:87-90).</li>
<li>Wait until
we implement
it - I've
created an
issue <a
moz-do-not-send="true"
href="https://jira.evolveum.com/browse/MID-3347" target="_blank">MID-3347</a>
for this.<br>
</li>
</ol>
<p>As for the
first one
(externally
defined data
source):
Please see <a
moz-do-not-send="true"
href="https://wiki.evolveum.com/display/midPoint/Repository+Configuration#RepositoryConfiguration-Datasourceconfiguration"
target="_blank">https://wiki.evolveum.com/disp<wbr>lay/midPoint/Repository+Config<wbr>uration#RepositoryConfiguratio<wbr>n-Datasourceconfiguration</a>
on how to
configure
midPoint
repository
with the data
source. This
data source
will be used
also by
Quartz, if not
overriden in
<taskManager>
section. It
should work
but I don't
remember if
someone
actually
tested this.</p>
<p>Concerning
autoReconnect:
I have no
experiences
with this
setting. I
agree with you
that about the
hesitation of
using it in
production
environment.
If really
needed, I'd
recommend to
separate
midPoint
repository
configuration
from Quartz
configuration
by using
different JDBC
URLs for the
two: the
standard one
for the
repository and
the one with
"autoReconnect=true"
for Quartz. In
this way,
potential
negative
effects should
be restricted
to task
management
functionality
only. But,
overall, I'd
suggest trying
to eliminate
the problem by
setting
validationQuery
first.<br>
</p>
<p>Best
regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a moz-do-not-send="true" href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
<div>
<div>
<div>On
14.08.2016
22:40, Florin.
Stingaciu
wrote:<br>
</div>
</div>
</div>
<blockquote
type="cite">
<div>
<div>
<div dir="ltr">Hello,
<div><br>
</div>
<div>I'm
trying to
configure an
active/active
configuration,
and
experiencing
some issues
with the
Quartz
scheduler. The
SQL connection
seems to
timeout quite
often and
result in many
warning
messages. I'm
also
experiencing
some errors --
as the timeout
closes the
connection,
some processes
are still
trying to
commit using
that stale
handler. </div>
<div><br>
</div>
<div>Here are
some relevant
logs from both
midPoint
instances in
this cluster: <a
moz-do-not-send="true" href="http://pastebin.com/JFjqrLnT"
target="_blank">http://pastebin.com/J<wbr>FjqrLnT</a> <a
moz-do-not-send="true" href="http://pastebin.com/1vy4iYPY"
target="_blank">http://pastebin.com/1v<wbr>y4iYPY</a><br>
<br>
Also here's
the relevant
portion of my
config.xml
file: <a
moz-do-not-send="true"
href="http://pastebin.com/WbydcCC8" target="_blank">http://pastebin.com/Wbyd<wbr>cCC8</a></div>
<div><br>
Following the
suggestions in
the warnings,
I've started
to look at
autoReconnect
propriety of
the JDBC
connector and
applied it to
my configs
like so:</div>
<div><br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span><jdbcUrl>jdbc:mysql://SERVER:3<wbr>306/midpoint_dev?autoReconnect<wbr>=true</jdbcUrl></span></blockquote>
<div><br>
</div>
<div>Since
implementing
this change,
the errors and
warnings seem
to have
disappeared. I
will continue
to monitor the
logs and
ensure this
actually the
case. </div>
<div><br>
</div>
<div>Reading
the mysql
docs, I found
that this is
not
recommended as
this may cause
data
inconsistency
issues and
that stale
connection
exceptions
should be
properly
caught within
the
application.
Namely:</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">The
use of this
feature is not
recommended,
because it has
side effects
related to
session state
and data
consistency
when
applications
don't handle
SQLExceptions
properly, and
is only
designed to be
used when you
are unable to
configure your
application to
handle
SQLExceptions
resulting from
dead and stale
connections
properly.</blockquote>
<div><br>
Do you have
any
recommended
configuration
for this
scenario? I'd
like to move
my current
production
environment in
an active
active
configuration,
however as of
right now I'm
hesitant to do
so in order to
avoid any data
corruption.
Especially
since it's
quite
difficult to
test for data
consistency
issues that
may arise from
using
autoReconnect
in my dev
environment. <br>
<br>
Thanks, </div>
<div>-F </div>
<div> </div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
</div></div></blockquote></div>
</div>
</div></div></blockquote></div>
</div>
</div></div></blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
</div></div></blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a moz-do-not-send="true" href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a moz-do-not-send="true" href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
</div></div></blockquote></div>
</div>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre wrap="">_______________________________________________
midPoint mailing list
<a class="moz-txt-link-abbreviated" href="mailto:midPoint@lists.evolveum.com">midPoint@lists.evolveum.com</a>
<a class="moz-txt-link-freetext" href="http://lists.evolveum.com/mailman/listinfo/midpoint">http://lists.evolveum.com/mailman/listinfo/midpoint</a>
</pre>
</blockquote>
</body></html>