<div dir="ltr">Hey Pavol, <div><br></div><div>Currently, for Quartz and miPoint, I'm using a tomcat7 defined datasource with the following config:<br><br><div> <Resource name="jdbc/mysql" auth="Container" type="javax.sql.DataSource"</div><div> username="midpoint" password="password"</div><div> url="jdbc:mysql://servder:<wbr>3306/midpoint_dev"</div><div> driverClassName="com.mysql.<wbr>jdbc.Driver"</div><div> accessToUnderlyingConnectionAl<wbr>lowed="true"</div><div> initialSize="5" maxWait="5000"</div><div> maxActive="30" maxIdle="5"</div><div> validationQuery="select 1"</div><div> readonly="false" </div><div> poolPreparedStatements="true"/<wbr>><br><br>And here's my relevant config.xml:<br><br><div> <repository></div><div> <embedded>false</embedded></div><div> <<wbr>repositoryServiceFactoryClass><wbr>com.evolveum.midpoint.repo.<wbr>sql.SqlRepositoryFactory</<wbr>repositoryServiceFactoryClass></div><div> <hibernateHbm2ddl>validate</<wbr>hibernateHbm2ddl></div><div> <hibernateDialect>com.<wbr>evolveum.midpoint.repo.sql.<wbr>util.MidPointMySQLDialect</<wbr>hibernateDialect></div><div> <dataSource>java:comp/env/<wbr>jdbc/mysql</dataSource></div><div> </repository></div><div> <taskManager></div><div> <clustered>true</clustered></div><div> <jmxUsername>midpoint</<wbr>jmxUsername></div><div> <jmxPassword>password</<wbr>jmxPassword></div><div> </taskManager></div></div><div><br></div><div>With this configuration, when I was trying to access the Shadows page, I got the errors mentioned in my previous message. </div><div><br></div><div>Following your last email, I added the following <dataSource>java:comp/env/jdbc/mysql</dataSource> under <taskManager>. Now when I try to access the Shadows Page I get the following error: <a href="http://imgur.com/a/391CU">http://imgur.com/a/391CU</a>. Please note there are four photos in that album. Also the reason, I chose an image is due to the fact that this error does not appear anywhere in the logs and it's in the GUI only. </div><div><br></div><div>I would also like to add that even though this error occurs, midPoint still continues to function. Before I added the <dataSource> in the taskManager section, even though I was using a datasource for the Repository, when I tried to access the shadows page, midPoint would not work anymore (due to lack of mysql connections) until I restarted tomcat. </div><div><br></div><div>The last point I want to make is that from what I've seen so far, this issue only happens when I try to access the shadow page. I don't see this issue while trying to access any other type of repository objects. </div><div><br></div><div>Thanks, </div><div>-F</div>
<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 26, 2016 at 3:41 PM, Pavol Mederly <span dir="ltr"><<a href="mailto:mederly@evolveum.com" target="_blank">mederly@evolveum.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello Florin,</p>
<p>it seems that either midPoint (i.e. hibernate), Quartz scheduler,
or Activiti BPMS use connections without properly closing them. <br>
</p>
<p>First step to diagnose it would be perhaps to setup things like
this:</p>
<ol>
<li>Let midPoint and Activiti use their own data source (by
configuring them with a JDBC URL): see <a href="https://wiki.evolveum.com/display/midPoint/Repository+Configuration" target="_blank">Repository
configuration</a> and <a href="https://wiki.evolveum.com/display/midPoint/Workflow+configuration" target="_blank">Workflow
configuration</a> wiki pages. (If you don't use approvals you
can disable workflows altogether by setting "enabled" to
"false".)</li>
<li>Keep Quartz schedule using tomcat data source: something like</li>
</ol>
<div><code><</code><code>taskManager</code><code>></code></div>
<div><code> </code><code><</code><code>clustered</code><code>>true</</code><code>clustered</code><code>><br>
<dataSource>...your data source name...</dataSource><br>
</code></div>
...<br>
<div><code></</code><code>taskManager</code><code>><br>
</code><br>
And you'll see if the problem would appear again. (Maybe you could
even configure three Tomcat data sources; one for midPoint, second
for Quartz, and the third for Activiti, if needed. But I have not
done this yet; these are pure speculations of mine...)<br>
<br>
Best regards,<br>
</div><span class="">
<pre cols="72">Pavol Mederly
Software developer
<a href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span><div><div class="h5"><div>On 24.08.2016 22:41, Florin. Stingaciu
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><font face="Source Sans Pro, Helvetica Neue,
Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px;line-height:20px">I just rebuilt the
latest snapshot (git-v3.5devel-221-g9744fa0) and am
experiencing the same problem. </span></font>
<div><font face="Source Sans Pro, Helvetica Neue, Helvetica,
Arial, sans-serif" color="#333333"><span style="font-size:14px;line-height:20px"><br>
</span></font></div>
<div><font face="Source Sans Pro, Helvetica Neue, Helvetica,
Arial, sans-serif" color="#333333"><span style="font-size:14px;line-height:20px">Thanks,</span></font></div>
<div><font face="Source Sans Pro, Helvetica Neue, Helvetica,
Arial, sans-serif" color="#333333"><span style="font-size:14px;line-height:20px">-F </span></font></div>
<div>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Aug 24, 2016 at 1:24 PM,
Florin. Stingaciu <span dir="ltr"><<a href="mailto:fstingaciu@mirantis.com" target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Upon further investigation I found that this
error only occurs when I try to access the shadow
repository via Configuration->Repository
Objects->Shadows
<div><br>
</div>
<div>It seems that something triggers the creation of DB
connections until it runs out of available connections.
For this particular instance, I'm at <span>git-v3.5devel-170-g26b0644.<wbr> </span></div>
<div><span><br>
</span></div>
<div><span>Thanks, </span></div>
<span><font color="#888888">
<div><span>-F </span></div>
</font></span></div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Aug 24, 2016 at 1:13
PM, Florin. Stingaciu <span dir="ltr"><<a href="mailto:fstingaciu@mirantis.com" target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hello Pavol,
<div><br>
</div>
<div>After running this configuration my dev
environment with two instances up for a couple
of days, I started to get the following error
messages: <a href="http://pastebin.com/nkxE9LNA" target="_blank">http://pastebin.com/<wbr>nkxE9LNA</a></div>
<div><br>
</div>
<div>It seems as though the connection pool is
not properly cleaned up. The database logs
indicate no failure and in fact my production
environment is hooked up to the same DB
instance and is working fine. </div>
<div><br>
</div>
<div>Any recommendations would be very
appreciated. </div>
<div><br>
</div>
<div>Thanks, </div>
<span><font color="#888888">
<div>-F</div>
</font></span></div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Aug 15,
2016 at 3:29 PM, Florin. Stingaciu <span dir="ltr"><<a href="mailto:fstingaciu@mirantis.com" target="_blank">fstingaciu@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Awesome! Thanks for all
your help today Pavol. I will try and
test in my environment and let you
know how that goes.
<div><br>
</div>
<div>Thanks again!</div>
<span><font color="#888888">
<div>-F </div>
</font></span></div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon,
Aug 15, 2016 at 3:00 PM, Pavol
Mederly <span dir="ltr"><<a href="mailto:mederly@evolveum.com" target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Florin,</p>
<p>I think I've fixed it: <a href="https://github.com/Evolveum/midpoint/commit/c00f2979cea6f6ada0a1194b13bad51c0ecec2e3" target="_blank">https://github.com/Evolveum/mi<wbr>dpoint/commit/c00f2979cea6f6ad<wbr>a0a1194b13bad51c0ecec2e3</a><br>
</p>
<p>Cannot verify as the
issue isn't visible on my
computer, but I am quite
convinced it should be
fixed now. The suspected
cause is that some
query-only transactions
weren't closed properly.
C3P0 closes them
automatically but <a href="http://stackoverflow.com/questions/19396563/does-hibernates-session-close-automatically-rollback-uncommitted-transactions" target="_blank">other
connection providers
probably not</a>.</p>
<p>You can safely use the
master, or you can apply
the above patch to 3.4
yourself. The master is
something like
"3.4-support" these days
(preparing for 3.4.1), so
it should be of
production-level quality,
or at least near it. <br>
(I'm going to run full
test suite during the
night, but I don't expect
any failures. Or you can
check after approx 3 hours
<a href="https://travis-ci.org/Evolveum/midpoint" target="_blank">here</a>.)</p>
<span>
<p>Best regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On 15.08.2016
22:57, Florin.
Stingaciu wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hey
Pavol,
<div><br>
</div>
<div>I uploaded the
log file you
requested to my
gdrive (<a href="https://drive.google.com/file/d/0B9f4-vWBsztoZy1zWThGYnpGeGs/view?usp=sharing" target="_blank">https://drive.google.com/file<wbr>/d/0B9f4-vWBsztoZy1zWThGYnpGeG<wbr>s/view?usp=sharing</a>)
It's a trace level
debug so the file
is quite large.
Let me know if
anything jumps out
at you. </div>
<div><br>
Thanks, </div>
<div>-F </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Mon, Aug 15, 2016
at 1:07 PM, Pavol
Mederly <span dir="ltr"><<a href="mailto:mederly@evolveum.com" target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Strange
enough. I've
replicated
your setup on
my computer
(using
PostgreSQL, as
I have no
MySQL
currently
installed),
and it works:
midPoint
starts
cleanly. I am
able to
add/delete
repository
objects.<br>
</p>
<p>Attached
are my
config.xml and
server.xml;
they are
pretty much
the same as
yours.</p>
<p>So, I have
no idea. You
could try
adding
<database>mysql</database>
into
<repository>
element in
your
config.xml,
but it would
most probably
make no
difference.</p>
<p>As last
resort, you
could turn on
the most
detailed
debugging for
the repository
on startup by
adding a line
like this</p>
<p><logger
name="<a href="http://com.evolveum.midpoint.re" target="_blank">com.evolveum.midpoint.re</a><wbr>po"
level="TRACE"
/></p>
<p>into
webapps\midpoint\WEB-INF\class<wbr>es\logback.xml
file and start
midPoint. In
idm.log there
should be
quite a
detailed
record of
what's going
on.<br>
</p>
<span>
<pre cols="72">Pavol Mederly
Software developer
<a href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On
15.08.2016
21:03, Florin.
Stingaciu
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Here's
my datasource
config:<br>
<br>
<div>
<Resource
name="jdbc/mysql"
auth="Container" type="javax.sql.DataSource"</div>
<div>
username="midpoint"
password="pass"</div>
<div>
url="jdbc:mysql://SERVER:3306/<wbr>midpoint_dev"</div>
<div>
driverClassName="com.mysql.jdb<wbr>c.Driver"</div>
<div>
accessToUnderlyingConnectionAl<wbr>lowed="true"</div>
<div>
initialSize="5"
maxWait="5000"</div>
<div>
maxActive="30"
maxIdle="5"</div>
<div>
validationQuery="select
1"</div>
<div>
poolPreparedStatements="true"/<wbr>></div>
<div><br>
</div>
<div>And
here's my
config.xml:<br>
<br>
<div>
<repository></div>
<div>
<embedded>false</embedded></div>
<div>
<repositoryServiceFactoryClass<wbr>>com.evolveum.midpoint.repo.sq<wbr>l.SqlRepositoryFactory</reposi<wbr>toryServiceFactoryClass></div>
<div>
<hibernateHbm2ddl>validate</hi<wbr>bernateHbm2ddl></div>
<div>
<hibernateDialect>com.evolveum<wbr>.midpoint.repo.sql.util.MidPoi<wbr>ntMySQLDialect</hibernateDiale<wbr>ct></div>
<div>
<dataSource>java:comp/env/jdbc<wbr>/mysql</dataSource></div>
<div>
</repository></div>
<div>
<taskManager></div>
<div>
<clustered>true</clustered></div>
<div>
<jmxUsername>midpoint</jmxUser<wbr>name></div>
<div>
<jmxPassword>password</jmxPass<wbr>word></div>
<div>
</taskManager></div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Mon, Aug 15,
2016 at 11:44
AM, Pavol
Mederly <span dir="ltr"><<a href="mailto:mederly@evolveum.com" target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello
Florin,</p>
<p>this is
really
interesting.
Please, could
you also share
your midPoint
config.xml, as
well as your
data source
configuration?
(except
credentials,
of course)</p>
<span>
<p>Best
regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
</span>
<div>
<div>
<div>On
15.08.2016
19:39, Florin.
Stingaciu
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello
Pavol,
<div><br>
</div>
<div>Thanks
for your
detailed
response. I
tried setting
up the
datasource
with
validationQuery
set up
properly for
the mySQL
backed I have.
However, upon
service
restart I get
the following
errors: <a href="http://pastebin.com/8dpGN0JC" target="_blank">http://pastebin.com/8d<wbr>pGN0JC</a><br>
<br>
To save you a
click, it
seems as
though the
connection is
set up in
readonly mode
or some other
strange things
happen. I've
tried setting <span>readonly</span><span>=</span><span>"false"</span><span> in
the resource
definition in
server.xml but
that didn't
help. I will
continue
researching
this, however
any guidance
would be quite
appreciated. </span></div>
<div><span><br>
</span></div>
<div><span>Thanks!</span></div>
<div><span>-F
</span></div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Mon, Aug 15,
2016 at 2:29
AM, Pavol
Mederly <span dir="ltr"><<a href="mailto:mederly@evolveum.com" target="_blank">mederly@evolveum.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hello
Florin,</p>
<p>having
looked at your
logs, it seems
that maybe
explicit
setting of
validationQuery
in Quartz data
source setup
would help.
(See <a href="http://www.quartz-scheduler.org/documentation/quartz-2.x/configuration/ConfigDataSources.html" target="_blank">http://www.quartz-scheduler.or<wbr>g/documentation/quartz-2.x/con<wbr>figuration/ConfigDataSources.h<wbr>tml</a>.)</p>
<p>Unfortunately,
current
midPoint
implementation
does not allow
to configure
Quartz data
source
parameters.
So, there are
the following
three
possibilities:</p>
<ol>
<li>Take an
alternative
route, and use
application
server-defined
data source
(with
validationQuery
set up).<br>
</li>
<li>Patch Task
Manager
implementation
by adding
appropriate
lines to
Quartz
configuration
(see
LocalNodeManager.java:87-90).</li>
<li>Wait until
we implement
it - I've
created an
issue <a href="https://jira.evolveum.com/browse/MID-3347" target="_blank">MID-3347</a>
for this.<br>
</li>
</ol>
<p>As for the
first one
(externally
defined data
source):
Please see <a href="https://wiki.evolveum.com/display/midPoint/Repository+Configuration#RepositoryConfiguration-Datasourceconfiguration" target="_blank">https://wiki.evolveum.com/disp<wbr>lay/midPoint/Repository+Config<wbr>uration#RepositoryConfiguratio<wbr>n-Datasourceconfiguration</a>
on how to
configure
midPoint
repository
with the data
source. This
data source
will be used
also by
Quartz, if not
overriden in
<taskManager>
section. It
should work
but I don't
remember if
someone
actually
tested this.</p>
<p>Concerning
autoReconnect:
I have no
experiences
with this
setting. I
agree with you
that about the
hesitation of
using it in
production
environment.
If really
needed, I'd
recommend to
separate
midPoint
repository
configuration
from Quartz
configuration
by using
different JDBC
URLs for the
two: the
standard one
for the
repository and
the one with
"autoReconnect=true"
for Quartz. In
this way,
potential
negative
effects should
be restricted
to task
management
functionality
only. But,
overall, I'd
suggest trying
to eliminate
the problem by
setting
validationQuery
first.<br>
</p>
<p>Best
regards,<br>
</p>
<pre cols="72">Pavol Mederly
Software developer
<a href="http://evolveum.com" target="_blank">evolveum.com</a>
</pre>
<div>
<div>
<div>On
14.08.2016
22:40, Florin.
Stingaciu
wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">Hello,
<div><br>
</div>
<div>I'm
trying to
configure an
active/active
configuration,
and
experiencing
some issues
with the
Quartz
scheduler. The
SQL connection
seems to
timeout quite
often and
result in many
warning
messages. I'm
also
experiencing
some errors --
as the timeout
closes the
connection,
some processes
are still
trying to
commit using
that stale
handler. </div>
<div><br>
</div>
<div>Here are
some relevant
logs from both
midPoint
instances in
this cluster: <a href="http://pastebin.com/JFjqrLnT" target="_blank">http://pastebin.com/J<wbr>FjqrLnT</a> <a href="http://pastebin.com/1vy4iYPY" target="_blank">http://pastebin.com/1v<wbr>y4iYPY</a><br>
<br>
Also here's
the relevant
portion of my
config.xml
file: <a href="http://pastebin.com/WbydcCC8" target="_blank">http://pastebin.com/Wbyd<wbr>cCC8</a></div>
<div><br>
Following the
suggestions in
the warnings,
I've started
to look at
autoReconnect
propriety of
the JDBC
connector and
applied it to
my configs
like so:</div>
<div><br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span><jdbcUrl>jdbc:mysql://SERVER:3<wbr>306/midpoint_dev?autoReconnect<wbr>=true</jdbcUrl></span></blockquote>
<div><br>
</div>
<div>Since
implementing
this change,
the errors and
warnings seem
to have
disappeared. I
will continue
to monitor the
logs and
ensure this
actually the
case. </div>
<div><br>
</div>
<div>Reading
the mysql
docs, I found
that this is
not
recommended as
this may cause
data
inconsistency
issues and
that stale
connection
exceptions
should be
properly
caught within
the
application.
Namely:</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">The
use of this
feature is not
recommended,
because it has
side effects
related to
session state
and data
consistency
when
applications
don't handle
SQLExceptions
properly, and
is only
designed to be
used when you
are unable to
configure your
application to
handle
SQLExceptions
resulting from
dead and stale
connections
properly.</blockquote>
<div><br>
Do you have
any
recommended
configuration
for this
scenario? I'd
like to move
my current
production
environment in
an active
active
configuration,
however as of
right now I'm
hesitant to do
so in order to
avoid any data
corruption.
Especially
since it's
quite
difficult to
test for data
consistency
issues that
may arise from
using
autoReconnect
in my dev
environment. <br>
<br>
Thanks, </div>
<div>-F </div>
<div> </div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div>
______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div>
______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mail<wbr>man/listinfo/midpoint</a>
</blockquote></div>
</div>
</div></div></blockquote></div>
</div>
</div></div></blockquote></div>
</div>
</div></div></blockquote></div>
</div>
<fieldset></fieldset>
<pre>______________________________<wbr>_________________
midPoint mailing list
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" target="_blank">http://lists.evolveum.com/<wbr>mailman/listinfo/midpoint</a>
</pre>
</blockquote>
</div></div></div><br>______________________________<wbr>_________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/<wbr>mailman/listinfo/midpoint</a><br>
<br></blockquote></div><br></div>