<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>I've just looked at the details...</p>
    <p><font face="monospace">2022-10-03 19:37:01,932 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        WARN (org.quartz.impl.jdbcjobstore.JobStoreTX): This scheduler
        instance (ilstu-midpoint-dev-02) is still active but was
        recovered by another instance in the cluster.  This may cause
        inconsistent behavior.<br>
        2022-10-03 19:48:25,090 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        INFO (org.quartz.impl.jdbcjobstore.JobStoreTX): ClusterManager:
        detected 1 failed or restarted instances.<br>
        2022-10-03 19:48:25,090 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        INFO (org.quartz.impl.jdbcjobstore.JobStoreTX): ClusterManager:
        Scanning for instance "ilstu-midpoint-dev-01"'s failed
        in-progress jobs.<br>
        2022-10-03 19:50:11,947 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        WARN (org.quartz.impl.jdbcjobstore.JobStoreTX): This scheduler
        instance (ilstu-midpoint-dev-02) is still active but was
        recovered by another instance in the cluster.  This may cause
        inconsistent behavior.<br>
        2022-10-03 20:05:53,283 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        WARN (org.quartz.impl.jdbcjobstore.JobStoreTX): This scheduler
        instance (ilstu-midpoint-dev-02) is still active but was
        recovered by another instance in the cluster.  This may cause
        inconsistent behavior.<br>
        2022-10-03 20:19:10,271 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        WARN (org.quartz.impl.jdbcjobstore.JobStoreTX): This scheduler
        instance (ilstu-midpoint-dev-02) is still active but was
        recovered by another instance in the cluster.  This may cause
        inconsistent behavior.<br>
        2022-10-03 20:32:45,142 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        INFO (org.quartz.impl.jdbcjobstore.JobStoreTX): ClusterManager:
        detected 1 failed or restarted instances.<br>
        2022-10-03 20:32:45,142 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        INFO (org.quartz.impl.jdbcjobstore.JobStoreTX): ClusterManager:
        Scanning for instance "ilstu-midpoint-dev-01"'s failed
        in-progress jobs.<br>
        2022-10-03 20:32:45,164 []
        [QuartzScheduler_midPointScheduler-ilstu-midpoint-dev-02_ClusterManager]
        INFO (org.quartz.impl.jdbcjobstore.JobStoreTX): ClusterManager:
        ......Scheduled 1 recoverable job(s) for recovery.<br>
        2022-10-03 20:32:45,193 [] [midPointScheduler_Worker-9] INFO
        (com.evolveum.midpoint.task.quartzimpl.run.JobExecutor): Task
        Task(id:1664827507317-0-1, name:identityTable (livesynch),
        oid:9d446a2e-0819-4041-9f66-bf716d76b084) is recovering<br>
      </font></p>
    <p>This points to the cluster issue with 100% certainty. :)<br>
    </p>
    <pre class="moz-signature" cols="72">-- 
Pavol Mederly
Software developer
evolveum.com</pre>
    <div class="moz-cite-prefix">On 04/10/2022 20:32, Haywood, Jeremiah
      via midPoint wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CH0PR03MB6050ED1D3775958554B6B85EE45A9@CH0PR03MB6050.namprd03.prod.outlook.com">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
      <style>@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        font-size:11.0pt;
        font-family:"Calibri",sans-serif;}span.EmailStyle19
        {mso-style-type:personal-compose;
        font-family:"Calibri",sans-serif;
        color:windowtext;}.MsoChpDefault
        {mso-style-type:export-only;
        font-family:"Calibri",sans-serif;}div.WordSection1
        {page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
      <div class="WordSection1">
        <p class="MsoNormal">Afternoon all,<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">We are experimenting with live synch in a
          clustered deployment and have come across some situations
          where it experiences a fatal error and moves to a suspended
          state. The most recent error we received was the following
          after container reboot: <o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">GUI Results:<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal"><img style="width:7.7708in;height:6.0729in"
            id="Picture_x0020_1"
            src="cid:part1.stOpolIJ.UT0skMd9@evolveum.com" class=""
            width="746" height="583"><o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">Operation:
com.evolveum.midpoint.repo.common.activity.run.LocalActivityRun.runLocally<o:p></o:p></p>
        <p class="MsoNormal">Status: FATAL_ERROR<o:p></o:p></p>
        <p class="MsoNormal">Message: <o:p></o:p></p>
        <p class="MsoNormal">Bucket
          PCV(4531):[PP({.../common/common-3}sequentialNumber):[PPV(Integer:1)],
PP({.../common/common-3}state):[PPV(WorkBucketStateType:COMPLETE)],
PP({.../common/common-3}content):[PPV(NullWorkBucketContentType:com.evolveum.midpoint.xml.ns._public.common.common_3.NullWorkBucketContentType@6e57bf8b[])]]
          is not ready<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">I have also attached the stack trace from
          the logs. Is it possible to configure Midpoint to continue
          subsequent runs after permanent errors? Additionally, have
          others experienced similar behavior?<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">Thank you,<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
        <p class="MsoNormal">Jeremiah Haywood<o:p></o:p></p>
        <p class="MsoNormal">Lead IAM Administrator<o:p></o:p></p>
        <p class="MsoNormal">Office of Technology Solutions | Illinois
          State University<o:p></o:p></p>
        <p class="MsoNormal">Phone Number (309) 438-3829<o:p></o:p></p>
        <p class="MsoNormal"><o:p> </o:p></p>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
midPoint mailing list
<a class="moz-txt-link-abbreviated" href="mailto:midPoint@lists.evolveum.com">midPoint@lists.evolveum.com</a>
<a class="moz-txt-link-freetext" href="https://lists.evolveum.com/mailman/listinfo/midpoint">https://lists.evolveum.com/mailman/listinfo/midpoint</a>
</pre>
    </blockquote>
  </body>
</html>