<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Wow, this is incredible hardware setup! Memory (in terms of Xmx for mp java) seems really excessive. The main factor of your deployment seems number of resources, number of users is quite low. <div><br></div><div>Difficult to say if cluster is the right way without knowing more info. I spent lot of time tuning midPoint myself, so here are some tips:<div><div>- As Arnost suggested switching off Consistency check will <b>greatly </b>(e.g. 5x) improve recomputing users with many projections. See <a href="https://jira.evolveum.com/browse/MID-4982">https://jira.evolveum.com/browse/MID-4982</a></div><div>- Switch on <b>midPoint's profiling</b>. Recommended setting on screenshot below, its actually very helpful.</div><div>- Are your perf. problems coming from GUI only? Or it might be delayed e.g. from user recompute? That would imply connector problems (what are e.g. GET stats for accounts?) Provide env. stats tab.</div><div>- Do you have some special problems in Org Tree? This is common and there are some tricks to improve it as well.</div><div>- Common problem is bad DB. Is it really configured right to utilize all memory to cache data?</div><div>- One of biggest causes of perf issues are big XML objects (in terms of KB). Mostly Users. Do you have any user, which when exported to file has >1MB size? You can easily print stats with below task Object size check.</div><div>- Run latest mp 3.9-support... it contains lots of perf and stability fixes.</div><div><br></div><div>Profiler (dont forget enabling it in config.xml):</div><div><div><img src="cid:ii_jsahvhtq0" alt="image.png" width="472" height="372"><br></div></div><div><br></div><div>Print object size stats to midpoint.log:</div><div><div><task xmlns="<a href="http://midpoint.evolveum.com/xml/ns/public/common/common-3">http://midpoint.evolveum.com/xml/ns/public/common/common-3</a>"></div><div> <name>Object size check</name></div><div> <extension xmlns:mext="<a href="http://midpoint.evolveum.com/xml/ns/public/model/extension-3">http://midpoint.evolveum.com/xml/ns/public/model/extension-3</a>"></div><div> <mext:objectQuery></div><div> <!-- put object query here, if needed --></div><div> </mext:objectQuery></div><div> </extension></div><div> <ownerRef oid="00000000-0000-0000-0000-000000000002" type="UserType"/></div><div> <executionStatus>runnable</executionStatus></div><div> <handlerUri><a href="http://midpoint.evolveum.com/xml/ns/public/model/object-integrity-check/handler-3">http://midpoint.evolveum.com/xml/ns/public/model/object-integrity-check/handler-3</a></handlerUri></div><div> <recurrence>single</recurrence></div><div></task> </div></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:13px"><p><strong>Martin Lízner</strong><br><span style="font-size:11px;color:rgb(128,128,128)">chief solution architect</span></p></div><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px">gsm: [+420] 737 745 571<br>e‑mail: <a href="mailto:martin.lizner@ami.cz" target="_blank">martin.lizner@ami.cz</a></p><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px"><strong>AMI Praha a.s.</strong><br>Pláničkova 11, 162 00 Praha 6</p><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px">tel.: [+420] 274 783 239 | web: <a href="http://www.ami.cz" target="_blank">www.ami.cz</a></p><p style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:10px;margin-top:20px"><img src="http://www.ami.cz/images/podpis/ami_logo.gif" alt="AMI Praha a.s." style="border: 0px;"></p><p style="font-family:Arial,sans-serif;font-size:11px;color:rgb(170,170,170)">Textem tohoto e‑mailu podepisující neslibuje uzavřít ani neuzavírá za společnost AMI Praha a.s.<br>jakoukoliv smlouvu. Každá smlouva, pokud bude uzavřena, musí mít výhradně písemnou formu.<br><span style="font-size:6px"> </span><br>Tento e‑mail je určen výhradně pro potřeby jeho adresáta/ů a může obsahovat důvěrné nebo osobní<br>informace. Nejste‑li zamýšleným příjemcem, je zakázáno jakékoliv zveřejňování, zprostředkování<br>nebo jiné použití těchto informací. Pokud jste obdrželi e‑mail neoprávněně, informujte o tom prosím<br>odesílatele a vymažte neprodleně všechny kopie tohoto e‑mailu včetně všech jeho příloh. Nakládáním<br>s neoprávněně získanými informacemi se vystavujete riziku právního postihu.</p></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">po 18. 2. 2019 v 16:21 odesílatel Arnošt Starosta - AMI Praha a.s. <<a href="mailto:arnost.starosta@ami.cz">arnost.starosta@ami.cz</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Wojciech,<div><br></div><div>that's a lot of HW and i have no true solution to this problem.</div><div><br></div><div>but ... there is a tiny magic configuration setting that might help you right now, </div><div><br></div><div>GUI > Internals Configuration > Internal Configuration > check consistency</div><div><br></div><div>i think it's switched off by default in 3.9-support and master, but not before.</div><div><br></div><div>unchecking this may speed things up. beware that it's on again after restart.</div><div><br></div><div>arnost</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">po 18. 2. 2019 v 16:03 odesílatel Wojciech Staszewski <<a href="mailto:wojciech.staszewski@diagnostyka.pl" target="_blank">wojciech.staszewski@diagnostyka.pl</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello!<br>
<br>
I am curious about your opinion and experience about optimal server hardware for midPoint deployment.<br>
<br>
At the moment I have over 7000 users, 190 active resources, 422 Org Units, 2793 roles, 111 reconciliation tasks.<br>
Our midPoint is working slow. GUI became very slow, if user has over 20 projections the user details page opens dramatically slow.<br>
Hardware: 26 CPU cores, 256GB RAM (128G reserved for Tomcat, 50G for DB), 8 x SSD RAID10 for the DB.<br>
Database: MySQL, DB size: 17GB.<br>
<br>
At the moment I moved all server tasks to evening hours and weekends to enable GUI routine works for end users.<br>
But I wonder if this is a good moment to start thinking about second midPoint node to make a cluster...<br>
<br>
And this is still pre-production time (production launch is planned on 07.2019r).<br>
Will the second node be a cure for slow GUI? Or maybe move the connectors to a separate host?<br>
Maybe re-planning server tasks?<br>
<br>
Thanks,<br>
WS<br>
<br>
-- <br>
Wojciech Staszewski<br>
Administrator Systemów Sieciowych<br>
<a href="http://www.diagnostyka.pl" rel="noreferrer" target="_blank">www.diagnostyka.pl</a><br>
Diagnostyka Sp. z o. o.<br>
ul. Prof. M. Życzkowskiego 16, 31-864 Kraków<br>
Numer KRS: 0000381559 (Sąd Rejonowy dla Krakowa-Śródmieścia w Krakowie, XI Wydział Gospodarczy KRS)<br>
NIP: 675-12-65-009; REGON: 356366975<br>
Kapitał zakładowy: 33 756 500 zł.<br>
<br>
Pomyśl o środowisku zanim wydrukujesz ten e-mail.<br>
_______________________________________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mailman/listinfo/midpoint</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_-8257200981995413586gmail_signature"><div dir="ltr"><div><div dir="ltr"><div style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:13px"><p><strong>Arnošt Starosta</strong><br><span style="font-size:11px;color:rgb(128,128,128)">solution architect</span></p></div><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px">gsm: [+420] 603 794 932<br>e‑mail: <a href="mailto:arnost.starosta@ami.cz" target="_blank">arnost.starosta@ami.cz</a></p><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px"><strong>AMI Praha a.s.</strong><br>Pláničkova 11, 162 00 Praha 6</p><p style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:11px">tel.: [+420] 274 783 239 | web: <a href="https://www.ami.cz" target="_blank">www.ami.cz</a></p><p style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:10px;margin-top:20px"><img src="http://www.ami.cz/images/podpis/ami_logo.gif" alt="AMI Praha a.s." style="border: 0px;"></p><p style="font-family:Arial,sans-serif;font-size:11px;color:rgb(170,170,170)">Textem tohoto e‑mailu podepisující neslibuje uzavřít ani neuzavírá za společnost AMI Praha a.s.<br>jakoukoliv smlouvu. Každá smlouva, pokud bude uzavřena, musí mít výhradně písemnou formu.<br><span style="font-size:6px"> </span><br>Tento e‑mail je určen výhradně pro potřeby jeho adresáta/ů a může obsahovat důvěrné nebo osobní<br>informace. Nejste‑li zamýšleným příjemcem, je zakázáno jakékoliv zveřejňování, zprostředkování<br>nebo jiné použití těchto informací. Pokud jste obdrželi e‑mail neoprávněně, informujte o tom prosím<br>odesílatele a vymažte neprodleně všechny kopie tohoto e‑mailu včetně všech jeho příloh. Nakládáním<br>s neoprávněně získanými informacemi se vystavujete riziku právního postihu.</p></div></div></div></div>
_______________________________________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mailman/listinfo/midpoint</a><br>
</blockquote></div></div></div></div>