[midPoint] Speeding up scriptedsql

[Wojciech Staszewski]

Hello Jason!

This is also my case.
I was trying to debug operations executed by reconciliation tasks, so I ran "simpleResourceObjectNotifier" with output to a log file
and noticed much more updates/modifications on the remote databases than I expected.

The first cause was simple in my case: matching rules in the attribute outbound mapping, eg. "givenName", mapped to the uppercase
on the resource, but matching rule id midPoint was "default", so midPoint was updating these attributes everytime.
There was a lot attributes like that with incorrect matching rules, after corrections reconciliation tasks runs a little faster.

Except that, I noticed that non-tolerant entitlement associations works strange.
I mean that for every user every time operation "DELETE" is executed for not assigned entitlements (eg. group membership),
even if the user is NOT a group member on the target system.

Log:
The account has been successfully modified on the resource. Modified attributes are:
...
   - Associations:
      - DELETE:
         - name: zabbixGroups
         - identifiers:
            - ConnId Name: Zabbix administrators
         - shadowRef: Zabbix administrators (shadow) on ZABBIX  [default]
      - DELETE:
         - name: zabbixGroups
         - identifiers:
            - ConnId Name: GROUP1
         - shadowRef: GROUP1 (shadow) on ZABBIX  [default]
      - DELETE:
         - name: zabbixGroups
         - identifiers:
            - ConnId Name: GROUP2
         - shadowRef: GROUP2 (shadow) on ZABBIX  [default]
      - DELETE:
         - name: zabbixGroups
         - identifiers:
            - ConnId Name: GROUP3
         - shadowRef: GROUP3 (shadow) on ZABBIX  [default]
      - DELETE:
         - name: zabbixGroups
         - identifiers:
            - ConnId Name: GROUP4
         - shadowRef: GROUP4 (shadow) on ZABBIX  [default]

...and so on,for every not assigned group and for every account. I guess it slows down the reconciliation.
But I don't know if I can do anything about that...


Regards,
WS


W dniu 03.01.2019 o 20:50, Jason Everling pisze:
> It may just be me but it seems like in 3.7 the ScriptedSQL connector is slower than previous versions. Script reload is also set to false.
> 
> I am running a recon now and it is processing 1000 per 10min, previously I remember it only taking about 45min to process this resource is 3.4. This resource has 15k+ accounts so I am looking at about 2 1/2 -3 hours. This is a new server when I migrated it to 3.7 , running stand-alone, not container on Ubnt 18.04, TC 8.x, MySQL 5.7, VM resides on ssd clustered datastore, 12Gb ram. I am going to see what I can do on the mysql side in the meantime.
> 
> Any have any pointers to speed it up or maybe you have done something to improve it?
> 
> Thanks!
> 
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
> 

-- 
Wojciech Staszewski
Administrator Systemów Sieciowych
www.diagnostyka.pl
Diagnostyka Sp. z o. o.
ul. Prof. M. Życzkowskiego 16, 31-864 Kraków
Numer KRS: 0000381559 (Sąd Rejonowy dla Krakowa-Śródmieścia w Krakowie, XI Wydział Gospodarczy KRS)
NIP: 675-12-65-009; REGON: 356366975
Kapitał zakładowy: 33 756 500 zł.

Pomyśl o środowisku zanim wydrukujesz ten e-mail.