[midPoint] Slow Active Directory getoperations for groups
Alcides Moraes
alcides.neto at gmail.com
Fri Jul 12 21:27:13 CEST 2024
Hi Emil
Thank you so much for the suggestion.
I disabled the inbound mapping to check if it was really the culprit.
But it had a small impact, only 5 minutes out of the 3:40 hours
Interesting enough, removing the mapping, removed AD group Get operations from the "Provisioning operations” part of the report.
But you can still see the AD time in the table below it, "Mappings evaluation information”
Still insanely high… This seems more AD related, the average time of get operations (367) seems a little hight to me.
I’m not really knowledgeable about ActiveDirectory, maybe there is some tuning to be made there?
Thanks again
AD
127,170
367
0
75,982
46,749,570
> Em 11 de jul. de 2024, à(s) 05:23, iam-mailing at tk.de escreveu:
>
> Hi Alcides,
>
> this is exactly the same issue we faced in our deployment. MidPoint will fetch every group from the ad which is mentioned in the memberOf attribute to fill the “entitlement” variable in the inbound mapping.
> If those groups have many members it will be especially bad because midpoint will fetch every member attribute value, which can even result in multiple pages for one group. This will have a big memory footpring and also long fetching times and a high CPU usage on the domain controller.
>
> We fixed this issue by removing the inbound mapping in the association and converting it to an inbound mapping for a simple attribute “memberOf”. Depending how your inbound mapping looks like this is possible quite easily.
> You will only have the dn of the group available but doing a raw repository search in midpoint is possible if you want to check the intent etc.
>
> Outbound mappings in metaroles for associations do not cause every group to be loaded but only the group which will be assigned/removed.
>
> If you have any more questions feel free to ask.
>
> Kind Regards
> Emil
>
> Von: Alcides Moraes <alcides.neto at gmail.com>
> Datum: Mittwoch, 10. Juli 2024 um 20:30
> An: midPoint General Discussion <midpoint at lists.evolveum.com>
> Cc: iam-mailing <iam-mailing at tk.de>
> Betreff: [Signatur ungueltig] Re: [midPoint] Slow Active Directory getoperations for groups
>
> EXTERNE E-MAIL - Bitte prüfen Sie die Vertrauenswürdigkeit der Absender-Informationen, bevor Sie Links oder Anhänge öffnen.
> Hi Emil,
>
> Thanks your reaching out.
> Yes, I have an inbound mapping restricted to the reconciliation channel. I’ll try disabling it, see if it changes anything.
> Here’s my association configuration, I have redacted some intents (I have about 20 group intents) and the inbound mapping (it’s a simple assignmentTargetSearch with createOnDemand)
> I have this many intents for groups in order to link many types of automatic groups to OrgTypes, with members based on the kind of employment contract (interns, externals, etc.).
>
>
> <association id="54">
> <ref>ri:group</ref>
> <displayName>AD Group Membership</displayName>
> <tolerant>true</tolerant>
> <intolerantValuePattern>.*(OU=AutomaticGroups|OU=ITProducts).*</intolerantValuePattern>
> <exclusiveStrong>false</exclusiveStrong>
> <fetchStrategy>explicit</fetchStrategy>
> <inbound id="542”>
> </inbound>
> <kind>entitlement</kind>
> <intent>adGroup</intent>
> <intent>intent1</intent>
> <intent>intent2</intent>
> . . .
> <intent>intent20</intent>
> <direction>objectToSubject</direction>
> <associationAttribute>ri:member</associationAttribute>
> <valueAttribute>ri:dn</valueAttribute>
> <shortcutAssociationAttribute>ri:memberOf</shortcutAssociationAttribute>
> <shortcutValueAttribute>ri:dn</shortcutValueAttribute>
> <explicitReferentialIntegrity>false</explicitReferentialIntegrity>
> </association>
>
> Here’s the task xml, with minor redaction.
>
> <task ...>
> <name>Reconcile Employees</name>
> <assignment id="6">
> <targetRef oid="00000000-0000-0000-0000-000000000501" relation="org:default" type="c:ArchetypeType"/>
> <activation>
> <effectiveStatus>enabled</effectiveStatus>
> </activation>
> </assignment>
> <archetypeRef oid="00000000-0000-0000-0000-000000000501" relation="org:default" type="c:ArchetypeType"/>
> <roleMembershipRef oid="00000000-0000-0000-0000-000000000501" relation="org:default" type="c:ArchetypeType"/>
> <ownerRef oid="00000000-0000-0000-0000-000000000002" relation="org:default" type="c:UserType"/>
> <channel>http://midpoint.evolveum.com/xml/ns/public/common/channels-3#reconciliation</channel>
> <unpauseAction>executeImmediately</unpauseAction>
> <category>Reconciliation</category>
> <objectRef oid=“----" relation="org:default" type="c:ResourceType"/>
> <binding>loose</binding>
> <schedule>
> <recurrence>recurring</recurrence>
> <cronLikePattern>1 0 0 ? * MON-FRI</cronLikePattern>
> <misfireAction>executeImmediately</misfireAction>
> </schedule>
> <threadStopAction>restart</threadStopAction>
> <activity>
> <work>
> <reconciliation>
> <resourceObjects>
> <resourceRef oid=“----" relation="org:default" type="c:ResourceType"/>
> <kind>account</kind>
> <intent>default</intent>
> <objectclass>ri:AccountObjectClass</objectclass>
> </resourceObjects>
> </reconciliation>
> </work>
> <distribution>
> <workerThreads>6</workerThreads>
> <subtasks/>
> </distribution>
> <tailoring>
> <change id="1">
> <reference>resourceObjects</reference>
> <distribution>
> <buckets>
> <stringSegmentation>
> <discriminator>attributes/ri:DOCUMENT_ID</discriminator>
> <boundary id="12">
> <position>1</position>
> <characters>0123456789</characters>
> </boundary>
> <boundary id="13">
> <position>2</position>
> <characters>0123456789</characters>
> </boundary>
> <comparisonMethod>prefix</comparisonMethod>
> </stringSegmentation>
> </buckets>
> <workers>
> <workersPerNode id="3">
> <count>1</count>
> </workersPerNode>
> </workers>
> <workerThreads>6</workerThreads>
> </distribution>
> </change>
> <change id="2">
> <reference>remainingShadows</reference>
> <distribution>
> <buckets>
> <oidSegmentation>
> <depth>1</depth>
> </oidSegmentation>
> </buckets>
> <workers>
> <workersPerNode id="4">
> <count>1</count>
> </workersPerNode>
> </workers>
> <workerThreads>6</workerThreads>
> </distribution>
> </change>
> </tailoring>
> </activity>
> </task>
>
>
> Em 10 de jul. de 2024, à(s) 12:18, iam-mailing--- via midPoint <midpoint at lists.evolveum.com> escreveu:
>
> Hi,
>
> can you provide your association configuration? Do you use an inbound mapping inside the association configuration?
> Also the task configuration for the reconciliation could help.
>
> We had an issue regarding get operations in the AD so I would have a look if it’s a similar problem.
>
> Kind Regards,
> Emil Militzer
>
>
>
>
> Am 28.06.24, 19:03 schrieb "midPoint im Auftrag von Alcides Moraes via midPoint" <midpoint-bounces at lists.evolveum.com <mailto:midpoint-bounces at lists.evolveum.com> im Auftrag von midpoint at lists.evolveum.com <mailto:midpoint at lists.evolveum.com>>:
>
>
> EXTERNE E-MAIL - Bitte prüfen Sie die Vertrauenswürdigkeit der Absender-Informationen, bevor Sie Links oder Anhänge öffnen.
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
> https://lists.evolveum.com/mailman/listinfo/midpoint <https://lists.evolveum.com/mailman/listinfo/midpoint>
>
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> https://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20240712/1c79b486/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6278 bytes
Desc: not available
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20240712/1c79b486/attachment-0001.bin>
More information about the midPoint
mailing list