[midPoint] LDAP role/group inducement
Ivan Noris
ivan.noris at evolveum.com
Tue Jan 28 10:32:24 CET 2020
Hi Jan,
so I pirated my own free time, ran the docker downloaded from your
repository and found out the following:
1. the OpenDJ resource was not working for me from the beginning: only
"dn" attribute in all projections was displayed. My fix was to:
a. remove <capabilities><native> and its content (keeping only
configured capabilities)
b. adding the following to the resource's <connectorConfiguration>:
<icfc:resultsHandlerConfiguration>
<icfc:enableNormalizingResultsHandler>false</icfc:enableNormalizingResultsHandler>
<icfc:enableFilteredResultsHandler>false</icfc:enableFilteredResultsHandler>
<icfc:enableAttributesToGetSearchResultsHandler>false</icfc:enableAttributesToGetSearchResultsHandler>
</icfc:resultsHandlerConfiguration>
This immediately helped me to see all resource account attributes
2. I created user in midPoint with name, givenName, familyName and
fullName and added Basic user role. Provisioning worked. All attributes
in projection are visible.
3. I created a new role "group1" and assigned "LDAP Group Metarole".
Group was created.
4. I edited my user and assigned "group1" role. Account was updated,
association/group member ship added.
5. I edited my user and unassigned "group1" role. Association is no more
displayed for my account.
6. I repeated steps 3-6 with a second role. Both roles were assigned to
the user and then unassigned in two steps. Associations are cleared as
supposed.
So to summarize, would you please retest this with your resource
(OpenDJ) updated by aforementioned information? I have no idea how it
should have worked with your configuration even for the projection
attributes. But it works for me in the scenario from GUI. Import from
CSV also works (even this is not assigning the LDAP groups of course).
P.S. Shadow objects don't store association information by design. It is
always computed by Provisioning.
P.P.S. I can perhaps recommend our trainings, although this concept
(Generic Synchronization) is covered by the Advanced one. It's a good
opportunity to ask questions during the course and see the concepts and
why they work.
Best regards,
Ivan
On 28. 1. 2020 8:40, Jan Lievens wrote:
> So now my conclusion is that the NPE in the admin points to the fact
> that together with name and identifiers there should also be a
> shadowRef saved in the shadow (PostgreSQL account) repo.
> We see that only the name and identifiers are saved in the beginning
> (first import task). I suppose the shadowRef is at that point not
> resolvable (perhaps the entitlements are not imported yet so the
> shadow of an entitlement is not yet available).
> Now we also see that if we perform a null safe check on the NPE site
> our use-case works (uniqueMember is cleared of the LDAP group when we
> remove entitlement from PostgreSQL) but we see that the shadow of the
> entitlement is lingering in the repo (and indeed when we again add the
> entitlement again in PostgreSQL an exception is thrown saying that
> there are conflicting shadows in the import task run).
> So my hunch is that the shadowRef should be saved on the association
> when possible. I see that during the reconciliation the shardowRef is
> correctly added beside the name and identifiers tags (but alas never
> saved to the repo).
> I will pursue this line of thinking. My best guess is to save this
> shadowRef in the association someplace in the ShadowManager.
>
> Op ma 27 jan. 2020 om 15:15 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> So I figured out what the NullPointerException is all about: The
> code at line ShadowAssociationWrapperFactoryImpl.java:193
> "shadowAss.add(associationValue.findReference(ShadowAssociationType.F_SHADOW_REF).getValue().clone());"
> expects to have a shadowRef in the association which it does not
> have (it only has name and identifiers tags, like can be seen from
> my previous post). Now I would say that this missing shadowRef
> needs to be fixed next because when I null check this (and thus
> this line is not performed) the lingering-"uniqueMember" bug in
> LDAP is resolved and the GUI admin does not crash with a NPE but
> the shadow (the shadow of the entitlement that is, for which the
> shadowRef should exist) is not cleaned up and can still be seen in
> the repo. So when I again add the entitlement in PostgreSQL and
> perform an import task on that resource it crashes on the fact
> that it sees conflicting shadows (no surprise there).
>
> Op za 25 jan. 2020 om 15:20 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> So I debugged this problem with the lacking "association" item
> in the account shadow of the PostgreSQL resource. So again a
> tl;dr for the people with no time to follow my lengthy
> explanation: the lingering uniqueMember reference gets cleaned
> up in LDAP when I remove
> line com.evolveum.midpoint.repo.sql.helpers.ObjectRetriever.java:557
> "prismObject.removeContainer(ShadowType.F_ASSOCIATION);"
>
> So what I see is that the shadow that gets written (xml in
> fullObject column) the first time into the repo and it
> contains the "association" item.
>
> <shadow
> xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
> oid="954b8cfe-5c84-4f82-946f-0196dcec92c0" version="0">
> <name>810710333333</name>
> <resourceRef
> oid="dafe0482-faef-47b7-ae08-66b150929bb7"
> relation="org:default" type="c:ResourceType">
> <!-- Scripted SQL -->
> </resourceRef>
>
> <objectClass>ri:AccountObjectClass</objectClass>
>
> <primaryIdentifierValue>3fd83cd4-d1bb-4d7f-9a1a-12a02ed85a95</primaryIdentifierValue>
> <kind>account</kind>
> <exists>true</exists>
> <attributes>
> <icfs:name>810710333333</icfs:name>
>
> <icfs:uid>3fd83cd4-d1bb-4d7f-9a1a-12a02ed85a95</icfs:uid>
> </attributes>
> <association id="1">
> <name>ri:ents</name>
> <identifiers>
>
> <icfs:name>20001-Milieumedewerker-01</icfs:name>
> </identifiers>
> </association>
> <activation/>
> </shadow>
>
> It is afterwards (in a following reconciliation wave) that
> this gets overwritten with a version that has no association.
> Poof gone...
>
> <shadow
> xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
> oid="954b8cfe-5c84-4f82-946f-0196dcec92c0" version="1">
> <name>810710333333</name>
> <resourceRef
> oid="dafe0482-faef-47b7-ae08-66b150929bb7"
> relation="org:default" type="c:ResourceType"/>
>
> <synchronizationTimestamp>2020-01-24T13:30:38.108Z</synchronizationTimestamp>
>
> <fullSynchronizationTimestamp>2020-01-24T13:30:38.108Z</fullSynchronizationTimestamp>
>
> <objectClass>ri:AccountObjectClass</objectClass>
>
> <primaryIdentifierValue>3fd83cd4-d1bb-4d7f-9a1a-12a02ed85a95</primaryIdentifierValue>
> <kind>account</kind>
> <intent>webidm</intent>
> <exists>true</exists>
> <attributes
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xsi:type="c:ShadowAttributesType">
> <icfs:name>810710333333</icfs:name>
>
> <icfs:uid>3fd83cd4-d1bb-4d7f-9a1a-12a02ed85a95</icfs:uid>
> </attributes>
> <activation/>
> </shadow>
>
> Immediately we see that the following modifications made to
> the just created shadow: REPLACE intent, kind,
> synchronizationTimestamp and fullSynchronizationTimestamp. It
> is in this replace that the "association"-part is removed and
> written into the repo. It is on position
> com.evolveum.midpoint.repo.sql.helpers.ObjectRetriever.java:557
> that the association from the shadow gets removed in a very
> explicit fashion with the ominous comment: "we store it
> because provisioning now sends it to repo, but it should be
> transient".
> When we remove this line that removes the association we get a
> fat NPE in the admin tool when opening the user, but our bug
> is fixed 😎.
> But somehow I can't shake the feeling that removing this line
> is hacky (I suppose other code might also depend on this line
> being there) and that the association should get fetched from
> the inbound source (i.e. PostgreSQL) at the right moment. But
> again I'm too novice to really know how to configure that.
> Something of the likes of
> <searchStrategy>onResourceIfNeeded</searchStrategy> maybe?
> If someone could shime in here to actually verify that this is
> a bug then I could send in a pull request with the fix. I will
> look at the NPE in de admin GUI this monday so then I will
> know more on how unorthodox this fix is.
>
>
>
> Op vr 24 jan. 2020 om 10:14 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> Before diving into the more technical description of why I
> came to my conclusion a short tl;dr for the impatient:
> something is probably not correct in my schemaHandling
> part of the Scripted SQL resource with respect to the
> entitlement association. What that is, I have no clue about.
>
> So I did a bit of code spelunking in midPoint to better
> understand the problem. Here is what I came up with:
>
> - To see what happened in the code with respect to the
> LDAP association I put logger
> com.evolveum.midpoint.provisioning into the debug level
> which provides me with some fine log statement when the
> association (user->group: uniqueMember on the group) is
> made in LDAP:
> 2020-01-23 08:58:22,737 [PROVISIONING]
> [midPointScheduler_Worker-2] DEBUG
> (com.evolveum.midpoint.provisioning.impl.ResourceObjectConverter):
> PROVISIONING MODIFY operation on
> resource:d0811790-6420-11e4-86b2-3c9755567874(OpenDJ)
> MODIFY object, object class Technical Role, identified by:
> {
> entryUUID: 88972d7b-6deb-4d94-b343-d039099d23c4
> dn: cn=jirauser,ou=groups,dc=didm,dc=be
> }
> changes: [
> PropertyModificationOperation:
> delta:
> attributes/uniqueMember
> ADD: uid=81071022511,ou=people,dc=didm,dc=be
> matchingRule:
> {http://prism.evolveum.com/xml/ns/public/matching-rule-3}stringIgnoreCase
> <http://prism.evolveum.com/xml/ns/public/matching-rule-3%7DstringIgnoreCase>
> ]
> - So basically I had to find out why the inverse is not
> happening when the association is severed (when we delete
> the entitlement in the PostgreSQL database)
> - The conclusion I get from the code is that this should
> happen in the
> ReconciliationProcessor.reconcileProjectionAssociations
> method. More specifically at the end of that method it is
> decided to tolerate the association or not (tolerant=false
> on the association is a required thing after all 😀).
> - In this decideIfTolerateAssociation method I see this
> beautiful call that is going to solve my world of pain:
> recordAssociationDelta(valueMatcher, accCtx, assocDef,
> ModificationType.DELETE, isCValue.getValue(), null, "it is
> not given by any mapping and the association is not
> tolerant");
> - In my configuration this statement is not evaluated
> because it requires that there is a
> PrismContainerValue<ShadowAssociationType> provided in the
> arguments of this method.
> - And why is there no ShadowAssociationType in this case
> I hear you ask? Well because in
> method com.evolveum.midpoint.prism.impl.PrismContainerValueImpl#findItemByQName
> it cannot find an "association" item in the shadow of the
> PostgreSQL account (see screenshot in attachment).
> - And indeed when I look at that specific shadow's
> repository object xml, I see nothing that looks like an
> association.
> - And why is there no "association" item on the
> PostgreSQL account shadow? That I do not know. Maybe
> somebody can put me out of my misery?
> - So basically I assume something is fishy about my
> schemaHandling part of the Scripted SQL resource with
> respect to the entitlement association.
>
> Sorry if this is somewhat too technical, I realise there
> is a dev mailing list but I wanted to add it here so that
> people find the solution and problem in the same space.
>
> Op di 21 jan. 2020 om 15:50 schreef Jan Lievens
> <jan.lievens at biggerfish.be
> <mailto:jan.lievens at biggerfish.be>>:
>
> Hi,
>
> May I also add that I tried this setup with midPoint
> versions 4.0.1, 4.0 and 3.9 all with the same result
> (lingering uniqueMember attributes in LDAP).
>
>
> Op di 21 jan. 2020 om 14:47 schreef Jan Lievens
> <jan.lievens at biggerfish.be
> <mailto:jan.lievens at biggerfish.be>>:
>
> Hi all,
>
> I have reduced the complexity of my use case and
> made it easy to reproduce. Starting from the unix
> story
> (https://wiki.evolveum.com/display/midPoint/Unix+Story+Test)
> Now I have the following situation:
> - Sync accounts from CSV to OpenDJ (works great)
> - I have one meta-role that is assigned to a test
> role which results in an LDAP group (works great)
> - Then I assign this test role to a user which
> results in a uniqueMember attribute on the LDAP
> group (nice)
> - Then I unassign the test role from said user
> and the uniqueMember attribute on the LDAP group
> is not getting removed (😭)
> - The uniqueMember attribute cannot be removed
> even after running reconcile/recompute on accounts
> and entitlements
>
> More succinct description to reproduce:
> - git checkout
> https://github.com/jalieven/midpoint-docker
> - cd midpoint-docker
> - git checkout simple-ldap-group
> - 'make restart' (starts docker with
> postgres/opendj/midpoint)
> - go to admin GUI http://localhost:8080/midpoint
> - login 'administrator' pwd '5ecr3t'
> - create a new role 'TestGroup' with an assignment
> to role 'LDAP Group Metarole'
> [- in OpenDJ: be.didm.group.TestGroup is created]
> => OK (checking can be done with 'make check_ldap'
> in root project)
> - in admin GUI: add 'TestGroup' to 'user01'
> assignments
> [- in OpenDJ: the be.didm.group.TestGroup gets the
> attribute uniqueMember set to
> 'uid=user01,ou=people,dc=didm,dc=be'] => OK
> - in admin GUI: remove 'TestGroup' from 'user01'
> assignments (minus button and SAVE)
> [- in OpenDJ: nothing happens: uniqueMember is not
> removed from be.didm.group.TestGroup] => Not OK!
> - to verify this stale uniqueMember attribute: in
> root of project do 'make check_ldap'
>
> Config files can be found in
> demo/simple/midpoint_server/container_files/mp-home/post-initial-objects
>
> As you can see reproducing this bug/miss-config
> takes 5min of work. If this is a
> miss-configuration you might want to check because
> the same is happening in the unix story test. If
> this is a bug in midPoint all the more so to look
> at it. It is hard for me as a novice to say which
> one it is.
>
> It seems to me that this is a common use-case. I
> have come across this issue a few times in this
> mailing list, none of which provide a sufficient
> solution:
> http://lists.evolveum.com/pipermail/midpoint/2016-April/001720.html
> (the 'it-works-on-my-machine'-stance is kind of
> lame in this case and it doesn't help that the
> referred zip files does not resolve)
> http://lists.evolveum.com/pipermail/midpoint/2016-November/002807.html
> (a lot of tolerant/strength stuff here)
> http://lists.evolveum.com/pipermail/midpoint/2016-November/002843.html
> (suggests association shortcuts and multivalued
> schema fix in scripted sql connector, the latter
> of which does not apply in my described use-case)
>
> The midPoint Confluence suggests also to set
> various packages to trace log level to debug what
> is going on but I must say that in so much logging
> I have no idea what to look for.
> When I set the projector/clockwork to debug all I
> see is that the remove of the association is not
> happening. More specifically this package on
> trace org.identityconnectors.framework.spi.operations
> points to the fact that the delete is not
> happening in the connector.
>
> May I also add that the
> page https://wiki.evolveum.com/display/midPoint/Initial+Logging+Setup+HOWTO is
> not up to date. Setting the
> skipRepositoryLoggingSettings does not work. So
> there is currently no way to add specific loggers
> at start up through the logback(-extra).xml. It
> always gets overwritten by what is inside the
> repository.
> It appears that the
> skipRepositoryLoggingSettings was removed from the
> code some time ago.
>
> Kind Regards,
> Jan Lievens
>
> Op wo 15 jan. 2020 om 16:19 schreef Jan Lievens
> <jan.lievens at biggerfish.be
> <mailto:jan.lievens at biggerfish.be>>:
>
> Hi,
>
> I have a question related to inducement
> construction on a role toward an LDAP resource.
> I have the following situation in midPoint
> (see the attachment for all the xml files)
> - Accounts and Entitlements (intent:
> privileges) are imported from a ScriptedSQL
> connector.
> - These imported entitlements have multiple
> privileges which are created as roles with an
> "assignmentTargetSearch"
> (post-initial-objects/210-entitlement-object-template.xml)
> - I also have these privilege/roles defined
> with an assignment to (multiple) fixed
> technical roles (kind:entitlement/intent:group).
> - Lastly I would like for these technical
> roles (groups) and the associated accounts to
> get synced to LDAP in an objectToSubject
> fashion. I do this with inducements and
> construction tags in
> post-initial-objects/100-profiles-webidm.xml.
> - The result I get in LDAP is that accounts
> are synced correctly but the group names are
> not what I expect (I expect technical role
> names eg. JiraUser) but get the names of the
> Entitlements (intent:privileges) defined in
> the DB.
> - Additionally the associations are not synced
> (no uniqueMember refs are synced on the LDAP
> groups).
>
> Surely I am missing something here. I think
> this use case is quiet standard (a hierarchy
> of roles and only the leafs get synced to LDAP).
> I have experimented with the order of the
> inducement but these came out even more
> negative (no accounts or groups were created).
> I also have experimented with the tolerant
> setting on the association (since I find a lot
> of answers in this mailing list suggesting
> this) but to no avail.
> It is quiet frustrating to be so close to
> having this use-case implemented DB->mP->LDAP
> but failing in the sync to LDAP.
>
> Kind regards,
> --
> Jan Lievens
> IT Consultant
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
--
Ivan Noris
Senior Identity Engineer
evolveum.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20200128/0b3c6eca/attachment.htm>
More information about the midPoint
mailing list