[midPoint] LDAP role/group inducement
Pavol Mederly
mederly at evolveum.com
Tue Jan 28 10:06:01 CET 2020
Hello Jan,
you attempted a really deep dive into midPoint code. You have my
respect. Although I am not able to give you a definite answer because my
time is very constrained, let me say a few notes:
> - The conclusion I get from the code is that this should happen in
> the ReconciliationProcessor.reconcileProjectionAssociations method.
> More specifically at the end of that method it is decided to tolerate
> the association or not (tolerant=false on the association is a
> required thing after all 😀).
> - In this decideIfTolerateAssociation method I see this beautiful
> call that is going to solve my world of pain:
> recordAssociationDelta(valueMatcher, accCtx, assocDef,
> ModificationType.DELETE, isCValue.getValue(), null, "it is not given
> by any mapping and the association is not tolerant");
MidPoint has two ways how it decides whether to make a change (in your
case, deleting an association or not):
1. It computes the change (we call it "delta") directly - e.g. from
some mapping.
2. Or it computes a required object state and - by comparing it with
the current state of the object - it computes the necessary changes
("deltas").
The latter case requires the setting of tolerant=false. The former one
should apply regardless of the tolerant flag setting.
This is the general approach as I see it. I think that in your case both
ways should work - midPoint should know that it has to create the delta
("1") and, if for some reason it knows that not, it should compute that
the association value is to be deleted ("2").
Why this does not work ... this requires more deep analysis.
But let me continue with the notes:
> - And why is there no ShadowAssociationType in this case I hear you
> ask? Well because in
> method com.evolveum.midpoint.prism.impl.PrismContainerValueImpl#findItemByQName
> it cannot find an "association" item in the shadow of the PostgreSQL
> account (see screenshot in attachment).
> - And indeed when I look at that specific shadow's repository object
> xml, I see nothing that looks like an association.
> - And why is there no "association" item on the PostgreSQL account
> shadow? That I do not know. Maybe somebody can put me out of my misery?
This is quite simple. :) Unless so called attribute caching (an
experimental feature) is enabled, midPoint does not store attributes
(and associations, which are basically the same as attributes) in the
repository. The exception are identifiers. And I think that associations
are not stored in repo even with attribute caching enabled.
But your use case should work even without the caching.
From your second mail:
> So I debugged this problem with the lacking "association" item in the
> account shadow of the PostgreSQL resource. So again a tl;dr for the
> people with no time to follow my lengthy explanation: the lingering
> uniqueMember reference gets cleaned up in LDAP when I remove
> line com.evolveum.midpoint.repo.sql.helpers.ObjectRetriever.java:557
> "prismObject.removeContainer(ShadowType.F_ASSOCIATION);"
This is an interesting attempt, but definitely not the way to go (stems
from what I wrote above).
> But somehow I can't shake the feeling that removing this line is hacky
> (I suppose other code might also depend on this line being there) and
> that the association should get fetched from the inbound source (i.e.
> PostgreSQL) at the right moment.
Exactly!
> But again I'm too novice to really know how to configure that.
> Something of the likes of
> <searchStrategy>onResourceIfNeeded</searchStrategy> maybe?
I am not sure without looking deeper.
> So my hunch is that the shadowRef should be saved on the association
> when possible. I see that during the reconciliation the shardowRef is
> correctly added beside the name and identifiers tags (but alas never
> saved to the repo).
> I will pursue this line of thinking. My best guess is to save this
> shadowRef in the association someplace in the ShadowManager.
Again, I would strongly advise against going this way. I am writing this
to spare your time and effort.
To wrap all this up: Although I really sympathize with your efforts and
devotion, I cannot help you much besides this. Generally, I think that
midPoint should know it has to delete the association when the
assignment goes away (case "1" in my list at the beginning). And if not,
it should somehow determine this from the current state of the object.
The information about the association should not definitely come from
the repository. It must come from the resource (i.e. from provisioning
module).
I would strongly recommend you to obtain professional services from
Evolveum. E.g. now I have to finish writing this and devote my time to
clients that have bought such services, as they have equally urgent
issues... :-)
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 24/01/2020 10:14, Jan Lievens wrote:
> Before diving into the more technical description of why I came to my
> conclusion a short tl;dr for the impatient: something is probably not
> correct in my schemaHandling part of the Scripted SQL resource with
> respect to the entitlement association. What that is, I have no clue
> about.
>
> So I did a bit of code spelunking in midPoint to better understand the
> problem. Here is what I came up with:
>
> - To see what happened in the code with respect to the LDAP
> association I put logger com.evolveum.midpoint.provisioning into the
> debug level which provides me with some fine log statement when the
> association (user->group: uniqueMember on the group) is made in LDAP:
> 2020-01-23 08:58:22,737 [PROVISIONING] [midPointScheduler_Worker-2]
> DEBUG
> (com.evolveum.midpoint.provisioning.impl.ResourceObjectConverter):
> PROVISIONING MODIFY operation on
> resource:d0811790-6420-11e4-86b2-3c9755567874(OpenDJ)
> MODIFY object, object class Technical Role, identified by:
> {
> entryUUID: 88972d7b-6deb-4d94-b343-d039099d23c4
> dn: cn=jirauser,ou=groups,dc=didm,dc=be
> }
> changes: [
> PropertyModificationOperation:
> delta:
> attributes/uniqueMember
> ADD: uid=81071022511,ou=people,dc=didm,dc=be
> matchingRule:
> {http://prism.evolveum.com/xml/ns/public/matching-rule-3}stringIgnoreCase
> <http://prism.evolveum.com/xml/ns/public/matching-rule-3%7DstringIgnoreCase>
> ]
> - So basically I had to find out why the inverse is not happening
> when the association is severed (when we delete the entitlement in the
> PostgreSQL database)
> - The conclusion I get from the code is that this should happen in
> the ReconciliationProcessor.reconcileProjectionAssociations method.
> More specifically at the end of that method it is decided to tolerate
> the association or not (tolerant=false on the association is a
> required thing after all 😀).
> - In this decideIfTolerateAssociation method I see this beautiful
> call that is going to solve my world of pain:
> recordAssociationDelta(valueMatcher, accCtx, assocDef,
> ModificationType.DELETE, isCValue.getValue(), null, "it is not given
> by any mapping and the association is not tolerant");
> - In my configuration this statement is not evaluated because it
> requires that there is a PrismContainerValue<ShadowAssociationType>
> provided in the arguments of this method.
> - And why is there no ShadowAssociationType in this case I hear you
> ask? Well because in
> method com.evolveum.midpoint.prism.impl.PrismContainerValueImpl#findItemByQName
> it cannot find an "association" item in the shadow of the PostgreSQL
> account (see screenshot in attachment).
> - And indeed when I look at that specific shadow's repository object
> xml, I see nothing that looks like an association.
> - And why is there no "association" item on the PostgreSQL account
> shadow? That I do not know. Maybe somebody can put me out of my misery?
> - So basically I assume something is fishy about my schemaHandling
> part of the Scripted SQL resource with respect to the entitlement
> association.
>
> Sorry if this is somewhat too technical, I realise there is a dev
> mailing list but I wanted to add it here so that people find the
> solution and problem in the same space.
>
> Op di 21 jan. 2020 om 15:50 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> Hi,
>
> May I also add that I tried this setup with midPoint
> versions 4.0.1, 4.0 and 3.9 all with the same result
> (lingering uniqueMember attributes in LDAP).
>
>
> Op di 21 jan. 2020 om 14:47 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> Hi all,
>
> I have reduced the complexity of my use case and made it easy
> to reproduce. Starting from the unix story
> (https://wiki.evolveum.com/display/midPoint/Unix+Story+Test)
> Now I have the following situation:
> - Sync accounts from CSV to OpenDJ (works great)
> - I have one meta-role that is assigned to a test role which
> results in an LDAP group (works great)
> - Then I assign this test role to a user which results in a
> uniqueMember attribute on the LDAP group (nice)
> - Then I unassign the test role from said user and the
> uniqueMember attribute on the LDAP group is not getting
> removed (😭)
> - The uniqueMember attribute cannot be removed even after
> running reconcile/recompute on accounts and entitlements
>
> More succinct description to reproduce:
> - git checkout https://github.com/jalieven/midpoint-docker
> - cd midpoint-docker
> - git checkout simple-ldap-group
> - 'make restart' (starts docker with postgres/opendj/midpoint)
> - go to admin GUI http://localhost:8080/midpoint
> - login 'administrator' pwd '5ecr3t'
> - create a new role 'TestGroup' with an assignment to role
> 'LDAP Group Metarole'
> [- in OpenDJ: be.didm.group.TestGroup is created] => OK
> (checking can be done with 'make check_ldap' in root project)
> - in admin GUI: add 'TestGroup' to 'user01' assignments
> [- in OpenDJ: the be.didm.group.TestGroup gets the attribute
> uniqueMember set to 'uid=user01,ou=people,dc=didm,dc=be'] => OK
> - in admin GUI: remove 'TestGroup' from 'user01' assignments
> (minus button and SAVE)
> [- in OpenDJ: nothing happens: uniqueMember is not removed
> from be.didm.group.TestGroup] => Not OK!
> - to verify this stale uniqueMember attribute: in root of
> project do 'make check_ldap'
>
> Config files can be found in
> demo/simple/midpoint_server/container_files/mp-home/post-initial-objects
>
> As you can see reproducing this bug/miss-config takes 5min of
> work. If this is a miss-configuration you might want to check
> because the same is happening in the unix story test. If this
> is a bug in midPoint all the more so to look at it. It is hard
> for me as a novice to say which one it is.
>
> It seems to me that this is a common use-case. I have come
> across this issue a few times in this mailing list, none of
> which provide a sufficient solution:
> http://lists.evolveum.com/pipermail/midpoint/2016-April/001720.html
> (the 'it-works-on-my-machine'-stance is kind of lame in this
> case and it doesn't help that the referred zip files does not
> resolve)
> http://lists.evolveum.com/pipermail/midpoint/2016-November/002807.html
> (a lot of tolerant/strength stuff here)
> http://lists.evolveum.com/pipermail/midpoint/2016-November/002843.html
> (suggests association shortcuts and multivalued schema fix in
> scripted sql connector, the latter of which does not apply in
> my described use-case)
>
> The midPoint Confluence suggests also to set various packages
> to trace log level to debug what is going on but I must say
> that in so much logging I have no idea what to look for.
> When I set the projector/clockwork to debug all I see is that
> the remove of the association is not happening. More
> specifically this package on
> trace org.identityconnectors.framework.spi.operations points
> to the fact that the delete is not happening in the connector.
>
> May I also add that the page
> https://wiki.evolveum.com/display/midPoint/Initial+Logging+Setup+HOWTO is
> not up to date. Setting the skipRepositoryLoggingSettings does
> not work. So there is currently no way to add specific loggers
> at start up through the logback(-extra).xml. It always gets
> overwritten by what is inside the repository.
> It appears that the skipRepositoryLoggingSettings was removed
> from the code some time ago.
>
> Kind Regards,
> Jan Lievens
>
> Op wo 15 jan. 2020 om 16:19 schreef Jan Lievens
> <jan.lievens at biggerfish.be <mailto:jan.lievens at biggerfish.be>>:
>
> Hi,
>
> I have a question related to inducement construction on a
> role toward an LDAP resource.
> I have the following situation in midPoint (see the
> attachment for all the xml files)
> - Accounts and Entitlements (intent: privileges) are
> imported from a ScriptedSQL connector.
> - These imported entitlements have multiple privileges
> which are created as roles with an
> "assignmentTargetSearch"
> (post-initial-objects/210-entitlement-object-template.xml)
> - I also have these privilege/roles defined with an
> assignment to (multiple) fixed technical roles
> (kind:entitlement/intent:group).
> - Lastly I would like for these technical roles (groups)
> and the associated accounts to get synced to LDAP in an
> objectToSubject fashion. I do this with inducements and
> construction tags in
> post-initial-objects/100-profiles-webidm.xml.
> - The result I get in LDAP is that accounts are synced
> correctly but the group names are not what I expect (I
> expect technical role names eg. JiraUser) but get the
> names of the Entitlements (intent:privileges) defined in
> the DB.
> - Additionally the associations are not synced (no
> uniqueMember refs are synced on the LDAP groups).
>
> Surely I am missing something here. I think this use case
> is quiet standard (a hierarchy of roles and only the leafs
> get synced to LDAP).
> I have experimented with the order of the inducement but
> these came out even more negative (no accounts or groups
> were created).
> I also have experimented with the tolerant setting on the
> association (since I find a lot of answers in this mailing
> list suggesting this) but to no avail.
> It is quiet frustrating to be so close to having this
> use-case implemented DB->mP->LDAP but failing in the sync
> to LDAP.
>
> Kind regards,
> --
> Jan Lievens
> IT Consultant
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
>
>
> --
> Jan Lievens
> IT Consultant
> Bigger Fish
> Hoogstraat 35
> 9450 Haaltert
> Belgium
> VAT: BE 0734.993.447
> Mob: +32 494 84 66 97
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20200128/b33031ea/attachment.htm>
More information about the midPoint
mailing list