[midPoint] - SciptedSQL connector misshandling inherited roles deletion
Radovan Semancik
radovan.semancik at evolveum.com
Mon Nov 21 15:09:26 CET 2016
Hi,
That's strange. The ScriptedSQL is somehow different. But it should not
be THAT different. Please once again look at the ConnId operation trace.
That's the most reliable source of debugging information in this case.
But based on your information I would guess that it really is midPoint
issue. If the connector is not getting the remove operation than that
means that midpoint is not sending it. If you are sure that the "model"
configuration is correct (e.g. tolerant setting, mapping strength, etc.)
then it is most likely that the provisioning part is filtering out the
operation. There may be several reasons for that. E.g. if the read
operation does not work properly midPoint may think that the value is
not there and therefore there is no need to remove it. Some resources
(namely LDAP) are quite touchy and they respond with an error if we try
to remove a value that is not there. Therefore we are often filtering
the deltas before sending them to connector. Or there may be several
other cases. Generally setting provisioning logging to DEBUG (and in
extreme cases to TRACE) should give you more information what it really
happening. To be more specific try setting:
com.evolveum.midpoint.provisioning: DEBUG
--
Radovan Semancik
Software Architect
evolveum.com
On 11/21/2016 01:38 PM, Nicolas Rossi wrote:
> Hi Radovan. It worked for ActiveDirectory connector but didn't for the
> ScriptedSQL. We have added an echo at the beginning of each groovy
> scripts printing the action and the object class received and It only
> receives an ADD_ATTRIBUTE_VALUE of the value that the user already
> had. There is no REMOVE_ATTRIBUTE_VALUE so I guess the issue is on the
> connector this time. I have an isolated set of resource, meta role and
> role to reproduce the issue. You can download it from here
> <https://dl.dropboxusercontent.com/u/9319179/ScriptedSQLTest.zip> if
> you want. The main difference with the Active Directory resource is in
> the association: subjectToObject vs objectToSubject. Do you think the
> problem could be there ? I'll try it.
>
> I guess it would be helpful add this info of tolerant attribute on
> this page: https://wiki.evolveum.com/display/midPoint/Entitlements.
>
> Best regards,
>
>
>
> Ing Nicolás Rossi
> Identicum S.A.
> Jorge Newbery 3226
> Tel: +54 (11) 4552-3050
> www.identicum.com <http://www.identicum.com>
>
> On Mon, Nov 21, 2016 at 7:15 AM, Radovan Semancik
> <radovan.semancik at evolveum.com <mailto:radovan.semancik at evolveum.com>>
> wrote:
>
> Hi,
>
> I have created the test. And surprisingly it is passing. This is
> 3.5-SNAPSHOT, but it is very likely that it works also in earlier
> versions. Therefore it looks it is really a misconfiguration. The
> cause is really most likely the tolerant flag. The tolerant flag
> is critical in this situation.
>
> For "normal" midPoint operations when you are adding or removing
> an assignment from user we have the delta. We know what has
> changed. Therefore we remove the group even if it is set to
> tolerant. Because we know that the last assignment that "induced"
> that group was just removed.
>
> But if you change the meta role (first operation) and then
> reconcile the user (second operation) then there is no delta.
> These operations are independent. MidPoint does not know what has
> changed in the meta-role. Therefore it cannot use the same logic
> to remove the user from the group. Slightly different logic is
> used in reconciliation. Logic that is not based on deltas (because
> there are none). And in this case the tolerant flag is important.
> If it is set to true then midPoint will NOT remove the extra
> values from the attribute or the extra entitlements. If it is set
> to false then midPoint will remove them.
>
> Please make sure you have the association set to non-tolerant in
> the schemaHandling section of the resource definition. Like this:
>
> <resource>
> <schemaHandling>
> ....
> <association>
> <ref>ri:group</ref>
> <tolerant>false</tolerant>
> ....
> </association>
> ...
>
> This has to be defined in the schemaHandling and NOT in the role
> or meta-role. The tolerance is the property of the
> attribute/association itself and NOT a property of any mapping,
> role or value. The values that are not given by any role and just
> that - not given by any role. So we do not have any role
> definition that we can apply to them. Therefore the setting
> whether the attribute/association is tolerant or not is somehow
> "global". Therefore it needs to be defined in schemaHandling.
>
> Also, please make sure that your mappings are strong, e.g.
>
> <role>
> ...
> <inducement>
> <construction>
> ...
> <association>
> <ref>ri:group</ref>
> <outbound>
> <strength>strong</strength>
> ...
> </outbound>
> </association>
> </construction>
> </inducement>
>
> Mappings that are of "normal" strength are inherently delta-based
> and they are usually NOT processed by the reconciliation at all.
> For "normal" mappings the last change wins. But in reconciliation
> we have no idea what change was the last one - whether the one on
> the resource or the one in midPoint. Therefore we prefer the
> conservative approach and we rather maintain status quo.
>
> --
> Radovan Semancik
> Software Architect
> evolveum.com <http://evolveum.com>
>
>
>
> On 11/20/2016 04:44 PM, Radovan Semancik wrote:
>> Hi,
>>
>> There is no update operation in the log. Therefor midPoint is not
>> invoking the group membership removal at all. I'm not sure what
>> exactly happens here. Your configuration seems to be OK at the
>> first sight and I would tell that your setup should work.
>> Therefore this may be a midPoint bug. I will try to reproduce
>> similar situation in midPoint tests. I'll let you know how it went.
>>
>> --
>> Radovan Semancik
>> Software Architect
>> evolveum.com <http://evolveum.com>
>>
>>
>> On 11/16/2016 01:49 PM, Nicolas Rossi wrote:
>>> Hi Radovan, here is the log of the operation as you suggested.
>>> At the beginning the "AD-SuperRole" had 3 inducements to roles
>>> (with MetaRole): AD-Group3, AD-Group4 and AD-Group5. The user
>>> ltroncoso has this AD-SuperRole and he has 3 groups assigned on
>>> AD. Then we removed the AD-Group3 from the AD-SuperRole and
>>> reconciled the User from the Admin-GUI but he still has the
>>> groupMembership on AD to Group3.
>>>
>>> Attached is the AD-SuperRole, the AD_GROUP-ENTITLEMENT
>>> (MetaRole), the AD-Group3 and the User's xml.
>>>
>>> Do you need any additional information ?
>>>
>>> Best regards,
>>>
>>>
>>>
>>> Ing Nicolás Rossi
>>> Identicum S.A.
>>> Jorge Newbery 3226
>>> Tel: +54 (11) 4552-3050
>>> www.identicum.com <http://www.identicum.com>
>>>
>>> On Wed, Nov 16, 2016 at 7:35 AM, Radovan Semancik
>>> <radovan.semancik at evolveum.com
>>> <mailto:radovan.semancik at evolveum.com>> wrote:
>>>
>>> Hi,
>>>
>>> This is a really interesting case. Initially I was
>>> suspecting a problem in the scripted SQL connector. We do
>>> not use these scripted connectors much as the configurations
>>> are very difficult to maintain. With the many possible uses
>>> of the scripted connectors these are likely to be a cause of
>>> problems. But if that issue affects AD/LDAP connector then
>>> it may indicate midPoint issue.
>>>
>>> Just to provide complete information: some time ago I have
>>> written a guide how to systematically diagnose issues like
>>> these. Here it is:
>>>
>>> https://wiki.evolveum.com/display/midPoint/Troubleshooting+Mappings
>>> <https://wiki.evolveum.com/display/midPoint/Troubleshooting+Mappings>
>>>
>>> However, to cut it short, first interesting thing would be
>>> to see what operation midPoint sends to the connector.
>>> Please enable the ConnId operation logging by setting
>>> following logger:
>>>
>>> org.identityconnectors.framework: TRACE
>>>
>>> Then re-try the operation (example of the message that you
>>> are looking for is in the guide). This should give us
>>> information whether the problem is that midPoint is sending
>>> wrong operation to connector or whether the connector is
>>> doing wrong thing. Then we will know where to focus further
>>> search for the problem.
>>>
>>> --
>>> Radovan Semancik
>>> Software Architect
>>> evolveum.com <http://evolveum.com>
>>>
>>> On 11/14/2016 04:11 PM, Nicolas Rossi wrote:
>>>> Hi guys, I'd like to add more information to this issue. We
>>>> are also facing the same issue with the AD-Ldap driver when
>>>> a Role loses an inducement to another Role. After reconcile
>>>> the user the group membership is not removed.
>>>> I've added the <tolerant>false</tolerant> flag to the Meta
>>>> Role as Ivan said but there was no change.
>>>> Regards,
>>>> Ing Nicolás Rossi Identicum S.A. Jorge Newbery 3226 Tel:
>>>> +54 (11) 4552-3050 www.identicum.com
>>>> <http://www.identicum.com>
>>>> On Fri, Nov 11, 2016 at 5:09 PM, Nicolas Rossi
>>>> <nrossi at identicum.com <mailto:nrossi at identicum.com>> wrote:
>>>>
>>>> Hi Ivan / Radovan
>>>> I guess there is a problem in the ScriptedSQL driver
>>>> (not the scripts) when an inducement is unassigned from
>>>> a Role because we are facing the same issue in two
>>>> different situations:
>>>>
>>>> 1. When a technical role with inducements to
>>>> entitlements is unassigned from user the script
>>>> does not receive the action REMOVE_ATTRIBUTE_VALUE
>>>> 2. When a technical role (with MetaRole) is unassigned
>>>> from a functional role assigned to user when
>>>> recompute the user the script does not receive the
>>>> action REMOVE_ATTRIBUTE_VALUE
>>>>
>>>> Both situations are working when you assign the
>>>> inducements. I have an isolated example here
>>>> <https://dl.dropboxusercontent.com/u/9319179/ScriptedSQLTest.zip>.
>>>> Best regards,
>>>> Ing Nicolás RossiIdenticum S.A. Jorge Newbery 3226 Tel:
>>>> +54 (11) 4552-3050 www.identicum.com
>>>> <http://www.identicum.com>
>>>> On Fri, Nov 11, 2016 at 11:00 AM, Rodrigo Yanis
>>>> <ryanis at identicum.com <mailto:ryanis at identicum.com>>
>>>> wrote:
>>>>
>>>> Ivan,
>>>> Just tried configuring the meta-role just like
>>>> that. Unfortunately no progress. We'll continue
>>>> analyzing this and keep you posted if we find anything.
>>>> Thanks a lot.
>>>> Regards,
>>>> *Rodrigo Yanis.* Identicum S.A. Jorge Newbery 3226
>>>> Tel: +54 (11) 4824-9971ryanis at identicum.com
>>>> <mailto:ryanis at identicum.com> www.identicum.com
>>>> <http://www.identicum.com/>
>>>> 2016-11-11 2:46 GMT-05:00 Ivan Noris
>>>> <ivan.noris at evolveum.com
>>>> <mailto:ivan.noris at evolveum.com>>:
>>>>
>>>> Hi Rodrigo,
>>>>
>>>> I meant this:
>>>>
>>>> ...
>>>>
>>>> <inducement> <construction>
>>>> <resourceRef
>>>> oid="00000000-dc00-dc00-0001-000000000021"
>>>> type="c:ResourceType"/><!-- Portal intranet
>>>> --><kind>account</kind>
>>>> <intent>default</intent> <association>
>>>> <ref>ri:wsEntitlements</ref>
>>>> <outbound>
>>>> *<strength>strong</strength>***
>>>> <source> ...
>>>> </source>
>>>> <expression> ...
>>>>
>>>> But I think your problem should be resolved by
>>>> tolerance (set to false) - strong mapping
>>>> strength is to allow midPoint to enforce the
>>>> group assignment when reconciling. Still I
>>>> don't have any other idea. I hope that's not a
>>>> problem with that specific connector because I
>>>> wouldn't be able help with Java.
>>>>
>>>> Best regards,
>>>>
>>>> IVan
>>>>
>>>> On 11/10/2016 09:36 PM, Rodrigo Yanis wrote:
>>>>> Ivan,
>>>>> I've compared your XML to my association
>>>>> attribute's deffinition on the resource and it
>>>>> looks the same. Can you please explain further
>>>>> what you mean by defining strength on the role
>>>>> itself? We've got a Meta-role -> Application
>>>>> role -> High level role architecture going (I
>>>>> believe it's just the same as yours except for
>>>>> the meta-role), and the group association is
>>>>> defined on the meta-role. Do you mean we
>>>>> should somehow define strength there? because
>>>>> it isn't explicitly set.
>>>>> This is the inducement for the group
>>>>> association on the meta-role definition:
>>>>> <inducement id="2"> <construction>
>>>>> <resourceRef
>>>>> oid="00000000-0000-1de4-0002-000000000003"
>>>>> type="c:ResourceType"><!-- BANNER_USUARIOS
>>>>> --></resourceRef> <kind>account</kind>
>>>>> <intent>default</intent>
>>>>> <association>
>>>>> <c:ref>ri:GroupObjectClass</c:ref>
>>>>> <outbound> <expression> <associationFromLink>
>>>>> <projectionDiscriminator>
>>>>> <kind>entitlement</kind>
>>>>> <intent>default</intent>
>>>>> </projectionDiscriminator>
>>>>> </associationFromLink> </expression>
>>>>> </outbound> </association>
>>>>> </construction> <order>2</order> </inducement>
>>>>> Don't mind me if I sound a bit confused.
>>>>> Thanks for your help.
>>>>> *Rodrigo Yanis.* Identicum S.A. Jorge Newbery
>>>>> 3226 Tel: +54 (11)
>>>>> 4824-9971ryanis at identicum.com
>>>>> <mailto:ryanis at identicum.com>
>>>>> www.identicum.com <http://www.identicum.com/>
>>>>> 2016-11-10 13:51 GMT-05:00 Ivan Noris
>>>>> <ivan.noris at evolveum.com
>>>>> <mailto:ivan.noris at evolveum.com>>:
>>>>>
>>>>> Hi Rodrigo,
>>>>>
>>>>> unfortunately no other idea yet. I was
>>>>> running recompute ca. two weeks ago to
>>>>> remove some application groups that were
>>>>> not added by midPoint, the goal was to
>>>>> have association configuration with
>>>>> tolerant=false and it worked (this was
>>>>> custom connector, not ScriptedSQL):
>>>>>
>>>>> <association> <ref>ri:wsEntitlements</ref>
>>>>> <tolerant>false</tolerant>
>>>>> <matchingRule>mr:stringIgnoreCase</matchingRule>
>>>>> <kind>entitlement</kind>
>>>>> <intent>ws-entitlement</intent>
>>>>> <direction>objectToSubject</direction>
>>>>> <associationAttribute>ri:accountId</associationAttribute>
>>>>> <valueAttribute>icfs:uid</valueAttribute>
>>>>> </association>
>>>>>
>>>>> In all roles where association is used,
>>>>> <strength>strong</strength> is used as
>>>>> well (but the tolerant=false is a must).
>>>>> The recompute then worked as supposed and
>>>>> removed all non-midpoint groups from the
>>>>> accounts. The accounts were constructed by
>>>>> hierarchical roles (User - assign -
>>>>> Business role - inducement - Application
>>>>> role) and the association was in the
>>>>> Application role.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Ivan
>>>>>
>>>>> On 11/10/2016 06:21 PM, Rodrigo Yanis wrote:
>>>>>>
>>>>>> Hello Ivan, thanks for you response.
>>>>>>
>>>>>> Unfortunatelly this didn't work. All our
>>>>>> association attributes are set to
>>>>>> tolerance=false by default.
>>>>>>
>>>>>> Strange thing is, this only happens when
>>>>>> reconciling on already assigned high
>>>>>> level roles, not on assignment time.
>>>>>>
>>>>>> Any other suggestion? Thanks again,
>>>>>>
>>>>>> *Rodrigo Yanis.* Identicum S.A. Jorge
>>>>>> Newbery 3226 Tel: +54 (11)
>>>>>> 4824-9971ryanis at identicum.com
>>>>>> <mailto:ryanis at identicum.com>
>>>>>> www.identicum.com <http://www.identicum.com/>
>>>>>> 2016-11-10 9:48 GMT-05:00 Ivan Noris
>>>>>> <ivan.noris at evolveum.com
>>>>>> <mailto:ivan.noris at evolveum.com>>:
>>>>>>
>>>>>> Hi Rodrigo,
>>>>>>
>>>>>> maybe <tolerant>false</tolerant> for
>>>>>> association or your group attribute
>>>>>> (if not using associations) could help...
>>>>>>
>>>>>> Ivan
>>>>>>
>>>>>> On 11/10/2016 03:33 PM, Rodrigo Yanis
>>>>>> wrote:
>>>>>>> Hello everyone,
>>>>>>> We're having issues with our
>>>>>>> ScriptedSQL connector misshandling
>>>>>>> group membership removals when said
>>>>>>> memberships come from roles that are
>>>>>>> inherited from a higher level role,
>>>>>>> that is assigned to the user.
>>>>>>> When we remove the database role
>>>>>>> (the one that is linked to the
>>>>>>> resource's meta-role, and represents
>>>>>>> a database group) from the higher
>>>>>>> level role, and perform a
>>>>>>> reconciliation on the user, this
>>>>>>> does not remove the group membership
>>>>>>> of this user in the database. This
>>>>>>> only happens if the database role is
>>>>>>> assigned directly to the user, and
>>>>>>> then removed.
>>>>>>> We've also tried with a recompute
>>>>>>> task on the user, still with no luck.
>>>>>>> Since our role hierarchy does not
>>>>>>> support this last option, we must
>>>>>>> find a way (either through a task or
>>>>>>> directly) to remove memberships to
>>>>>>> roles that are no longer induced
>>>>>>> into the high level role.
>>>>>>> Do you have an idea on how to proceed?
>>>>>>> Thanks for your help
>>>>>>> *Rodrigo Yanis.* Identicum S.A.
>>>>>>> Jorge Newbery 3226 Tel: +54 (11)
>>>>>>> 4824-9971ryanis at identicum.com
>>>>>>> <mailto:ryanis at identicum.com>
>>>>>>> www.identicum.com
>>>>>>> <http://www.identicum.com/>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> midPoint mailing list
>>>>>>> midPoint at lists.evolveum.com
>>>>>>> <mailto:midPoint at lists.evolveum.com>
>>>>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>>>
>>>>>> --
>>>>>> Ivan Noris
>>>>>> Senior Identity Engineer
>>>>>> evolveum.com <http://evolveum.com>
>>>>>>
>>>>>> _______________________________________________
>>>>>> midPoint mailing list
>>>>>> midPoint at lists.evolveum.com
>>>>>> <mailto:midPoint at lists.evolveum.com>
>>>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> midPoint mailing list
>>>>>> midPoint at lists.evolveum.com
>>>>>> <mailto:midPoint at lists.evolveum.com>
>>>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>> --
>>>>> Ivan Noris
>>>>> Senior Identity Engineer
>>>>> evolveum.com <http://evolveum.com>
>>>>>
>>>>> _______________________________________________
>>>>> midPoint mailing list
>>>>> midPoint at lists.evolveum.com
>>>>> <mailto:midPoint at lists.evolveum.com>
>>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> midPoint mailing list
>>>>> midPoint at lists.evolveum.com
>>>>> <mailto:midPoint at lists.evolveum.com>
>>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>> --
>>>> Ivan Noris
>>>> Senior Identity Engineer
>>>> evolveum.com <http://evolveum.com>
>>>>
>>>> _______________________________________________
>>>> midPoint mailing list
>>>> midPoint at lists.evolveum.com
>>>> <mailto:midPoint at lists.evolveum.com>
>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>
>>>>
>>>> _______________________________________________
>>>> midPoint mailing list midPoint at lists.evolveum.com
>>>> <mailto:midPoint at lists.evolveum.com>
>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>>
>>>> _______________________________________________
>>>> midPoint mailing list
>>>> midPoint at lists.evolveum.com
>>>> <mailto:midPoint at lists.evolveum.com>
>>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>> _______________________________________________ midPoint
>>> mailing list midPoint at lists.evolveum.com
>>> <mailto:midPoint at lists.evolveum.com>
>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>>>
>>> _______________________________________________
>>> midPoint mailing list
>>> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>> <http://lists.evolveum.com/mailman/listinfo/midpoint>
> _______________________________________________ midPoint mailing
> list midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
> http://lists.evolveum.com/mailman/listinfo/midpoint
> <http://lists.evolveum.com/mailman/listinfo/midpoint>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20161121/34911261/attachment.htm>
More information about the midPoint
mailing list