[midPoint] Disable instead of delete
Pavol Mederly
mederly at evolveum.com
Wed Aug 26 10:24:21 CEST 2020
Hello Richard,
let me answer your messages from the bottom up.
> The bit I see under Mapping evaluation is:
>
> Mapping - OUTBOUND (Target - UAMS Test Table):
> activation/effectiveStatus -> activation/administrativeStatus
> | Strength: STRONG
> | Input administrativeStatus: (no values)
> | Input legal: true
> | Input assigned: false
> | Input focusExists: true
> | Input input: ENABLED
> | Condition: true -> true
> | Time validity: true
> | Output: ENABLED
Yes. Exactly the same I see here in my trace viewer.
> So legal is true, and input goes to enabled. However, assigned is false
> when I am not part of the org that grants the resource. However, when I
> am part of the org that grants the resource:
>
> Mapping - OUTBOUND (Target - UAMS Test Table):
> activation/effectiveStatus -> activation/administrativeStatus
> | Strength: STRONG
> | Input administrativeStatus: (no values)
> | Input legal: true
> | Input assigned: true
> | Input focusExists: true
> | Input input: ENABLED
> | Condition: true -> true
> | Time validity: true
> | Output: ENABLED
>
> Assigned is true. So assigned appears to follow what I want.
Yes.
> It's
> slightly confusing to me because:
>
> legality: set to true if the object is legal (based on assignment
> evaluation). See Assignment
>
> So I don't quite understand why legal would be true when assigned is
> false. But it appears if I change the legal part in the if check to
> assigned, I will get the results I want?
Most probably yes. I was confused originally a little bit as well,
because I do not know this part of midPoint into great depth.
This is crucial (from
https://wiki.evolveum.com/display/midPoint/Disable+instead+of+Delete):
"That means if there is a valid assignment for that account *or
if the account is allowed by any other policy (such as **Projection
Policy <https://wiki.evolveum.com/display/midPoint/Projection+Policy>**).*"
In your case, the default (/relative/) projection policy ensures that -
assuming there is no change in the assignment - any existing account is
considered legal. On the contrary, under /full /projection policy, only
really assigned accounts are considered legal. (As I say, I didn't
realize this fact.)
It is quite nicely seen also in midPoint source code:
https://github.com/Evolveum/midpoint/blob/3887e4f2d1f33679f6d15cf37e9987840a8003e6/model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/projector/focus/AssignmentProcessor.java#L713-L755
and in your log file:
2020-08-25 15:55:36,723 [MODEL] [midPointScheduler_Worker-2] TRACE
(com.evolveum.midpoint.model.impl.lens.projector.focus.AssignmentProcessor):
Projection (account (default) on
resource:e7e1a406-5a1d-456c-aa05-af7da589e33f(Target - UAMS Test Table))
*legal: no change in RELATIVE policy*
2020-08-25 15:55:36,723 [MODEL] [midPointScheduler_Worker-2] TRACE
(com.evolveum.midpoint.model.impl.lens.projector.focus.AssignmentProcessor):
Finishing legal decision for (account (default) on
resource:e7e1a406-5a1d-456c-aa05-af7da589e33f(Target - UAMS Test
Table)), thombstone false, enforcement mode RELATIVE, legalize false:
true -> true
So, as far as I understand, you have basically two options:
1. As you suggested, using "assigned" instead of "legal". As far as I
know, this should suit your needs.
2. Or switching from default projection policy to full projection
policy. This is perhaps the recommended policy when your
configuration is fine-tuned. But until that, it can be dangerous,
because it causes automatic deletion of all accounts that have no
assignment for them.
As for your first message:
> Done. Don't entirely understand what the short output from the GUI is
> telling me. And that's one giant XML document. I have sanitized the
> doc, mostly just replacing attributes about me. Also deleted credential
> blocks that showed up in there. https://wings.it.ndsu.edu/trace.zip
Yes. The XML document is so large because it contains almost everything
I need to analyze the situation - all evaluations of all
scripts/mappings, almost all objects used, detailed logs from the
processing, etc. It is so to avoid round trips with customers like "send
me logs from X", then "run it again and please give me also Y and Z", etc.
And yes, it can contain sensitive information. I thought you have only a
kind of playground installation, so no real data. Anyway, as you said,
you managed to remove sensitive info. (You can now remove the whole file.)
BTW, here is also the reason one should use config properties for
passwords, etc: such properties will not be part of these trace files,
only if there would be a bug somewhere. (Unlike things that are present
in resource XML objects. They are part of these files by design.)
> Setup:
>
> 1) Grouper bits from grouper-demo are installed and appear to work,
> this involves a function library, user template, org, a metarole, and
> an archetype.
> 2) Grouper group is turned into a midPoint org by grouper connector
> 3) midPoint org has a inducement on it to a DatabaseTableConnector
> resource (was CSV, switching over to something more useful to me,
> result is the same)
> 4) Resource has simulated disable capability and the configuration from
> disable instead of delete
> 5) Add person to Grouper group, they show up in the midPoint org and
> with a projection to the resource and in the table enabled.
> 6) Remove person from Grouper group, they are removed from midPoint org
> and show up in the table disabled.
> 7) Trigger a different resource on that user and they end back up
> enabled and not in the midPoint org.
Yes.
> We have one resource that takes people from a different midPoint org
> and populates a table using the DatabaseTable connector for use as the
> subject table by Grouper. That's the Target - Grouper NDSU Subjects
> resource. I stuck the trace bit on the live sync task for that
> resource. I suspended that task, updated the last modified on my row in
> that table, ran the live sync task, then suspended to generate the
> trace.
This is a point I don't understand: The Target - Grouper NDSU Subjects
is your target resource. You are managing its content from midPoint. And
you use the content in Grouper. So far so good.
But why you use this same resource as a /source/ for midPoint (using
live sync)? Was it just for the sake of demonstration of this issue?
> Maybe this is normal, but I find this odd in what the GUI will spit
> out:
>
> Clockwork run with focus '40506'
> Clockwork click: state INITIAL, execution wave 0, projection wave 0
> Focus loaded
> Mapping - TEMPLATE (grouper-template-user): name -> assignment
> Mapping - OUTBOUND (Target - UAMS Test Table): legal ->
> shadowExists
> Mapping - OUTBOUND (Target - UAMS Test Table):
> activation/effectiveStatus -> activation/administrativeStatus
> Mapping - OUTBOUND (Target - Grouper NDSU Subjects):
> $user/extension/username -> attributes/email
>
> It goes from the template, to the disabled resource, then to the
> resource triggered in the live sync.
Well, this is about what level of details you set in the trace viewer.
(What kinds of operations should it show.) So if I select e.g. only an
overview, I will see all the mappings, but with no particular context.
If I select all operations, I will find the context but then I can get
lost in the processing. (I am speaking of the viewer used in IntelliJ
IDEA plugin, because I haven't used the web GUI viewer for a long time.)
So, currently the viewers require some amount of midPoint knowledge to
be used effectively. I hope that gradually they will be improved to be
suitable also for wider audience.
But, overall, you managed to find the issue and resolution yourself, so
I think it was of some help to you. :-)
> I tried adding a trace to the
> Grouper Async task, but that didn't yield any log files.
It is done in a slightly different way because the asynchronous
processing is architecturally different from standard processing. So the
tracing has to be set in async update connector configuration, e.g.
https://github.com/Evolveum/midpoint/blob/7887936b52e133b6a7c7476490926827db29dace/provisioning/provisioning-impl/src/test/resources/async/resource-async-caching.xml#L20-L49.
You know, these things are very, very experimental, so many of them are
not documented.
---
A personal note: The analysis I have done for you is something that I
really love to do. We do that regularly for our customers. I did it here
just because I liked your work (e.g. the blogpost about Grouper
integration) and your approach to midPoint. And I love the academy as
such, being myself from this environment. But it's not sustainable - I
did this on my own time (which I have almost zero, and this work was
almost 2 hours for me), and now I have to return to work for our
customers and midPoint development as such. So, please, do really
consider purchasing a subscription. Maybe you'll be surprised that the
price is not that high for HE sector. Then we would be able to cooperate
in a regular way, not on "best effort" (which I have depleted for longer
time now) basis. :-)
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 25/08/2020 23:50, Richard Frovarp wrote:
> Done. Don't entirely understand what the short output from the GUI is
> telling me. And that's one giant XML document. I have sanitized the
> doc, mostly just replacing attributes about me. Also deleted credential
> blocks that showed up in there. https://wings.it.ndsu.edu/trace.zip
>
> Setup:
>
> 1) Grouper bits from grouper-demo are installed and appear to work,
> this involves a function library, user template, org, a metarole, and
> an archetype.
> 2) Grouper group is turned into a midPoint org by grouper connector
> 3) midPoint org has a inducement on it to a DatabaseTableConnector
> resource (was CSV, switching over to something more useful to me,
> result is the same)
> 4) Resource has simulated disable capability and the configuration from
> disable instead of delete
> 5) Add person to Grouper group, they show up in the midPoint org and
> with a projection to the resource and in the table enabled.
> 6) Remove person from Grouper group, they are removed from midPoint org
> and show up in the table disabled.
> 7) Trigger a different resource on that user and they end back up
> enabled and not in the midPoint org.
>
> We have one resource that takes people from a different midPoint org
> and populates a table using the DatabaseTable connector for use as the
> subject table by Grouper. That's the Target - Grouper NDSU Subjects
> resource. I stuck the trace bit on the live sync task for that
> resource. I suspended that task, updated the last modified on my row in
> that table, ran the live sync task, then suspended to generate the
> trace.
>
> Maybe this is normal, but I find this odd in what the GUI will spit
> out:
>
> Clockwork run with focus '40506'
> Clockwork click: state INITIAL, execution wave 0, projection wave 0
> Focus loaded
> Mapping - TEMPLATE (grouper-template-user): name -> assignment
> Mapping - OUTBOUND (Target - UAMS Test Table): legal ->
> shadowExists
> Mapping - OUTBOUND (Target - UAMS Test Table):
> activation/effectiveStatus -> activation/administrativeStatus
> Mapping - OUTBOUND (Target - Grouper NDSU Subjects):
> $user/extension/username -> attributes/email
>
> It goes from the template, to the disabled resource, then to the
> resource triggered in the live sync. I tried adding a trace to the
> Grouper Async task, but that didn't yield any log files.
>
> Thanks,
> Richard
>
>
> On Tue, 2020-08-25 at 17:44 +0200, Pavol Mederly wrote:
>> This looks OK to me. The "legal" variable should be false if there is
>> no
>> assignment creating particular account.
>>
>> (Although the code is a bit more complex than this. So there might be
>> a
>> bug in midPoint or some strange misconfiguration on your side.)
>>
>> If you would create and send us so called trace file (see
>>
> https://wiki.evolveum.com/display/midPoint/Troubleshooting+with+traces,
>>
>> in particular the section "Recording traces for background tasks"),
>> I
>> would try to find a couple of minutes to analyze it.
>>
>> In your particular situation - if the issue emerges during
>> reconciliation of a different resource, then you'd need to include
>> XML
>> snipped starting with "mext:tracing" to the reconciliation task for
>> that
>> resource. Note that tracing will slow down the processing
>> significantly,
>> so you need the reconciliation to execute only a couple (tens at
>> most)
>> of records.
>>
>> Best regards,
>>
>> Pavol Mederly
>> Software developer
>> evolveum.com
>>
>> On 25/08/2020 17:29, Richard Frovarp wrote:
>>> I'm copying the
>>>
> https://wiki.evolveum.com/display/midPoint/Disable+instead+of+Delete
>>> without the extra "Even more" complex logic:
>>>
>>> <activation>
>>> <existence>
>>> <outbound>
>>> <strength>weak</strength>
>>> <expression>
>>> <path>$focusExists</path>
>>> </expression>
>>> </outbound>
>>> </existence>
>>> <administrativeStatus>
>>> <outbound>
>>> <strength>strong</strength>
>>> <expression>
>>> <script>
>>> <code>
>>> import
>>> com.evolveum.midpoint.xml.ns._public.common.common_3.ActivationStat
>>> usTy
>>> pe;
>>> if (legal) {
>>> input;
>>> } else {
>>> ActivationStatusType.DISABLED;
>>> }
>>> </code>
>>> </script>
>>> </expression>
>>> </outbound>
>>> </administrativeStatus>
>>> </activation>
>>>
>>> The disabled bit works. My confusion comes from the other resource
>>> reconciling causing this one to go to enabled. Still trying to wrap
>>> my
>>> head around legal. Guessing under that condition input become
>>> enabled.
>>>
>>> On Tue, 2020-08-25 at 17:18 +0200, Pavol Mederly wrote:
>>>> Hello Richard,
>>>>
>>>> I have no particular experience with this disable-instead-of-
>>>> delete
>>>> configuration but I think it should be doable.
>>>>
>>>> Could you, please, share relevant parts of your configuration? I
>>>> mean
>>>> mainly the activation mappings for your resources.
>>>>
>>>> Best regards,
>>>>
>>>> Pavol Mederly
>>>> Software developer
>>>> evolveum.com
>>>>
>>>> On 25/08/2020 01:12, Richard Frovarp wrote:
>>>>> I'm trying to figure out how to do disable instead of delete on
>>>>> a
>>>>> single resource. I've read the wiki, and mostly, kinda, sort of
>>>>> understand it. In fact, I have it working as I think it is
>>>>> intended
>>>>> to
>>>>> work. Which isn't how I need it to work and I'm getting stuck
>>>>> on
>>>>> terms
>>>>> I think.
>>>>>
>>>>> I have a test resource that is an inducement on an org that is
>>>>> populated by Grouper. The resource is a CSV file with a
>>>>> simulated
>>>>> disable capability. I add someone to the Grouper group, the
>>>>> async
>>>>> handler adds them to the org, and they are added to the CSV.
>>>>> Life
>>>>> is
>>>>> good. I remove them from the Grouper group, they are removed
>>>>> from
>>>>> the
>>>>> org, and the user has a disabled administrative status on their
>>>>> shadow
>>>>> on the resource. The user has an undefined administrative
>>>>> status,
>>>>> and
>>>>> the other resources which don't have the disabled capability
>>>>> are
>>>>> however they are.
>>>>>
>>>>> However, the next morning an unrelated reconciliation on a
>>>>> different
>>>>> resource runs and that turns the administrative status for the
>>>>> CSV
>>>>> resource back to enabled. That's not what I want. I want the
>>>>> resource
>>>>> to remain in the disabled state. I think this is because it is
>>>>> setting
>>>>> the user back to enabled, and the example makes it so that the
>>>>> resource
>>>>> follows the user. That's not what I want in my instance. I may
>>>>> be
>>>>> disabling a resource because the person is no longer an
>>>>> employee
>>>>> and is
>>>>> only a student. Thus their employee resources are disabled for
>>>>> a
>>>>> period
>>>>> of time before we delete them.
>>>>>
>>>>> What am I missing? The other key here is that if they are added
>>>>> back to
>>>>> the Grouper resource, they should be set back to enabled.
>>>>>
>>>>> Thanks,
>>>>> Richard
>>>>> _______________________________________________
>>>>> midPoint mailing list
>>>>> midPoint at lists.evolveum.com
>>>>> https://lists.evolveum.com/mailman/listinfo/midpoint
>>>> _______________________________________________
>>>> midPoint mailing list
>>>> midPoint at lists.evolveum.com
>>>> https://lists.evolveum.com/mailman/listinfo/midpoint
>>> _______________________________________________
>>> midPoint mailing list
>>> midPoint at lists.evolveum.com
>>> https://lists.evolveum.com/mailman/listinfo/midpoint
>> _______________________________________________
>> midPoint mailing list
>> midPoint at lists.evolveum.com
>> https://lists.evolveum.com/mailman/listinfo/midpoint
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> https://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20200826/5655b584/attachment.htm>
More information about the midPoint
mailing list