[midPoint] Couldn't determine shadow name

Thu Jul 7 15:26:16 CEST 2022

Yes, I have implemented an executeQuery so I can do kind of fetch by UID. The follows is the entire stack trace. By the way, I am using midpoint 4.5 on a local machine.

15:12:31.577 [pool-3-thread-15] ERROR com.evolveum.midpoint.model.impl.lens.LensUtil - Couldn't determine name for shadow -- continuing with no name; shadow:
shadow: (3f01b3de-7a1c-4ab5-9735-e2cd8ed6e108, ShadowType)
  oid=3f01b3de-7a1c-4ab5-9735-e2cd8ed6e108, version=null
    objectClass: {...resource/instance-3}csAccount
    resourceRef: oid=a343fc2e-3954-4034-ba1a-2b72c21e577a(ResourceType)
    kind: ACCOUNT
    intent: default
    attributes:
        pid: 017
    iteration: 0
    iterationToken: .
com.evolveum.midpoint.util.exception.SchemaException: Could not determine shadow name.
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowStringName(ShadowUtil.java:695)
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowName(ShadowUtil.java:667)
          at com.evolveum.midpoint.model.impl.lens.LensUtil.createObjectDeltaOperation(LensUtil.java:911)
          at com.evolveum.midpoint.model.impl.lens.executor.DeltaExecution.addToExecutedDeltas(DeltaExecution.java:362)
          at com.evolveum.midpoint.model.impl.lens.executor.DeltaExecution.execute(DeltaExecution.java:174)
          at com.evolveum.midpoint.model.impl.lens.executor.ProjectionChangeExecution.execute(ProjectionChangeExecution.java:129)
          at com.evolveum.midpoint.model.impl.lens.ChangeExecutor.executeProjectionsChanges(ChangeExecutor.java:98)
          at com.evolveum.midpoint.model.impl.lens.ChangeExecutor.executeChanges(ChangeExecutor.java:61)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.lambda$processSecondary$0(ClockworkClick.java:229)
          at com.evolveum.midpoint.model.impl.lens.ClockworkMedic.partialExecute(ClockworkMedic.java:343)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.processSecondary(ClockworkClick.java:227)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.moveStateForward(ClockworkClick.java:194)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.click(ClockworkClick.java:118)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.click(Clockwork.java:382)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.runWithConflictDetection(Clockwork.java:150)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.run(Clockwork.java:104)
          at com.evolveum.midpoint.model.impl.controller.ModelController.executeChangesNonRaw(ModelController.java:444)
          at com.evolveum.midpoint.model.impl.controller.ModelController.executeChanges(ModelController.java:405)
          at com.evolveum.midpoint.gui.impl.page.admin.ProgressAwareChangesExecutorImpl$1.callWithContextPrepared(ProgressAwareChangesExecutorImpl.java:148)
          at com.evolveum.midpoint.gui.impl.page.admin.ProgressAwareChangesExecutorImpl$1.callWithContextPrepared(ProgressAwareChangesExecutorImpl.java:133)
          at com.evolveum.midpoint.web.component.SecurityContextAwareCallable.call(SecurityContextAwareCallable.java:50)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:829)
15:12:31.598 [pool-3-thread-15] ERROR c.e.midpoint.model.impl.lens.LensProjectionContext - Couldn't determine shadow name for shadow:3f01b3de-7a1c-4ab5-9735-e2cd8ed6e108(null).
com.evolveum.midpoint.util.exception.SchemaException: Could not determine shadow name.
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowStringName(ShadowUtil.java:695)
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowName(ShadowUtil.java:667)
          at com.evolveum.midpoint.model.impl.lens.LensProjectionContext.resolveNameIfKnown(LensProjectionContext.java:1529)
          at com.evolveum.midpoint.model.impl.lens.LensContext.lambda$getNameResolver$1(LensContext.java:1740)
          at com.evolveum.midpoint.model.common.util.AuditHelper.lambda$resolveNamesInDeltas$0(AuditHelper.java:117)
          at com.evolveum.midpoint.schema.util.ObjectDeltaSchemaLevelUtil.lambda$resolveNames$0(ObjectDeltaSchemaLevelUtil.java:68)
          at com.evolveum.midpoint.prism.impl.PrismValueImpl.accept(PrismValueImpl.java:192)
          at com.evolveum.midpoint.prism.impl.PrismReferenceValueImpl.accept(PrismReferenceValueImpl.java:731)
          at com.evolveum.midpoint.prism.impl.delta.ItemDeltaImpl.accept(ItemDeltaImpl.java:176)
          at com.evolveum.midpoint.prism.impl.delta.ObjectDeltaImpl.accept(ObjectDeltaImpl.java:99)
          at com.evolveum.midpoint.prism.impl.delta.ObjectDeltaImpl.accept(ObjectDeltaImpl.java:89)
          at com.evolveum.midpoint.schema.util.ObjectDeltaSchemaLevelUtil.resolveNames(ObjectDeltaSchemaLevelUtil.java:83)
          at com.evolveum.midpoint.model.common.util.AuditHelper.resolveNamesInDeltas(AuditHelper.java:140)
          at com.evolveum.midpoint.model.common.util.AuditHelper.audit(AuditHelper.java:87)
          at com.evolveum.midpoint.model.impl.lens.ClockworkAuditHelper.auditEvent(ClockworkAuditHelper.java:215)
          at com.evolveum.midpoint.model.impl.lens.ClockworkAuditHelper.audit(ClockworkAuditHelper.java:78)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.processSecondary(ClockworkClick.java:235)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.moveStateForward(ClockworkClick.java:194)
          at com.evolveum.midpoint.model.impl.lens.ClockworkClick.click(ClockworkClick.java:118)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.click(Clockwork.java:382)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.runWithConflictDetection(Clockwork.java:150)
          at com.evolveum.midpoint.model.impl.lens.Clockwork.run(Clockwork.java:104)
          at com.evolveum.midpoint.model.impl.controller.ModelController.executeChangesNonRaw(ModelController.java:444)
          at com.evolveum.midpoint.model.impl.controller.ModelController.executeChanges(ModelController.java:405)
          at com.evolveum.midpoint.gui.impl.page.admin.ProgressAwareChangesExecutorImpl$1.callWithContextPrepared(ProgressAwareChangesExecutorImpl.java:148)
          at com.evolveum.midpoint.gui.impl.page.admin.ProgressAwareChangesExecutorImpl$1.callWithContextPrepared(ProgressAwareChangesExecutorImpl.java:133)
          at com.evolveum.midpoint.web.component.SecurityContextAwareCallable.call(SecurityContextAwareCallable.java:50)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:829)

From: Pavol Mederly <mederly at evolveum.com>
Date: Thursday, July 7, 2022 at 9:23 AM
To: Wang, Xiaoshu <xiaoshuw at email.unc.edu>, midPoint General Discussion <midPoint at lists.evolveum.com>
Subject: Re: [midPoint] Couldn't determine shadow name

That's great. I am now not sure - have you implemented a (limited) executeQuery method that would allow midPoint to fetch your resource objects "by UID"?

Also, I become more and more interested to see the stack trace of the "SchemaException" you mentioned. :) I would like to know the path between points 3 and 4 in your description below, i.e. creating an object and trying to determine its name.

Best regards,

--

Pavol Mederly

Software developer

evolveum.com
On 07/07/2022 15:19, Wang, Xiaoshu wrote:
Yes, I can execute a single query given a UID.

Xiaoshu

From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> on behalf of Pavol Mederly via midPoint <midpoint at lists.evolveum.com><mailto:midpoint at lists.evolveum.com>
Date: Thursday, July 7, 2022 at 6:32 AM
To: midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com> <midPoint at lists.evolveum.com><mailto:midPoint at lists.evolveum.com>
Cc: Pavol Mederly <mederly at evolveum.com><mailto:mederly at evolveum.com>
Subject: Re: [midPoint] Couldn't determine shadow name

Hello Xiaoshu,

OK, I understand that you cannot execute a "blank" query that would return all objects on a resource. But can you execute - at least - a query for one specific object, identified by UID?

--

Just by the way, could you share the whole stack trace of the "SchemaException: Could not determine shadow name"? I am quite interested in the whole execution path there. And, the exact midPoint version.

Best regards,

--

Pavol Mederly

Software developer

evolveum.com
On 06/07/2022 20:00, Wang, Xiaoshu via midPoint wrote:
Pavol:

I am not sure how.

The icf connector’s create signature is this. And see the implementation

public Uid create(ObjectClass objectClass, Set<Attribute> attributes, OperationOptions operationOptions) {

Optional<String> pid = CsUtil.findPid(attributes);

if (pid.isPresent()) {

    LOG.info(">>> Done CsCreate.create");

    Name name = new Name(pid.get());

    Uid uid = new Uid(pid.get(), name);

    return uid;

} else {

    throw new IllegalArgumentException("Cannot create because PID is empty");

}
}

I can only return a Uid object. Providing the nameHint seems have no effect.

Most “standard” resource will go through the executeQuery method (e.g, through an inititial import).

public void executeQuery(ObjectClass objectClass, PersonSearch personSearch, ResultsHandler resultsHandler, OperationOptions operationOptions) {

Here, you can construct a connectorObject and ask the resultHandler to handle it. So, I can set both uid and name to whatever I want.

But, because my resource cannot do the blank query so the midpoint will call the create method when needed. But the create() is unaware of connectorObject, it only returns a unique handle, i.e., the UID.

What I guess is this.

  1.  A property-change triggers an outbound mapping
  2.  Midpoint notices the absence of a shadow account for the outbound resource
  3.  Midpoint called the connector.create() method to create the intended resource and subsequently create the shadowObject with the returned uid.
  4.  At a later stage, it triggers the ShadowUtil.determineShadowName method and throw an exception when found absent.

A potential remedy is step #3, when midpoint can use the nameHint of the Uid object to fill in the name of the shadow account.
Xiaoshu
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> on behalf of Pavol Mederly via midPoint <midpoint at lists.evolveum.com><mailto:midpoint at lists.evolveum.com>
Date: Wednesday, July 6, 2022 at 1:35 PM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com> <midpoint at lists.evolveum.com><mailto:midpoint at lists.evolveum.com>
Cc: Pavol Mederly <mederly at evolveum.com><mailto:mederly at evolveum.com>
Subject: Re: [midPoint] Couldn't determine shadow name

Hello Xiaoshu,

I am still not sure I completely understand your use case, but maybe the simplest thing your connector can do is to mirror __UID__ to __NAME__ itself (if the name is not known). This is actually what many connectors do.

See also https://docs.evolveum.com/connectors/connid/1.x/connector-development-guide/#__uid__-and-__name__-are-the-same<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.evolveum.com%2Fconnectors%2Fconnid%2F1.x%2Fconnector-development-guide%2F%23__uid__-and-__name__-are-the-same&data=05%7C01%7C%7Cf10df076c0ab4d91263408da601bcfb2%7C58b3d54f16c942d3af081fcabd095666%7C1%7C0%7C637927969823060003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2dxaENoPuGwfljm9jgLK9cgg1ZA9BxNk57l%2Fm%2F%2BNVPI%3D&reserved=0>.

I am not sure why both of these are obligatory in ConnId (originally ICF, then OpenICF).

The ShadowUtil.determineShadowName method is maybe not very consistent, I must admit. (I have looked at it right now.) For example, it is called from a couple of places in midPoint, and a caller usually catches SchemaException, and continues with "no name" situation. I am not sure about your specific situation, because I don't know exact midPoint version, whole stack trace, and so on. But I think the best you can do is simply to return both __UID__ and __NAME__ from your connector, and (hopefully) midPoint will be happy with it.

Best regards,

--

Pavol Mederly

Software developer

evolveum.com
On 06/07/2022 17:34, Wang, Xiaoshu via midPoint wrote:
Pavol:

Conceptually, I know “why” it throws the exception but I wonder if it can be considered as a “bug” or “improvement” for the midpoint.

I have a particular use case, where I have a resource that I only need to do outbound mapping. The resource, however, does not provide an API for me to do something like getAll() operations. In other words, I cannot do an initial import kind of task to force midpoint to create all the shadow account.

Thus, when a property change that triggers an outbound mapping, midpoint noted that the resource does not exist, so it calls the create() method on the connector. Unlike the executeQuery method, which can construct a connectorObject and ask midpoint to handle it. The create method can only return a “Uid”. I understand the rational under this as the connector.create() is meant to create a remote resource as opposed to create the shadow account, (I guess is mirrored by the connectorObject). Upon receiving the UID, the midpoint then creates a matching shadow account with it (due to the absence of it). However, as both __UID__ and __NAME__ are *required* attributes for a connectorObject (i.e., shadow account), then a later verification step failed the check and throws the exception.

The exception appears non-preemptive, i.e., it does create the shadow account, which name, I can later fill when the updateDelta is called. The exception trace would be more of a nuance than error. Yet, I think it would be nice that midpoint can do some auto remedy for it. For example, during the validation phase, copy the __uid__ to __name__ if the latter is absent etc.

Or, if there might be some other way to get around this particular use case of mine, I would be glad to hear it.

Xiaoshu

From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> on behalf of Pavol Mederly via midPoint <midpoint at lists.evolveum.com><mailto:midpoint at lists.evolveum.com>
Date: Wednesday, July 6, 2022 at 5:10 AM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com> <midpoint at lists.evolveum.com><mailto:midpoint at lists.evolveum.com>
Cc: Pavol Mederly <mederly at evolveum.com><mailto:mederly at evolveum.com>
Subject: Re: [midPoint] Couldn't determine shadow name

Hi Xiaoshu,

I always recommend to have a look at the code that throws the exception. In your case it's most probably this one:

https://github.com/Evolveum/midpoint/blob/d317537626a99db32764c486507f1c1ac5a47fb6/infra/schema/src/main/java/com/evolveum/midpoint/schema/util/ShadowUtil.java#L688-L736<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FEvolveum%2Fmidpoint%2Fblob%2Fd317537626a99db32764c486507f1c1ac5a47fb6%2Finfra%2Fschema%2Fsrc%2Fmain%2Fjava%2Fcom%2Fevolveum%2Fmidpoint%2Fschema%2Futil%2FShadowUtil.java%23L688-L736&data=05%7C01%7C%7Cf10df076c0ab4d91263408da601bcfb2%7C58b3d54f16c942d3af081fcabd095666%7C1%7C0%7C637927969823060003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fZ21g8ptR60mCiYL2SEsk5nYH05Mynt1Xf7NCePmexo%3D&reserved=0>

(lines are shifted probably because of midPoint version mismatch)

It should give you a clue - e.g. does your object have a primary identifier set? Etc...

Regards,

--

Pavol Mederly

Software developer

evolveum.com
On 23/06/2022 15:27, Wang, Xiaoshu via midPoint wrote:
Hi,

I am developing a custom connector. When I assign the resource connected with that connector, I got the following exception.

com.evolveum.midpoint.util.exception.SchemaException: Could not determine shadow name.
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowStringName(ShadowUtil.java:695)
          at com.evolveum.midpoint.schema.util.ShadowUtil.determineShadowName(ShadowUtil.java:667)
          ….

The exception seems not do anything “harmful” as far as I can tell because the new account would be listed under the resource. But I want to know why. Hope someone can shed some light on for me.

Xiaoshu

_______________________________________________

midPoint mailing list

midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>

https://lists.evolveum.com/mailman/listinfo/midpoint<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.evolveum.com%2Fmailman%2Flistinfo%2Fmidpoint&data=05%7C01%7C%7Cf10df076c0ab4d91263408da601bcfb2%7C58b3d54f16c942d3af081fcabd095666%7C1%7C0%7C637927969823060003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JwVUZv5V31TZFvezi%2Bsz0izk6kbsb1SNFO%2BqXhwMu%2F0%3D&reserved=0>

_______________________________________________

midPoint mailing list

midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>

https://lists.evolveum.com/mailman/listinfo/midpoint<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.evolveum.com%2Fmailman%2Flistinfo%2Fmidpoint&data=05%7C01%7C%7Cf10df076c0ab4d91263408da601bcfb2%7C58b3d54f16c942d3af081fcabd095666%7C1%7C0%7C637927969823060003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JwVUZv5V31TZFvezi%2Bsz0izk6kbsb1SNFO%2BqXhwMu%2F0%3D&reserved=0>

_______________________________________________

midPoint mailing list

midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>

https://lists.evolveum.com/mailman/listinfo/midpoint<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.evolveum.com%2Fmailman%2Flistinfo%2Fmidpoint&data=05%7C01%7C%7Cf10df076c0ab4d91263408da601bcfb2%7C58b3d54f16c942d3af081fcabd095666%7C1%7C0%7C637927969823060003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JwVUZv5V31TZFvezi%2Bsz0izk6kbsb1SNFO%2BqXhwMu%2F0%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220707/dcefb1d7/attachment-0001.htm>