[midPoint] duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"

Wed Aug 3 15:36:18 CEST 2022

Hello Pavol,

Thank you very much for the reply!

It seems like midpoint cannot find the either the user account or the associated groups in that user account.
But the shadow exists in midpoint
[cid:image001.png at 01D8A74C.D7C7FAB0]

[cid:image002.png at 01D8A74C.BE307730]

And from the REPO CONFLICT log, the resourceObject given to findConflictingShadow seems to be the right object but without oid. The potential conflicting repo shadow is exactly the object that exists in database.

What’s your advice to proceed?

Thank you very much for your help!

Best regards,
Hsin-Fang

Below is just part of the log I copied:
022-08-03 13:21:03,928 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction): Start adopting associations: 1
2022-08-03 13:21:03,928 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction): Determining shadowRef for PCV(null):[PP({.../common/common-3}name):[PPV(ItemName:group)], RAC(identifiers):[PCV(null):[RA({.../resource/instance-3}dn):[PPV(String:CN=grp-staff,OU=xxxxxx,DC=xxxxx,DC=xx)]]]]
2022-08-03 13:21:03,929 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction): Processing kind=ENTITLEMENT, intent=group (from the definition)
2022-08-03 13:21:03,929 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow using filter (repo):
2022-08-03 13:21:03,949 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primary identifier (attributes) using filter:
2022-08-03 13:21:03,950 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Found 0 shadows (live or dead)
2022-08-03 13:21:03,951 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Shadow object (in repo) corresponding to the resource object (on the resource) was not found. The repo shadow will be created. The resource object:
shadow:null(null)
2022-08-03 13:21:03,951 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowManager): Adding new shadow from resource object:
  shadow: (null, ShadowType)
2022-08-03 13:21:03,961 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Attempt to create new repo shadow for shadow:null(null) ended up in conflict, re-trying the search for repo shadow
2022-08-03 13:21:03,962 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primary identifier (attributes) using filter:
2022-08-03 13:21:03,962 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Found 0 shadows (live or dead)
2022-08-03 13:21:03,963 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primaryIdentifierValue using filter:
2022-08-03 13:21:03,964 [PROVISIONING] [http-nio-8088-exec-7] ERROR (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
  Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
com.evolveum.midpoint.util.exception.ObjectAlreadyExistsException: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
  Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.

2022-08-03 13:21:03,964 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: resource shadow
  shadow: (null, ShadowType)
    oid=null, version=null
      attributes:
          objectSid: S-1-5-21-4039012443-3458975651-2582944172-1142
          uSNChanged: 1721254
          sAMAccountName: grp-staff
          whenCreated: 2021-04-20T07:16:16.000Z
          cn: grp-staff
          member: [ CN=Midpoint Testneun, name: grp-staff
          uSNCreated: 76200
          objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=xxxx,DC=xx
          dSCorePropagationData: [ 2021-07-07T07:22:32.000Z, 2021-06-21T07:18:39.000Z, 2021-06-21T06:52:50.000Z, 2021-05-21T10:54:59.000Z, 1601-07-14T22:36:48.000Z ]
          dn: CN=grp-staff,OU=xxxx-xxxxx,DC=xxxx,DC=xx
          distinguishedName: CN=grp-staff,OU=xxxx-xxxx,DC=xxxx,DC=xx
          instanceType: 4
          groupType: -2147483646
          whenChanged: 2022-07-20T06:55:32.000Z
          objectGUID: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
      objectClass: {...resource/instance-3}group
      exists: true
2022-08-03 13:21:03,970 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: resource shadow: determined primaryIdentifierValue: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
2022-08-03 13:21:03,970 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: potential conflicting repo shadow (by primaryIdentifierValue)
  shadow: (fe35d3e8-7fc2-481d-9063-e9285fbb6164, v0, ShadowType)
    oid=fe35d3e8-7fc2-481d-9063-e9285fbb6164, version=0
      name: CN=grp-staff,OU=xxx-xxx,DC=xxxx,DC=xx
      resourceRef: oid=412a9b5d-4c88-4a64-a281-2995d46146c8(ResourceType)[default]
      synchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
      fullSynchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
      objectClass: {...resource/instance-3}group
      primaryIdentifierValue: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
      kind: ENTITLEMENT
      intent: group
      exists: true
      attributes:
          dn: [ cn=grp-staff,ou=xxxxxx,dc=xxx,dc=xx (raw) ]
          objectGUID: [ 756d6959-9a3a-4f33-a5e4-0c08ef72db37 (raw) ]
2022-08-03 13:21:03,971 [PROVISIONING] [http-nio-8088-exec-7] ERROR (com.evolveum.midpoint.model.impl.ModelObjectResolver): Error resolving object with oid 000425a8-6b34-4cad-83ef-f1e8177f0bf7, expected type was class com.evolveum.midpoint.xml.ns._public.common.common_3.ShadowType..
com.evolveum.midpoint.util.exception.SystemException: Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
  Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.

From: midPoint <midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly via midPoint
Sent: Wednesday, August 3, 2022 10:57 AM
To: midpoint at lists.evolveum.com
Cc: Pavol Mederly <mederly at evolveum.com>
Subject: Re: [midPoint] duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"

Hello Hsin-Fang,

there is a troubleshooting piece of code related to that exception:

https://github.com/Evolveum/midpoint/blob/5e841416523a6d46431443d36b056e91ada0b9f1/provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowAcquisition.java#L188-L196

You should enable the respective DEBUG logging and midPoint will provide you with more information.

Best regards,

--

Pavol Mederly

Software developer

evolveum.com
On 03/08/2022 10:53, Hsin-Fang Hsu via midPoint wrote:
Hi,

I upgraded midpoint from 4.4.1 (postgresql) to 4.4.2 (postgresql native).
The data was exported and imported into the database using ninja and the content in the new database also seems to be fine.
ADLdapConnector v3.3 is used.

AD test connection works. But when I tried to access any AD account, I got these kind of errors:
com.evolveum.midpoint.util.exception.SystemException: Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key" Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(bc1cd126-81dc-4c2a-acd0-b30eafdc40ef, 11, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.

Error resolving object with oid 'eb303210-59dd-499f-bbca-9c57cefa3295': Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key" Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(b035b94c-e127-466a-9011-1e4a2dc41fc5, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.

I guess the error occurs because midpoint try to save what it reads from AD to a new shadow but that shadow already exists in the database. But the configuration works fine in 4.4.1. How can I solve this?

Thank you very much for your help in advance!

Best regards,
Hsin-Fang

_______________________________________________

midPoint mailing list

midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>

https://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/3c20e1ba/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 90727 bytes
Desc: image002.png
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/3c20e1ba/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 68653 bytes
Desc: image001.png
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/3c20e1ba/attachment-0003.png>