[midPoint] duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Hsin-Fang Hsu
hsin-fang.hsu at itconcepts.ch
Wed Aug 3 17:50:02 CEST 2022
Hello Pavol,
Thank you so much for the super useful reply!!
In my case, the content of m_shadow.attributes (JSONB) column is null for all AD entries!
It turns out that I didn’t use the raw option (-r) in my export with ninja. I deleted the m_shadow table and re-import all the shadows. Now m_shadow has values in the attribute column and everything works!
Thanks again for your help!!
Best regards,
Hsin-Fang
From: midPoint <midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly via midPoint
Sent: Wednesday, August 3, 2022 4:38 PM
To: midpoint at lists.evolveum.com
Cc: Pavol Mederly <mederly at evolveum.com>
Subject: Re: [midPoint] duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Hello,
from the log it looks like the shadow
shadow: (fe35d3e8-7fc2-481d-9063-e9285fbb6164, v0, ShadowType)
oid=fe35d3e8-7fc2-481d-9063-e9285fbb6164, version=0
name: CN=grp-staff,OU=xxx-xxx,DC=xxxx,DC=xx
resourceRef: oid=412a9b5d-4c88-4a64-a281-2995d46146c8(ResourceType)[default]
synchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
fullSynchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
objectClass: {...resource/instance-3}group
primaryIdentifierValue: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
kind: ENTITLEMENT
intent: group
exists: true
attributes:
dn: [ cn=grp-staff,ou=xxxxxx,dc=xxx,dc=xx (raw) ]
objectGUID: [ 756d6959-9a3a-4f33-a5e4-0c08ef72db37 (raw) ]
cannot be found when querying by attributes/objectGUID.
This seems like some repository inconsistency. Unfortunately, I am not able to devote the time necessary to diagnose it; this is a typical case for our paid support.
If you would like to resolve this "on your own", you may try one of the following:
1. either look at DB records related to this shadow, analysing e.g. the content of its m_shadow.attributes (JSONB) column;
2. or simply delete and re-import the shadow (make sure that the exported version has correct xsi:type markings of the attributes).
Best regards,
--
Pavol Mederly
Software developer
evolveum.com
On 03/08/2022 15:36, Hsin-Fang Hsu via midPoint wrote:
Hello Pavol,
Thank you very much for the reply!
It seems like midpoint cannot find the either the user account or the associated groups in that user account.
But the shadow exists in midpoint
[cid:image001.png at 01D8A75A.2B2EEB20]
[cid:image002.png at 01D8A75A.2B2EEB20]
And from the REPO CONFLICT log, the resourceObject given to findConflictingShadow seems to be the right object but without oid. The potential conflicting repo shadow is exactly the object that exists in database.
What’s your advice to proceed?
Thank you very much for your help!
Best regards,
Hsin-Fang
Below is just part of the log I copied:
022-08-03 13:21:03,928 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction):Start adopting associations: 1
2022-08-03 13:21:03,928 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction):Determining shadowRef for PCV(null):[PP({.../common/common-3}name):[PPV(ItemName:group)], RAC(identifiers):[PCV(null):[RA({.../resource/instance-3}dn):[PPV(String:CN=grp-staff,OU=xxxxxx,DC=xxxxx,DC=xx)]]]]
2022-08-03 13:21:03,929 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowedObjectConstruction): Processing kind=ENTITLEMENT, intent=group (from the definition)
2022-08-03 13:21:03,929 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow using filter (repo):
2022-08-03 13:21:03,949 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primary identifier (attributes) using filter:
2022-08-03 13:21:03,950 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Found 0 shadows (live or dead)
2022-08-03 13:21:03,951 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Shadow object (in repo) corresponding to the resource object (on the resource) was not found. The repo shadow will be created. The resource object:
shadow:null(null)
2022-08-03 13:21:03,951 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowManager): Adding new shadow from resource object:
shadow: (null, ShadowType)
2022-08-03 13:21:03,961 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Attempt to create new repo shadow for shadow:null(null) ended up in conflict, re-trying the search for repo shadow
2022-08-03 13:21:03,962 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primary identifier (attributes) using filter:
2022-08-03 13:21:03,962 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Found 0 shadows (live or dead)
2022-08-03 13:21:03,963 [PROVISIONING] [http-nio-8088-exec-7] TRACE (com.evolveum.midpoint.provisioning.impl.shadows.manager.ShadowFinder): Searching for shadow by primaryIdentifierValue using filter:
2022-08-03 13:21:03,964 [PROVISIONING] [http-nio-8088-exec-7] ERROR (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
com.evolveum.midpoint.util.exception.ObjectAlreadyExistsException: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
2022-08-03 13:21:03,964 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: resource shadow
shadow: (null, ShadowType)
oid=null, version=null
attributes:
objectSid: S-1-5-21-4039012443-3458975651-2582944172-1142
uSNChanged: 1721254
sAMAccountName: grp-staff
whenCreated: 2021-04-20T07:16:16.000Z
cn: grp-staff
member: [ CN=Midpoint Testneun, name: grp-staff
uSNCreated: 76200
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=xxxx,DC=xx
dSCorePropagationData: [ 2021-07-07T07:22:32.000Z, 2021-06-21T07:18:39.000Z, 2021-06-21T06:52:50.000Z, 2021-05-21T10:54:59.000Z, 1601-07-14T22:36:48.000Z ]
dn: CN=grp-staff,OU=xxxx-xxxxx,DC=xxxx,DC=xx
distinguishedName: CN=grp-staff,OU=xxxx-xxxx,DC=xxxx,DC=xx
instanceType: 4
groupType: -2147483646
whenChanged: 2022-07-20T06:55:32.000Z
objectGUID: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
objectClass: {...resource/instance-3}group
exists: true
2022-08-03 13:21:03,970 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: resource shadow: determined primaryIdentifierValue: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
2022-08-03 13:21:03,970 [PROVISIONING] [http-nio-8088-exec-7] DEBUG (com.evolveum.midpoint.provisioning.impl.shadows.ShadowAcquisition): REPO CONFLICT: potential conflicting repo shadow (by primaryIdentifierValue)
shadow: (fe35d3e8-7fc2-481d-9063-e9285fbb6164, v0, ShadowType)
oid=fe35d3e8-7fc2-481d-9063-e9285fbb6164, version=0
name: CN=grp-staff,OU=xxx-xxx,DC=xxxx,DC=xx
resourceRef: oid=412a9b5d-4c88-4a64-a281-2995d46146c8(ResourceType)[default]
synchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
fullSynchronizationTimestamp: 2021-04-20T11:44:06.815+02:00
objectClass: {...resource/instance-3}group
primaryIdentifierValue: 756d6959-9a3a-4f33-a5e4-0c08ef72db37
kind: ENTITLEMENT
intent: group
exists: true
attributes:
dn: [ cn=grp-staff,ou=xxxxxx,dc=xxx,dc=xx (raw) ]
objectGUID: [ 756d6959-9a3a-4f33-a5e4-0c08ef72db37 (raw) ]
2022-08-03 13:21:03,971 [PROVISIONING] [http-nio-8088-exec-7] ERROR (com.evolveum.midpoint.model.impl.ModelObjectResolver): Error resolving object with oid 000425a8-6b34-4cad-83ef-f1e8177f0bf7, expected type was class com.evolveum.midpoint.xml.ns._public.common.common_3.ShadowType..
com.evolveum.midpoint.util.exception.SystemException: Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(756d6959-9a3a-4f33-a5e4-0c08ef72db37, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly via midPoint
Sent: Wednesday, August 3, 2022 10:57 AM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Cc: Pavol Mederly <mederly at evolveum.com><mailto:mederly at evolveum.com>
Subject: Re: [midPoint] duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key"
Hello Hsin-Fang,
there is a troubleshooting piece of code related to that exception:
https://github.com/Evolveum/midpoint/blob/5e841416523a6d46431443d36b056e91ada0b9f1/provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/shadows/ShadowAcquisition.java#L188-L196
You should enable the respective DEBUG logging and midPoint will provide you with more information.
Best regards,
--
Pavol Mederly
Software developer
evolveum.com
On 03/08/2022 10:53, Hsin-Fang Hsu via midPoint wrote:
Hi,
I upgraded midpoint from 4.4.1 (postgresql) to 4.4.2 (postgresql native).
The data was exported and imported into the database using ninja and the content in the new database also seems to be fine.
ADLdapConnector v3.3 is used.
AD test connection works. But when I tried to access any AD account, I got these kind of errors:
com.evolveum.midpoint.util.exception.SystemException: Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key" Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(bc1cd126-81dc-4c2a-acd0-b30eafdc40ef, 11, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
Error resolving object with oid 'eb303210-59dd-499f-bbca-9c57cefa3295': Unexpected repository behavior: object already exists error even after we double-checked shadow uniqueness: Conflicting object already exists, constraint violation message: ERROR: duplicate key value violates unique constraint "m_shadow_primidval_objcls_resrefoid_key" Detail: Key (primaryidentifiervalue, objectclassid, resourcereftargetoid)=(b035b94c-e127-466a-9011-1e4a2dc41fc5, 17, 412a9b5d-4c88-4a64-a281-2995d46146c8) already exists.
I guess the error occurs because midpoint try to save what it reads from AD to a new shadow but that shadow already exists in the database. But the configuration works fine in 4.4.1. How can I solve this?
Thank you very much for your help in advance!
Best regards,
Hsin-Fang
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
https://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
https://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/1c2c727c/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 68653 bytes
Desc: image001.png
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/1c2c727c/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 90727 bytes
Desc: image002.png
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20220803/1c2c727c/attachment-0003.png>
More information about the midPoint
mailing list