[midPoint] Technical inquiries regarding CSV Connector and Shadow Object behavior

[David Klement]

Hi Frost,

1/ For all available information about the CSV connector, please see https://docs.evolveum.com/connectors/connectors/com.evolveum.polygon.connector.csv.CsvConnector/

In short, a standard CSV format is accepted; you can have a look the First steps training - https://evolveum.com/training-and-certification/midpoint-deployment-first-steps/ - and the CSV used there for a reference). Without more details on what it means that "midPoint's UI fails", I cannot provide more pointers as to what may be the issue.

In general, see https://docs.evolveum.com/midpoint/reference/support-4.10/expressions/ for mapping configuration reference. You could also find this whole section useful if you use GUI: https://docs.evolveum.com/midpoint/reference/support-4.10/admin-gui/resource-wizard/ 

2/ To prevent midPoint deleting a focus (user in midPoint) when its projection in CSV is deleted, it is enought to NOT tell midPoint to "delete focus" as a reaction to synchronization situation "deleted". Without being told so, midPoint does not delete anything.

See 

- https://docs.evolveum.com/midpoint/reference/support-4.10/resources/resource-configuration/schema-handling/synchronization/
- https://docs.evolveum.com/midpoint/reference/support-4.10/admin-gui/resource-wizard/object-type/synchronization/
- https://docs.evolveum.com/midpoint/reference/support-4.10/synchronization/

3/ Shadows

See https://docs.evolveum.com/midpoint/architecture/concepts/shadow/ for the conceptual explanation and raison d'etre of shadows.

As for "persistent shadows are causing operational instability", I suggest to review the synchronization and reconciliation rules of your resource in midPoint - from what I gathered from your message, the synchronization rules in your case may be:

- unmatched -> add focus
- unlinked -> link
- linked -> synchronize

The "operational instability" cause by "persistent shadows" may be because if you import user from CSV to midPoint, then delete the record in CSV and leave remain in midPoint (i.e., no synchro rule deleted -> delete focus), and then return the record to the CSV and attempt to reconcile, midPoint sees a conflicting  record (in its DB and has no instructions on what to do with it.

The solution to this is an appropriate correlation rule - see 

- https://docs.evolveum.com/midpoint/reference/support-4.10/correlation/
- https://docs.evolveum.com/midpoint/reference/support-4.10/admin-gui/resource-wizard/object-type/correlation/

You need to find a unique attribute (e.g., employee number) and tell midPoint to match the resource objects with focal objects based on that attribute.

Hope this helps. If you need further help configuring your resources, please provide more details about your configuration and the errors you get (without any sensitive info, of course).

Best regards

David Klement | Technical Writer
Evolveum, s. r. o.
dakle at evolveum.com | www.evolveum.com

----- Original Message -----
From: "midPoint General Discussion" <midpoint at lists.evolveum.com>
To: "midPoint General Discussion" <midpoint at lists.evolveum.com>
Cc: "Frost K" <kfrost3217058 at gmail.com>
Sent: Monday, March 9, 2026 6:18:26 AM
Subject: [midPoint] Technical inquiries regarding CSV Connector and Shadow Object behavior

Hello. Dear midPoint community. 

I would like to raise the following technical inquiries regarding midPoint's behavior and configurations. 


【1. Data Inconsistency in CSV Connector Integration】 

We have utilized the CSV Connector to import user information and display it within the midPoint UI. 

While this integration previously functioned correctly, we have recently confirmed that synchronization between data in a csv file and midPoint's UI fails. 


Request: Could you kindly provide some related documents below? 

・a reference XML model (Resource Configuration ＝CSV connector) 

・recommended CSV formatting guidelines, or a CSV file template, format to sync data into midPoint correctly 

・any technical documentation that ensures robust data mapping between CSV sources and midPoint 


【2. Deletion Protection for User Data via CSV Connector】 

We use a CSV connector to create users on midPoint so that means you can confirm users on a connector are displayed "Linked" and users on UI would be deleted when deleting users on a csv file and invoking a import task. 

Even if we remove user records from a source CSV file, we would like to ensure users in midPoint never be deleted. 
We would like to lower the risks of IDs sudden deletion as much as we can. 


Inquiry: 

・Is there a specific configuration (e.g., synchronization reactions or "Keep" situations) to protect existing users from being deleted after deleting users on a csv file and invoking a import task? 

Would you kindly provide us with documentation explaining the specifications for this scenario. 


【3. Rationale and Management of "Shadow Objects"】 

We have encountered intermittent errors during import tasks and recomputations, caused by "Shadow" objects that exist in the system backend but are not visible in the UI. 

These persistent shadows are causing operational instability. 


Inquiry: 

Would you kindly tell us what the fundamental architectural purpose of Shadow objects in midPoint is? 

While we can utilize tasks to delete shadows, we are concerned that these errors may occur under some conditions. 

Could you kindly provide us with best practices or a comprehensive technical guide on how to prevent Shadow-related errors and maintain system integrity? 


I look forward to your response. 
Best regards. 

Ken 
A member of a company in Japan. 

_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com
https://lists.evolveum.com/mailman/listinfo/midpoint