[midPoint] CSV Import

[Pavol Mederly]

Hello Anand,

the problem is that CSV connector cannot update the file while it is 
being read.

And the midPoint import works like this: it opens a "iterative search" 
operation, and while receives individual accounts, it immediately 
imports them. The "iterative search" operation finishes only after all 
accounts are imported.

This wouldn't pose a problem in itself. However, it is almost sure that 
in your case, you also do an *update* of the accounts during import. For 
example, our CSV file sample updates account password, because of the 
mappings:

                 <credentials>
                     <password>
<outbound>
                             <expression>
                                 <asIs/>
                             </expression>
                         </outbound>
                         <inbound>
<strength>weak</strength>
                             <expression>
                                 <generate/>
                             </expression>
                         </inbound>
                         <passwordPolicyRef 
oid="81818181-76e0-59e2-8888-3d4f02d3ffff" type="c:ValuePolicyType"/>
                     </password>
                 </credentials>

I.e. during import, a password is generated (via inbound mapping), and 
then pushed back onto CSV resource (via outbound mapping).

The solution is simple - if you want to use CSV file as a pure data 
source, just delete all outbound mappings from the configuration. If you 
want to distribute changes also in the direction midPoint->CSV, leave 
only these mappings that do not cause updating the file during import.

You might be curious why the 100-entries limit. It is because of the 
buffering built into ConnId framework. (Maybe turning off the buffering 
could also help, but I haven't tried.)

Best regards,
Pavol

On 17. 3. 2015 9:39, Anand Kothekar wrote:
> Hi,
>
>
> I was working with CSV file where I tried to import users. I was able 
> to import approximately 100 users from CSV file at a time.I actually 
> have requirement where I need to import thousands of users through CSV.
>
> When I try to import more than 100 users, the task simply keeps 
> running. This time it has "DefaultNode " value under "Executing at" 
> attribute which might be because of clustering to handle workload.
>
> After waiting for near about an hour it gives stalled error.
>
> please suggest me some way which will import users smoothly
>
>
> Regards,
> Anand
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20150318/f19ff753/attachment.htm>