<div dir="ltr">Thanks for the ideas!<div><br></div><div>I was able to create a couple of new processes in my talend job, so basically instead of what I was doing, checking connection > database query > write file , I am now checking connection > database query > write temp file in temp location > get total enrollment count from external source > analyze temp file to match enrollment count to row count in temp csv > if correct > send to folder where midpoint syncs > else > error, send email alert and do not copy file.</div><div><br></div><div>I feel much more confident that it will not send over an empty file now. Awesome</div><div><br></div><div>JASON</div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">JASON</div></div></div>
<br><div class="gmail_quote">On Thu, Nov 12, 2015 at 12:48 AM, Petr Gašparík <span dir="ltr"><<a href="mailto:petr@gasparik.cz" target="_blank">petr@gasparik.cz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Also, you can incorporate some logic like If there is more than 100 changes, wait for manual confirmation of changes. </p>
<p dir="ltr">P. </p>
<br><div class="gmail_quote"><div dir="ltr">Dne st 11. 11. 2015 15:36 uživatel Jason Everling <<a href="mailto:jeverling@bshp.edu" target="_blank">jeverling@bshp.edu</a>> napsal:<br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I didn't actually think to run anything on the file itself or check total count before transfer to file on source system. I am pretty sure I could write something up in Talend to check the source and count before processing any further.<div><br></div><div>Thanks!</div><div>JASON</div></div><div class="gmail_extra"></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 11, 2015 at 5:11 AM, <span dir="ltr"><<a href="mailto:midpoint@mybtinternet.com" target="_blank">midpoint@mybtinternet.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Jason,<br><br> I've had to solve these kinds of problems for various projects in the past, admittedly<br> not using midPoint. There are also other things that could be happening that cause<br> similar issues; e.g. in-flight / broken transfers etc.<br><br> Solving the problem with midPoint would imply having some sort of preview or a running<br> count of changes vs non-changes, or validation in the connector. Not aware of anything<br> like this ...<br><br> Having a system with some decent scripting can help. Some strategies I used in the past:<br><ul><li>Check that modification time on target is at least # mins ago; e.g. not in-flight transfer<br></li><li>Generate a checksum on the source and validate that on the target (md5sum, sha1sum, ...)</li><li>A status line at the end; e.g. end of feed, a count, checksum, etc<br></li></ul><br> You could, if using (UNIX / Linux) do a grep -v on header - check result for zero size.<br> Incoming transmission write to a filename different from what your resource expects; validation<br> copies / renames to resource input name is validation ok.<br><br>Regards,<br> Anton<br><br><blockquote style="margin-right:0px;margin-left:15px">----Original message----<br>From : <a href="mailto:jeverling@bshp.edu" target="_blank">jeverling@bshp.edu</a><br>Date : 05/11/2015 - 18:16 (GMT)<br>To : <a href="mailto:midpoint@lists.evolveum.com" target="_blank">midpoint@lists.evolveum.com</a><br>Subject : [midPoint] Any type of fail safe tricks or settings?<div><div><br><br><div dir="ltr">I was thinking about this today..<div><br></div><div>What if one day my CSV file sends out a file that is blank with just the headers, like if something just goes wrong with the connection or base query. Since my CSV resource is authoritative for who should be enabled or disabled, it would disable everyone because they are missing from the file.</div><div><br></div><div>I don't think that could happen, I wrote in as many fail safes as I could into the Talend job that creates the CSV but....</div><div><br></div><div>Is there any type of mechanism that I can set that says if accounts to be changed is more than {n} then stop or suspend task?</div><div><br></div><div>Just thinking!</div><div><br></div><div>JASON
</div></div>
<br>
</div></div><font size="2"><br><br>CONFIDENTIALITY NOTICE:<br>This e-mail together with any attachments is proprietary and confidential; intended for only the recipient(s) named above and may contain information that is privileged. You should not retain, copy or use this e-mail or any attachments for any purpose, or disclose all or any part of the contents to any person. Any views or opinions expressed in this e-mail are those of the author and do not represent those of the Baptist School of Health Professions. If you have received this e-mail in error, or are not the named recipient(s), you are hereby notified that any review, dissemination, distribution or copying of this communication is prohibited by the sender and to do so might constitute a violation of the Electronic Communications Privacy Act, 18 U.S.C. section 2510-2521. Please immediately notify the sender and delete this e-mail and any attachments from your computer. </font><br><br></blockquote><br><p></p><br>_______________________________________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mailman/listinfo/midpoint</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div></div><div class="gmail_extra">-- <br><div><div dir="ltr">JASON</div></div>
</div>
<br>
<font size="2"><br><br>CONFIDENTIALITY NOTICE:<br>This e-mail together with any attachments is proprietary and confidential; intended for only the recipient(s) named above and may contain information that is privileged. You should not retain, copy or use this e-mail or any attachments for any purpose, or disclose all or any part of the contents to any person. Any views or opinions expressed in this e-mail are those of the author and do not represent those of the Baptist School of Health Professions. If you have received this e-mail in error, or are not the named recipient(s), you are hereby notified that any review, dissemination, distribution or copying of this communication is prohibited by the sender and to do so might constitute a violation of the Electronic Communications Privacy Act, 18 U.S.C. section 2510-2521. Please immediately notify the sender and delete this e-mail and any attachments from your computer. </font><br>_______________________________________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com" target="_blank">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mailman/listinfo/midpoint</a><br>
</blockquote></div></div></div><span class="HOEnZb"><font color="#888888"><div dir="ltr">-- <br></div><div dir="ltr">--<div>Petr G.</div></div>
</font></span><br>_______________________________________________<br>
midPoint mailing list<br>
<a href="mailto:midPoint@lists.evolveum.com">midPoint@lists.evolveum.com</a><br>
<a href="http://lists.evolveum.com/mailman/listinfo/midpoint" rel="noreferrer" target="_blank">http://lists.evolveum.com/mailman/listinfo/midpoint</a><br>
<br></blockquote></div><br></div>
<br>
<font size="2"><br><br>CONFIDENTIALITY NOTICE:<br>This e-mail together with any attachments is proprietary and confidential; intended for only the recipient(s) named above and may contain information that is privileged. You should not retain, copy or use this e-mail or any attachments for any purpose, or disclose all or any part of the contents to any person. Any views or opinions expressed in this e-mail are those of the author and do not represent those of the Baptist School of Health Professions. If you have received this e-mail in error, or are not the named recipient(s), you are hereby notified that any review, dissemination, distribution or copying of this communication is prohibited by the sender and to do so might constitute a violation of the Electronic Communications Privacy Act, 18 U.S.C. section 2510-2521. Please immediately notify the sender and delete this e-mail and any attachments from your computer. </font><br>