system carries log of imports so in case of failure log can be used to ignore all already imported items
we have had 3 to 4 failures of this system, the system is brilliant but if you end up importing all the duplicates it is a NIGHTMARE to fix, suggestion in the "my idea" would fix that and very easy to do....