Reject function in onCustomValidation event not working

Try to remove the ( ) from the return statement, try returning an object.

return {
 'status': true,
 'iban': IBAN.electronicFormat(iban),
 'err':null
};