When creating a new invoice and entering a new contact name that doesn’t exist yet, it should ask for an email address at minimum.
Yes, the actual handling of new contacts while creating a new invoice is not the best. We’re looking forward to enhance this soon.
Maybe an option to change address details directly from the invoice.