Just checked it. Indeed its possible to delete clients. No matter if they have open invoices or quotes. Let us evaluate how to fix this. Clients which do have payed anything or have any outstanding quotes should not be deletable.
Just clients you have never interacted with should be deletable. At least thats what in germany the goverment wants. If you have contacted anyone regarding a quote or invoice you are also allowed to save its data in your system and thats what you should do to proof all of your finances.
As this may vary from country to country we should probably add another option to the
ipconfig.php file like:
# Enable the deletion of invoices
# Enable the deletion of clients
Let us talk about this and wait if other people also need this feature.