I have found a workaround that seems to work fine (would be great to be added to the official version eventually).
I found out that the tax amount are being stored in the database with only 2 decimal places (data type: DECIMAL(5,2)). I have updated the data type to DECIMAL(5,3) but the application is still displaying only 2 decimal places.
So I went in
and changed the following line under the format_amount function:
return number_format($amount, ($decimal_point) ? 2 : 0, $decimal_point, $thousands_separator);
return number_format($amount, ($decimal_point) ? 3 : 0, $decimal_point, $thousands_separator);
Once I did that, I went back in the Tax settings and set the tax rate to 9.975 and the value remained exactly as I typed it.
I have since created a few test invoices and confirmed that the right amount is being calculated.