Are you perhaps trying to add a self-signed certificate? Note that security.pki.certificateFiles
expects CA, not server certificates.
If it can be of help, here’s a test that checks a custom CA in a few browsers. You can see the result that shows it’s working as expected.