RSA: Fix buffer overflow in PSS signature verification
Fix buffer overflow in RSA-PSS signature verification when the hash is
too large for the key size. Found by Seth Terashima, Qualcomm.
Added a non-regression test and a positive test with the smallest
permitted key size for a SHA-512 hash.
diff --git a/tests/data_files/rsa528.key b/tests/data_files/rsa528.key
new file mode 100644
index 0000000..fd463b5
--- /dev/null
+++ b/tests/data_files/rsa528.key
@@ -0,0 +1,9 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIBRQIBAAJDAOMcJG1GSFmEJh/RdMqz1DVzRGAuzXk8R9vlQlLTe7NQvGNDWbGV
+FVQggORySktnIpG+V8dkj1Finq7yNOhH2ZzGXwIDAQABAkMAsWYyLglQSlwnS4NZ
+L1z4zieTqW3lomWr2+BgxkHbxl2w0Rx4L+Ezp+YK6mhtIQWNkoytPvWJJMS7Jrkg
+agMAHQJBAiIA+F1y5GO0Bv+igsNLXwwtbCqs8hAkavU9W8egt/oDbhzbAiIA6hds
+PZp/s1X7n7dwfmebSs+3vLZFuQfifN8XZLw0CXHNAiEuEzgDQrPdMIN3er96zImI
+rYoUBgabiQ9u/WPFfa4xOU0CIgDDYC089Tfjy72pPgcr2PkpZVhqro5esg/8PI5f
+yxx7TXkCIgCYoE8Y5IxomtL1ub1AQzPe9UyyUGzQB1yWeiloJh6LjxA=
+-----END RSA PRIVATE KEY-----