Java RSA Cryptography: RSA modulus has a small prime factor

Dinura Seneviratne :

Trying to encrypt a message using RSA: I keep getting this error, but Im not sure what it means:

Code to generate the key

    generator = KeyPairGenerator.getInstance("RSA");
    RSAKeyGenParameterSpec kpgSpec = new RSAKeyGenParameterSpec(2048, BigInteger.valueOf(17489));
    generator.initialize(kpgSpec);

    KeyPair keyPair = generator.generateKeyPair();

    publicKey = (RSAPublicKey) keyPair.getPublic();
    privateKey = (RSAPrivateKey) keyPair.getPrivate();

Code to Encrypt Messages

        RSAPublicKeySpec pubKeySpec = new RSAPublicKeySpec(new BigInteger(publicKeyBytes), BigInteger.valueOf(17489));

    Cipher cipher;
    KeyFactory keyFactory = KeyFactory.getInstance("RSA");
    PublicKey currentKey = (RSAPublicKey) keyFactory.generatePublic(pubKeySpec);
    cipher = Cipher.getInstance("RSA");
    cipher.init(Cipher.ENCRYPT_MODE, currentKey);
    byte[] encryptedBytes = cipher.doFinal(data.getBytes());
    encrypted = bytesToString(encryptedBytes);

Error:

W/System.err: java.lang.IllegalArgumentException: RSA modulus has a small prime factor
W/System.err:     at com.android.org.bouncycastle.crypto.params.RSAKeyParameters.validate(RSAKeyParameters.java:46)
    at com.android.org.bouncycastle.crypto.params.RSAKeyParameters.<init>(RSAKeyParameters.java:28)
    at com.android.org.bouncycastle.jcajce.provider.asymmetric.rsa.RSAUtil.generatePublicKeyParameter(RSAUtil.java:44)
    at com.android.org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineInit(CipherSpi.java:288)
    at com.android.org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineInit(CipherSpi.java:406)

kelalaka :

This is a warning against security catastrophe.

In RSA, we expect that the composite modulus n have two prime factors close to sqrt{n}, otherwise if one prime is small then one can easily factor your modulus into n=p q and drive your secret exponentd , BOOM.

Run the key-gen, again. Also, prefer to use a public exponent 3, 5, 17, 257 or 65537. This helps to have faster calculations. You used 17489 which requires 4 multiplication however 65537 requires 2 - not counting the squaring.

Also, you should call

Cipher.getInstance("RSA/ECB/PKCS1Padding");

Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");

not

Cipher.getInstance("RSA");

This is insecure, malleable and has many attacks.

RSA is a trapdoor function. In encryption and signatures, it should never be used without proper padding.

For encryption, PKCS1Padding and OAEP can be used. OAEP is a better choice.
For signature use RSA-PSS

note that RSA Signing is Not RSA Decryption.

also note that: Actually, it is not preferred to use RSA for encryption, in general, RSA is used for signatures. We combine symmetric and asymmetric encryption schemes in Hybrid-Cryptosystem. For example, one can use Diffie-Hellman Key exchange to establish a common key with forward secrecy so that the data can be encrypted with a symmetric encryption algorithm like AES which is much faster than any asymmetric cryptosystem. There is, also, Key Encapsulation Mechanism (KEM) and it applicable to RSA, known as RSA-KEM which can be used to establish a key.