Encryption and digital signature with openssl

The asymmetric encryption is one of the security advances that makes the internet connections and online transactions trustable. Without this trust, nobody are willing to do business or perform any purchase on any e-commerce no matter how fast and convenient it could be.

This type of encryption alongside with the symmetric one provides to the internet the security properties like confidentiality, integrity and accountability. There is another security property called availability but it is not something that any encryption normally provides.

How asymmetric encryption works?

Let’s suppose that Alice wants to communicate with Bob through the internet by using an asymmetric crypto system.

To do so, she will generate the asymmetric keys, normally known as key pair, keeping secret the one called private key and distributing the public key to Bob or any communicating party that wants to send any information securely to Alice.

Next, Bob will use Alice public key and encrypt the information that he wants to share with her and send it encrypted to Alice. This encrypted message it is only possible be decrypted by the private key that Alice has kept secretly.

Then Alice, after receiving the encrypted message that Bob sent to her, she will use her private key to decrypt the message in order to view or further processing it.

As long as the private key was kept unique and secure by Alice and the appropriate algorithm has been used with a suitable key size, the confidentiality of the communication is protected.

This is one of the use cases that asymmetric encryption provides for secure communications but, there’s another usage called digital signature which covers the accountability or non-repudiation security property.

The digital signature leverages the same key pair but, this time the private key (Alice) is the one that will encrypt the information and then sent it to the communicating party (Bob). After that, Bob will use Alice public key and perform the decryption of the message for further purposes.

You may wonder why would Alice encrypt a message that anybody, who is in possession of her public key could decrypt it and view it. Well, the main reason is that the digital signature is not meant for protecting the secrecy of a message but for verifying that the message came from the unique individual that possess the private key (Alice) thus, guaranteeing the accountability or non-repudiation of the communication.

On the other hand, the digital signature normally is not calculated on the original message but rather on its “message digest” or a “hash”. A hash or a message digest it is a fixed length/size value that results from applying a hash algorithm on the original message. The length of the hash depends on the algorithm used, and it is normally smaller than the original message used for calculating the hash value. Moreover, it is not computationally possible to find two messages with the same hash value, meaning that any slight change on the original document it will be reflected on the hash with a different outcome.

But why is the message digest encrypted instead of the original message? the reason resides in providing the integrity protection to the message.

Back to the use case described previously where this time Alice digitally signs the hash of the document with her private key, once Bob receives the signed message, he proceeds to verify the signature with Alice public key and, if it is correct, he will use the hash obtained from this verification and then compares it with the hash value calculated from the original message received. If the values are exactly the same then, the message was not tampered, hence the integrity of the communication is verified.

Therefore, if both Alice and Bob exchanges their public keys without sharing their private keys, Alice is able to communicate with Bob securely preserving the confidentiality, integrity and accountability by using Bob’s public key to encrypt the message and digitally sign it with her private key. On the other side, Bob is able to do the same but combining his private key and Alice’s public key.

Before jumping to the next sections, it is important to mention that in the real world, the asymmetric encryption (public key encryption and digital signature) has a poor performance and it is only recommended for small messages not bigger than the asymmetric key length. That’s why the encryption normally used to protect the data is the symmetric since it is way more fast, and the asymmetric is only used for encrypting the symmetric key.

Asymmetric encryption and decryption with RSA key pair

First, generate the RSA key pair:

$ openssl genrsa -out priv_rsa.pem 4096
$ openssl rsa -in priv_rsa.pem -pubout -out pub_rsa.pem

Afterwards, encrypt a small message with the RSA public key:

$ echo "Hello World" > message
$ openssl rsautl -encrypt -in message -out encrypt_rsa -inkey pub_rsa.pem -pubin

Finally, to retrieve back the message, decrypt it with the RSA private key:

$ openssl rsautl -decrypt -in encrypt_rsa -inkey priv_rsa.pem -out decrypted_rsa

As mentioned in the previous section, this purely asymmetric encryption is possible if the message to be encrypted is small. In the following example will show you that a bigger message is not possible to be encrypted with only asymmetric keys:

$ seq 500000 > big_message
$ ls -lh big_message
$ openssl rsautl -encrypt -in big_message -out big_encrypt_rsa -inkey pub_rsa.pem -pubin

As you may see in the above picture, openssl fails when trying to encrypt with the public key a bigger message.

Combining symmetric and asymmetric encryption: The Hybrid Encryption

So in a real world scenario, in order to overcome the low performance of the asymmetric encryption, what actually is done is:

1. Generate a symmetric key.
2. Encrypt the message with the symmetric key.
3. Encrypt the symmetric key with the public key of the communicating party to who you want to send the message securely.

Suppose that both Alice and Bob already shared their public key with each other and Bob wants to share a file with Alice.

Therefore, Bob will proceed to perform the step 1, creating and saving the key in environment variable (in memory) instead of leaving traces at rest which is more secure, and the IV of 128 bits because it is the block size of AES, the algorithm that will be used for encrypting the message:

$ export SYMMETRIC_KEY=$(openssl rand -hex 32)
$ openssl rand -hex 16 > iv

Next, after generating a 256 bits and 128 bits of key and IV respectively, Bob will encrypt the message with the symmetric key recently created using the following commands:

$ echo "hello alice" > message
$ openssl enc -aes-256-cbc -K $SYMMETRIC_KEY -iv $(cat iv) -in message -out message.enc

After that it will generate the encrypted symmetric key by using Alice public key:

$ echo $SYMMETRIC_KEY | openssl rsautl -encrypt -out symmetric_key.enc -inkey alice_pub.pem -pubin

Subsequently, Bob will send the encrypted message (message.enc), the encrypted symmetric key (symmetric_key.enc) and the IV to Alice who will:

1. Decrypt the symmetric key with her private key
2. Decrypt the message with the decrypted symmetric key and the IV

Alice commands to decrypt the symmetric key with her private key, and saving it in an environment variable to avoid leaving traces in disk:

export SYMMETRIC_KEY_DECRYPTED=$(openssl rsautl -decrypt -in symmetric_key.enc -inkey alice_priv.pem)

Afterwards, she proceeds to decrypt the message with the decrypted symmetric key obtained:

$ openssl enc -d -aes-256-cbc -K $SYMMETRIC_KEY_DECRYPTED -iv $(cat iv) -in message.enc -out message.decrypted

Finally, she obtains the message in clear text that Bob send to her for further processing. Note that the above picture shows the message decrypted obtained, and the sha1sum comparison of both clear text message to verify that it is exactly the same.

The Hybrid Encryption with digital signature

The previous process can be improved if we add the digital signature on the encrypted message to be sent in order to add integrity and non-repudiation. Thus, assuming that both Alice and Bob has exchanged their public keys between them, the process should be:

1. Generate a symmetric key.
2. Encrypt the message with the symmetric key.
3. Encrypt the symmetric key with the public key of the communicating party to who you want to send the message securely.
4. Digitally sign both the encrypted symmetric key and the encrypted message with the sender’s private key

The steps 1 to 3 are the same from the previous section so let’s show how to do the step 4:

$ cat message.enc symmetric_key.enc | openssl dgst -sha256 -sign bob_priv.pem -out signature

The above command will calculate the sha256 of both encrypted message and symmetric key and digitally sign the sha256 value with Bob’s private key. This signature alongside with the encrypted message, the encrypted symmetric key and the IV will be sent to Alice. If a third party manipulates any of those files, it will invalidate the reverting process that Alice will perform and then she will stop from further processing. This process consist of:

1. Verify the signature with Bob’s public key
2. Decrypt the symmetric key Alice’s private key
3. Decrypt the message with the decrypted symmetric key and the IV

If the following command (to cover step 1) passes then Alice will continue with the steps 2 and 3 which are the same as described in previous section:

$ cat message.enc symmetric_key.enc | openssl dgst -sha256 -verify bob_pub.pem -signature signature

However, if the encrypted message was tampered, then the verification will fail and Alice will stop processing the data received. For this example, the encrypted message will be slightly changed by appending an “a” character for simulating the tampering:

$ echo "a" >> message.enc.tampered
$ cat message.enc.tampered symmetric_key.enc | openssl dgst -sha256 -verify bob_pub.pem -signature signature
Verification Failure

Note that if the order of the cat command in this case matters and if it is different from the signature generation it will fail as well. Bear in mind that in this examples are missing some standards, and it shown this way for you to have a better understanding of how the symmetric and asymmetric encryption works.

Nevertheless, you might want to check the OpenPGP and s/MIME standards that makes this whole encryption and decryption process easier.