Omnindex Journal 25/jan - Encrypted Ecto
This is a journal I write while I build Omnindex, a search app, and post here, without much editing, just writing what comes to mind
Okay so how can I make my Ecto columns encrypted? I could always manually encrypt before saving but that's very error prone, I'd like something at the ecto level. Fortunately I found about this Ecto.Type, which allows me to create custom type and it's the approach that cloak_ecto takes to encrypt fields. Unfortunately for my case I don't have a global encryption key, but one per account, which is held in a different table, and unfortunately from an Ecto.Type specification I won't have access to this.
So one solution I though for solving it would be to have a type that is actually a tuple of `{public_key, text}`, so we could always encrypt before saving to the database, and throw an error in case I ever forget to pass the public_key. Unfortunately, I don't have access to the public_key when retrieving the column from the DB on the `load` method, so I will have to manually decrypt it later, but I guess that's okay, there is no long-term danger in forgetting to decrypt, it will just throw an exception or look weird to the user, and I can fix it.
I tried it out on live book, implementing it like this:
Seems like it will work, let's write some tests to check
So I'm saving a document and expecting that when I retrieve it back, the content will be encrypted, and I can decrypt it to get the same result back. I already had a save_and_index function for saving documents, so I just added a public_key parameter to it. The test fails as it should:
Then I implement it, creating a EncryptedColumn type:
Turns out I noticed I don't need to encrypt during cast, only during dump, and I use it on my field:
It works like a charm!
This breaks some tests because I require to have a public key now, I'm going to fix that and also make title, url and extra_data columns encrypted.
Okay unit tests for Documents are passing, but a bunch of other tests are failing now, specially the crawlers which need to pass a public_key now, and I have nowhere to take it from. Thinking about it, maybe I should have implemented the part of the public/private key of the account first then using it to encrypt documents already. So let's do it, this is how my Account schema looks like right now:
I had created this `encryption_key` because I was not considering the public/private pair before, so I will write a migration to drop that and instead have two columns
Then I write a test to see that a public/private pair is generated when user registers. I don't know exactly how to write the assertion yet, so I'll just do this:
And it fails as it should
Okay so I'll just create a small function which I will pipe on the changeset, let's see if that works…
Cool, the test passes, all green. Now let me generate the proper thing…
It won't work will it? Public and private keys are not strings, this will probably fail
Et voilà , it failed. Hmmmm there is this ExPublicKey.pem_encode available…. Hmmmm I can use the same trick with Ecto.Types as before can't I? Let's create a column type that will automatically encode and decode it for us then!
There you gooo
Now this is my schema
Oh noes
Gosh, I need to update my migration to use text I guess. Changed, migrated, success!!!
Or almost, but it's doing what I want, I can just check for the types on the assertions and…
Done!
This test is good enough, the implementation is good enough, let's commit and move on!
Now I can fix the other tests that were failing, for example, here is the code of my WebCrawler
On line 39, I now need to not only pass the title and content but also the public key. I didn't need to query the Account here before but now I will. I created a small helper function to query the public_key and changed that smaller :ok block as such:
Now my test is failing for a different reason:
Aháaa, it's encrypted! To make the test pass again I created a small helper that finds the private key in the account and decrypts the document
Tests are green again :)