How to remove sensitive data from your winston logs?
In this article I'm going to show you how you can remove sensitive information from winston logs.
You’ve written your application, it’s all done and ready to be shipped so what happens next? Your job is not over yet, not even close. Now comes the monitoring, debugging and troubleshooting part. You will have a chance now to see how your masterpiece behaves in real world. We need tools to get feedback about the performance and functioning of our code. One of the many tools we can use to get an insight into our app’s life is by creating logs about different kinds of events happening inside our app.
Sometimes we might log information that are private and confidential or just should not be seen by any employee. Chances are you have already seen data you thought it didn’t belong there. What can we do? We can either check the inputs of the logs which can be a bit tedious since for example if we log a request body object, you know, many things which we’ve never thought about while developing can be there like password or other credentials. We might log something we were not aware that it could contain some sensitive data. In order to try to prevent this we can leverage winston’s feature of creating custom outputs. In this article I’m going to show you how I removed sensitive data from our logs to prevent misusing it by anyone who has access to the logs.
Let’s create a Node.js app so you can also follow me along. Create a folder and cd into it; then initialize an npm project.
We’re going to make a very basic server first. We’re going to create 2 endpoints: /register where the user sends us their username, password and name. We receive this data in body and as you can see it’ll contain sensitive information. The other endpoint is /orders where the user can fetch their orders. We assume the app uses JWT tokens for authentication so in the headers we expect to get some sensitive data too.
Let’s install the required dependencies
First we're going to set winston up; then create a basic logger. Create _logger.js_ file.
Open it with your favourite editor and put this code there.
Then create an index.js file and open it up. Copy this code which sets up a simple server:
Start the server.
And if all is good we should see this log message.
Now let’s add the register endpoint. Nothing special here, data comes in the body, we log that a new user’s just registered and return a 200 and a descriptive message. Let’s put this code under the body parser part.
After restarting the server we can try our app using cURL (or any other tool like Postman).
This is the log we should be seeing now.
We can see that the password got logged and anyone could take advantage of this who has access to the logs. Let’s create the /order endpoint to allow the user to see their made up orders. Paste this code under the post endpoint.
I’m using the Bearer authorization format here and let’s say bigsecret is a valid token. If we log the headers here, we log the user’s token which can lead to misuse. Let’s hit this endpoint too after restarting our server.
The expected log message now is the line below which contains our token.
So far, we’ve seen two examples on how logging request related data also logs private data. Let’s look at now how we can customize the output format of the logs. Now we’ll go back to the logger.js file and add some new lines. Using the printf method of the format object we can customize the output format. I’m adding the level and the stringified message. It’s still very similar to the basic format. This is how the logger.js should look now.
Now we can implement a procedure that basically checks the properties of this message object recursively and remove all the keys, or at least modifies the content of the keys which we think can contain sensitive information. For now, in our app we’ll have two such keys: Authorization and password. So we want to modify the content of them. Let’s create an array and store them. And we’ll create a function that does the replacement for us. It’ll need a dependency so we need to require it at the beginning of the file. This code should now stand before the createLogger call.
As you can see. I go through the properties of the object. If it’s an array I call the function for each member. When there’s a key which has been added to our excluded keys array then I modify the value so I still know it was there but the value can no longer be seen. Let’s call this function before stringifying the object. The print function will first remove all sensitive data and only then returns with the stringified log.
If we call our endpoints again, now we should see that the password and our very secret token is gone. We could further customize our winston logger but that should be another blog post. The log when we the /register endpoint looks like this. We can see that the password key’s value got replaced. Try the other endpoint too and see what happened.
You can find the full source code on my GitHub
This is the end of my first blog post. I hope you’ve founded it useful, if so, leave a comment and subscribe to my newsletter so you’ll be notified if I post a new article.#nodejs #winston #logging