×
Simon Abrams

Redact Elasticsearch Passwords from Microsoft Azure Application Insights Using C#

Written by Ken Dale
3
This post is days old.

We use Microsoft Azure Application Insights in our web applications. It logs tons of useful data about our applications, including web requests and requests to dependencies like databases. We also use Elasticsearch communicating over HTTPS. When an Elasticsearch url contains a password (https://user:[email protected]) we found it is logged to Application Insights in cleartext.

That said, here’s how to redact the password from any urls containing passwords:

Code

Here’s an ITelemetryProcessor implementation that redacts passwords from HTTP and HTTPS urls.

using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
using System.Text.RegularExpressions;

namespace WebApplication
{
    public class RemoveHttpUrlPasswordsTelemetry : ITelemetryProcessor
    {
        private static readonly Regex removePasswordRegex =
            new Regex(@"http(s)?:\/\/.+:(?<password>.+)@", RegexOptions.IgnoreCase | RegexOptions.Compiled);

        private readonly ITelemetryProcessor next;

        public RemoveHttpUrlPasswordsTelemetry(ITelemetryProcessor next)
        {
            this.next = next;
        }

        public void Process(ITelemetry item)
        {
            var request = item as DependencyTelemetry;

            if (request != null && request.Type == "Http")
            {
#pragma warning disable CS0618 // Type or member is obsolete
                request.CommandName = RemovePasswordFromUrl(request.CommandName);
#pragma warning restore CS0618 // Type or member is obsolete
                request.Data = RemovePasswordFromUrl(request.Data);
            }

            next.Process(item);
        }

        private static string RemovePasswordFromUrl(string url)
        {
            var match = removePasswordRegex.Match(url).Groups["password"];

            if (match.Success)
            {
                url = url.Replace(match.Value, "REDACTED");
            }

            return url;
        }
    }
}

Next, use the RemoveHttpUrlPasswordsTelemetry class with Application Insights. There’s ASP.NET and ASP.NET Core examples at https://docs.microsoft.com/en-us/azure/azure-monitor/app/api-filtering-sampling#filtering-itelemetryprocessor.

This can help raise your security by not storing passwords in logs!

Ken Dale
Ken Dale
Senior Application Developer (Former)
.NET
Suggested reading

Elasticsearch With NEST Using C# nameof Not Working

.NET
Imagine you’re using Elasticsearch with your strongly typed C# models and nam...
Read now

How to avoid NullReferenceException in C#

.NET
I have been working as a software developer for almost three years, the most ...
Read now

Building ASP.NET Core 3.1 Apps To Organizational Standards Using Extension Methods

.NET
We’ve been working through upgrading our core applications from ASP.NET full ...
Read now

What Works and What Doesn't With ConfigurationManager.AppSettings Using ASP.NET Core 3.1

.NET
ConfigurationManager has long been used by .NET Framework developers prior to...
Read now

Regex Performance With and Without RegexOptions.Compiled Using .NET Framework 4.8 and .NET Core 3.1 (December 2019)

.NET
We’ve had some internal discussion around the usage of RegexOptions.Compiled ...
Read now

Logging ASP.NET Web API 2 Failed Request Body to Application Insights

.NET
Debugging web applications can be difficult sometimes. When debugging a faile...
Read now

Comments

Stay fresh! The brighter green, the fresher the article!