×
Simon Abrams

Redact Elasticsearch Passwords from Microsoft Azure Application Insights Using C#

Written by Ken Dale
3
This post is days old.

We use Microsoft Azure Application Insights in our web applications. It logs tons of useful data about our applications, including web requests and requests to dependencies like databases. We also use Elasticsearch communicating over HTTPS. When an Elasticsearch url contains a password (https://user:[email protected]) we found it is logged to Application Insights in cleartext.

That said, here’s how to redact the password from any urls containing passwords:

Code

Here’s an ITelemetryProcessor implementation that redacts passwords from HTTP and HTTPS urls.

using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
using System.Text.RegularExpressions;

namespace WebApplication
{
    public class RemoveHttpUrlPasswordsTelemetry : ITelemetryProcessor
    {
        private static readonly Regex removePasswordRegex =
            new Regex(@"http(s)?:\/\/.+:(?<password>.+)@", RegexOptions.IgnoreCase | RegexOptions.Compiled);

        private readonly ITelemetryProcessor next;

        public RemoveHttpUrlPasswordsTelemetry(ITelemetryProcessor next)
        {
            this.next = next;
        }

        public void Process(ITelemetry item)
        {
            var request = item as DependencyTelemetry;

            if (request != null && request.Type == "Http")
            {
#pragma warning disable CS0618 // Type or member is obsolete
                request.CommandName = RemovePasswordFromUrl(request.CommandName);
#pragma warning restore CS0618 // Type or member is obsolete
                request.Data = RemovePasswordFromUrl(request.Data);
            }

            next.Process(item);
        }

        private static string RemovePasswordFromUrl(string url)
        {
            var match = removePasswordRegex.Match(url).Groups["password"];

            if (match.Success)
            {
                url = url.Replace(match.Value, "REDACTED");
            }

            return url;
        }
    }
}

Next, use the RemoveHttpUrlPasswordsTelemetry class with Application Insights. There’s ASP.NET and ASP.NET Core examples at https://docs.microsoft.com/en-us/azure/azure-monitor/app/api-filtering-sampling#filtering-itelemetryprocessor.

This can help raise your security by not storing passwords in logs!

Ken Dale
Ken Dale
Senior Application Developer
.NET
Suggested reading

Upcoming Google Chrome SameSite Change Breaks Our IdentityServer3 Azure Active Directory Login (and, how to fix it!)

.NET
According to https://www.chromestatus.com/feature/5088147346030592 at the tim...
Read now

A Case Of Newtonsoft.Json, TypeNameHandling.All, And JsonSerializationException

.NET
Our RimDev.FeatureFlags library uses Newtonsoft.Json as part of roundtripping...
Read now

Avoiding a LINQ FirstOrDefault mishap

.NET
.NET’s LINQ library has extension-methods that will return a default-value if...
Read now

Retry Transient Failures Using SqlClient / ADO.NET With Polly

.NET
Our team maintains a private package that contains our strategy for database ...
Read now

Avoiding ASP.Net Core Configuration Pitfalls With Array Values

.NET
ASP.NET Core continues to improve on the legacy of the .NET Framework. Our te...
Read now

Filter Hangfire Requests from Microsoft Azure Application Insights

.NET
We use Microsoft Azure Application Insights in our web applications. It logs ...
Read now

Comments

Stay fresh! The brighter green, the fresher the article!