blog post image
Andrew Lock avatar

Andrew Lock

~8 min read

Using named pipes with ASP.NET Core and HttpClient

Share on:

In this post I describe Windows named pipes, what they are, the scenarios where they're useful, how to use them with ASP.NET Core, and how to call a named pipe ASP.NET Core app using HttpClient.

What are Windows named pipes?

Windows named pipes provide a named, one-way or duplex pipe for communication between a client and a server. They provide a way to communicate between different processes, typically on the same machine.

Named pipes support calling remote servers by network name too, though that seems less common from what I understand.

You can think of a duplex named pipe as being similar to a TCP/IP connection, in that you can send data to the server and can receive a response. Named pipes are a Windows kernel feature, so they're not specific to .NET, although they have been accessible from .NET since .NET Framework 3.5 (to a greater or lesser extent).

Named pipes each have a unique name that looks something like this:

\\<ServerName>\pipe\<PipeName>

where <PipeName> is the name of the pipe, which is up-to 256 characters long, can contain any character except \, and is not case-sensitive. If you're using a named pipe on the same server (the typical usage), then you must use . for the server name. For example:

\\.\pipe\my-pipe-name

Note that ASP.NET Core only support local named-pipes, so the server name will always be ..

Named pipes have various modes, and types, so you can create named pipes that are write-only for example (think UDP equivalent). But duplex is probably more common, in which clients can both send requests and receive responses.

There's a whole load of additional modes and configuration for named pipes but I'm going to gloss over those in this post, mostly because I don't fully understand them 😅 But one final option to be aware of is that you need to explicitly opt-in to asynchronous operations for named pipes, otherwise the pipe will be synchronous only.

Why use named pipes?

Named pipes allow for inter-process communication, and are primarily geared towards communication on a single machine. So why would you choose them over TCP/IP for example, where you could use the loopback address (localhost) for single-server communication?

There are a few reasons you might choose named pipes:

The first two points are very Windows-specific. If that's something you need, then it may be easier to use named pipes to get these features, though I suspect there's not a massive need for that these days.

Impersonation in particular is a feature which sounds great on the surface, but which also kind of sounds like a privilege escalation issue waiting to happen. Luckily, the server has to explicitly choose to impersonate a client, and the client can prevent this by setting a flag when it connects.

If you need the security behaviour of named pipes, then that's a clear reason for choosing them. But we actually use named pipes in the Datadog .NET Azure App Service (AAS) Extension, and it has nothing to do with the security features.

In the AAS extension, we instrument your app using the .NET profiling APIs using the Datadog client library, enabling automatic instrumentation that generate traces. In addition, we also run:

  • trace-agent.exe which is responsible for receiving the traces, processing them, and forwarding them to Datadog's backend.
  • dogstats.exe which receives metrics generated by your application, aggregates them, and forwards them to Datadog's backend.

Due to limitations in the AAS extension API, we create those processes as children of your app's process. But an AAS host may have multiple apps we want to trace, yet we only want to run a single instance of trace-agent.exe. Unfortunately, in the AAS extension environment, processes cannot communicate with TCP ports owned by another process tree, so this design doesn't work 🙁

Take my assertions about how AAS works above with a pinch of salt—this design was mostly implemented before I joined Datadog, so I'm regurgitating second hand information and it's very possible I've misunderstood some of the details😅

Additionally, running the trace-agent.exe as a child-process is a little…hacky…in AAS, so occasionally AAS kills it entirely. At that point, we would need to re-negotiate a new port between the app client library and the trace-agent which is a headache.

I assume that we have to re-negotiate a new port because of the classic TIME_WAIT issue which used to be a problem for causing socket exhaustion with HttpClient.

By using named pipes we can skirt around all these issues. All the processes can talk to each other using a single named pipe, regardless of their process tree. And if the trace-agent.exe process is killed by AAS, we don't need to change the pipe. When the process restarts, it can start listening on the same named pipe. No port renegotiation or anything required, so no more headaches!

Creating a named pipe server with ASP.NET Core and Kestrel

Until .NET 8, if you wanted to implement a named pipe server you certainly could, but it's not exactly simple or easy. For a start, you would need to implement your own HTTP server on top of it if that's the protocol you want to use for communication.

Luckily, in .NET 8, ASP.NET Core added direct support for Windows named pipes to Kestrel, so you can use all the same features and programming model of ASP.NET Core as you're used to using with TCP 🎉

Changing your application to listen using named pipes is as simple as doing one of the following:

  • Configuring Kestrel in code using ListenNamedPipe()
  • Setting the URLs for your application to http://pipe:/<pipename>

The example below shows a very simple .NET 8 test app (adapted from the empty template: dotnet new web). The only change here is that I specifically configured the app to listen using named pipes:

var builder = WebApplication.CreateBuilder(args);

//    Configure Kestrel to listen on \\.\pipe\my-test-pipe
// 👇 Note that you only provide the final part of the pipe name
builder.WebHost.ConfigureKestrel(
    opts => opts.ListenNamedPipe("my-test-pipe"));

var app = builder.Build();

app.MapGet("/test", () =>  "Hello world!");

app.Run();

If you start the app with dotnet run, you'll see that the logs show the listen address as:

info: Microsoft.Hosting.Lifetime[14]                 
      Now listening on: http://pipe:/my-pipe-name

The http://pipe: looks a little odd, but it kind of makes sense. After all, we're still sending HTTP requests, we're just sending them over a named pipe instead of a TCP socket.

As I described in my previous post, you can also configure Kestrel entirely using IConfiguration, so instead of adding the ConfigureKestrel() call above, you could instead define the listen URL in your appsettings.json file like this:

{
  "Kestrel": {
    "Endpoints": {
      "NamedPipeEndpoint": {
        "Url": "http://pipe:/my-pipe-name"
      }
    }
  }
}

You can also use all the other approaches I described in my previous post to opt-in to binding named pipes using the same binding address pattern as the above configuration does. For example:

export ASPNETCORE_URLS="http://pipe:/my-pipe-name"

And in case you were wondering, yes, you can also specify https in your URLs and kestrel will use HTTPS for the requests!

ASP.NET Core exposes various settings for customizing the named pipe, such as buffer sizes and pipe security options. You can configure these using IConfiguration or by passing a configuration lambda to builder.WebHost.UseNamedPipes():

var builder = WebApplication.CreateBuilder(args);

// Customize the named pipe configuration
builder.WebHost.UseNamedPipes(opts =>
{ 
    // Bump the buffer sizes to 4MB (defaults to 1MB)
    opts.MaxWriteBufferSize = 4 * 1024 * 1024;
    opts.MaxReadBufferSize = 4 * 1024 * 1024;
});

// ...

The call to UseNamedPipes() is actually a prerequisite for using named pipes, as it adds the required services. You don't need to call it explicitly as it's added to the DI container automatically by default.

Calling the named pipe server with HttpClient

So in .NET 8, creating an HTTP server (or gRPC server, or anything else supported by ASP.NET Core!) that listens over named pipes is surprisingly easy. But you will likely also want to be able to send requests to the server, so we need a client.

For this, we need our trusty HttpClient and SocketsHttpHandler. SocketsHttpHandler was introduced in .NET Core 2.1, but we need to use the ConnectCallback property specifically, which was only added in .NET 5.

Technically, the NamedPipeClientStream that underlies the code below is available as far back as .NET Framework 3.5. But using this directly requires you build your own HTTP client to use it. Much like building your own HTTP server, I don't recommend it if you can avoid it!

To configure an HttpClient to use named pipes you need to specify a custom ConnectCallback() which will create a NamedPipeClientStream instance and connect to the server. After configuring the client, you just use it like you would to call any other ASP.NET Core app.

using System.IO.Pipes;

var httpHandler = new SocketsHttpHandler
{
    // Called to open a new connection
    ConnectCallback = async (ctx, ct) =>
    {
        // Configure the named pipe stream
        var pipeClientStream = new NamedPipeClientStream(
            serverName: ".", // 👈 this machine
            pipeName: "my-test-pipe", // 👈 
            PipeDirection.InOut, // We want a duplex stream 
            PipeOptions.Asynchronous); // Always go async

        // Connect to the server!
        await pipeClientStream.ConnectAsync(ct);
        
        return pipeClientStream;
    }
};

// Create an HttpClient using the named pipe handler
var httpClient = new HttpClient(httpHandler)
{
    BaseAddress = new Uri("http://localhost"); // 👈 Use localhost as the base address
};

var result = await httpClient.GetStringAsync("/test");
Console.WriteLine(result);

The only "oddity" here is that you need to give use a valid hostname in the BaseAddress of the HttpClient, or alternatively in the GetStringAsync() if you don't specify a BaseAddress. I've used localhost for simplicity, but it doesn't really matter what we use here. If you run the app, it should print "Hello World!" 🎉

Remember to use https: in the BaseAddress if you have configured ASP.NET Core to listen for HTTPS over named pipes!

And that's all there is to it. Using named pipes is so much easier in .NET 8 than it was in earlier versions, it should Just Work™. That said, the named pipe support is obviously relatively new, so it's worth keeping an eye out for any issues and reporting them to the .NET team!

Summary

In this post I described Windows named pipes. I discussed what they are, how they work, and some of the scenarios where they may be useful. They're particularly useful when you want to use specific Windows security features or where you're running in scenarios in which cross-process TCP communication is problematic. Next I described the named pipe support added to ASP.NET Core in .NET 8 and how to configure your app to listen on a named pipe. Finally, I showed how you can use HttpClient with a custom SocketsHttpHandler.ConnectCallback to make requests to a named pipe HTTP server.

Andrew Lock | .Net Escapades
Want an email when
there's new posts?