Create Persistent Connections to Processes Using FIFOs

Mar 26, 2024
Modified on Apr 2, 2024
Categories: Shell , Bash , TIL
#fifo , #named pipe , #persistent , #scripting
6 minute read

In this post, we are going to make a Bash script interact with an HTTP server and have multiple exchanges using the same connection by leveraging named pipes/FIFOs.

My first, naive approach was to first create named pipes i (for “input”) and o (for “output”). I ran openssl s_client with its stdin reading from i and its stdout writing to o and backgrounded it. Then, I used the shell builtin printf and cat to write and read from i and o respectively:

#!/bin/bash

mkfifo i o

h=bol.com
openssl s_client -servername $h $h:443 <i >o &

printf '%s\r\n' 'GET / HTTP/1.1' 'Host: bol.com' '' >i

cat <o

This results in the following (output is truncated for brevity):

CONNECTED(00000003)
---
Certificate chain
 0 s:CN=www.bol.com
   i:C=US, O=Let's Encrypt, CN=R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 17 10:19:22 2024 GMT; NotAfter: Jun 15 10:19:21 2024
GMT
 1 s:C=US, O=Let's Encrypt, CN=R3
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025
GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
... snip ...

In Bash, redirections (e.g. reading from or writing to a named pipe) are processed before starting a command. Therefore, Bash blocks while trying to open the named pipes until both the reading and writing ends are connected. That means that openssl will not be executed until it has some other process writing to its input pipe and reading from its output pipe.

(We can prove this to ourselves by doing a process listing:

$ h=bol.com; openssl s_client -servername $h $h:443 <i >o &
$ pgrep openssl; echo $?
1

Even when writing to the input pipe, but not reading yet from the output pipe, the process is not started yet:

$ printf '%s\r\n' 'GET / HTTP/1.1' 'Host: bol.com' '' >i
$ pgrep openssl; echo $?
1

So, only when some other process starts reading from its output will openssl actually start.

Trying to run strace(1) on cat <o was an eye-opener for me:

$ strace cat <o

This actually blocks any strace output until the writing end is written to, showing clearly that “redirection is done before any process is started”.

By having a new Bash process execute this code, we can bypass this. Say, we have a file test:

#!/bin/bash

cat <i

We execute this with strace -f -yy bash test. When that blocks at openat (see code listing below), we execute /bin/echo foo >i from another terminal window:

... snip ...
[pid  4497] openat(AT_FDCWD</tmp/tmp.EwoMhpZw3f>, "i", O_RDONLY) = 3</tmp/tmp.EwoMhpZw3f/i>
[pid  4497] dup2(3</tmp/tmp.EwoMhpZw3f/i>, 0</dev/pts/8<char 136:8>>) = 0</tmp/tmp.EwoMhpZw3f/i>
[pid  4497] close(3</tmp/tmp.EwoMhpZw3f/i>) = 0
[pid  4497] execve("/usr/bin/cat", ["cat"], 0x5c51da8fdb20 /* 52 vars */) = 0
... snip ...

We see that the system call execve on /usr/bin/cat was only invoked after the writing end of the pipe was opened.

Likewise, when straceing the script containing /bin/echo foo >i, we see:

[pid  9261] openat(AT_FDCWD</tmp/tmp.wVg9PTBQxY>, "i", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3</tmp/tmp.wVg9PTBQxY/i>
[pid  9261] dup2(3</tmp/tmp.wVg9PTBQxY/i>, 1</dev/pts/6<char 136:6>>) = 1</tmp/tmp.wVg9PTBQxY/i>
[pid  9261] close(3</tmp/tmp.wVg9PTBQxY/i>) = 0
[pid  9261] execve("/bin/echo", ["/bin/echo", "foo"], 0x5b99e9dfeb90 /* 52 vars */) = 0

/bin/echo is only execved after opening i.)

In our HTTP case, the writing happens by printf >i, the reading by cat <o.

However, the output reader only sees TLS handshake information. The connection to the remote server is closed. Although strace confirms that openssl did sent the request to the remote server, we did not see any HTTP response in the output. Why not? According to pipe(7):

If all file descriptors referring to the write end of a pipe have been closed, then an attempt to read(2) from the pipe will see end-of-file (read(2) will return 0).

That means that openssl has seen an EOF when it tried to read from i, and terminated itself immediately (openssl-s_client(1) provides -ign-eof to prevent that, BTW).

(To empirically show that the last writer closes the writing end of the pipe:

$ mkfifo io
$ { cat < io; echo cat received EOF; } &
$ cat > io &
$ cat > io &
$ jobs
[1]   Running                 { cat < io; echo cat received EOF; } &
[2]-  Stopped                 cat > io &
[3]+  Stopped                 cat > io &
$ kill %3
[3]+  Terminated              cat > io
$ kill %2
[2]+  Terminated              cat > io
cat received EOF

When terminating all writers to io, the reader cat < io received EOF and terminated itself.)

So, when cat <o reads from the o pipe, the openssl process starts. It reads input from i while sending output to o. It quits once it sees EOF when trying to read again, before it receives the response from the remote site.

Now that we know that a reader sees EOF when the last writer quits, we can open a file descriptor to the pipe. That file descriptor will stay open until we close the terminal (or explicitly close it with exec 3>&-):

$ exec 3>i

This looks good:

CONNECTED(00000003)
---
Certificate chain
 0 s:CN=www.bol.com
   i:C=US, O=Let's Encrypt, CN=R3
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 17 10:19:22 2024 GMT; NotAfter: Jun 15 10:19:21 2024 GMT
... snip ...
Verify return code: 0 (ok)
---
HTTP/1.1 301 Moved Permanently
date: Mon, 01 Apr 2024 13:34:24 GMT
... snip ...

This works beautifully. We can send requests from another terminal and cat <o keeps the output pipe open. If we want to interact with the output (e.g. grep it), however, need something else. We would like to just keep the output pipe open, read whatever we want from the output stream, play with it, make the next request, etc. We cannot simply terminate cat <o, because it is the last reader from o. When openssl writes to o while there are no readers left, openssl will terminate itself. That is because when a process tries to write to a pipe that’s closed, it will receive the SIGPIPE signal from the kernel:

If all file descriptors referring to the read end of a pipe have been closed, then a write(2) will cause a SIGPIPE signal to be generated for the calling process.

And we did close the pipe by terminating cat <o. But openssl is not writing anything, it just sits there, right? Right, but a TLS connection is a two-way street and the remote server might send something, like a session ticket renewal. openssl will try to write this to the output pipe and terminate.

Opening another file descriptor that reads from the output should do it:

$ exec 3>i 4<o

This will keep both the input and output streams open its process ends. Finally, we’re ready to have as many exchanges as we want:

$ h=bol.com
$ openssl s_client -servername $h $h:443 <i >o &
[1] 5414
$ exec 3>i 4<o
[2] 5415
$ printf '%s\r\n' 'GET / HTTP/1.1' 'Host: bol.com' '' >i
$ cat <o
... snip certificate stuff ...
HTTP/1.1 301 Moved Permanently
date: Thu, 28 Mar 2024 09:14:54 GMT
... snip ...
^C
$ printf '%s\r\n' 'GET / HTTP/1.1' 'Host: bol.com' '' >i
$ cat <o
HTTP/1.1 301 Moved Permanently
date: Thu, 28 Mar 2024 09:15:24 GMT
... snip ...
^C