Kagi Assistant Times Out when on VPN

I use Kagi as my search engine and for LLM interactions. I am also always connected to ProtonVPN. I and others have noticed that the output from their LLM Assistant is cut of after a couple of seconds when on ProtonVPN. Why does that happen and what can we do about it?

What’s Wrong with VPNs?

Instead of a typical LLM response flow, we see:

At first, the problem seemed to be related to being on a VPN:

vpn$ H=kagi.com; time openssl s_client -quiet -servername $H $H:443 
Connecting to 34.111.242.115
depth=2 C=US, O=Google Trust Services LLC, CN=GTS Root R1
verify return:1
depth=1 C=US, O=Google Trust Services, CN=WR3
verify return:1
depth=0 CN=kagi.com
verify return:1
80CB5B021D740000:error:0A000126:SSL routines::unexpected eof while
reading:ssl/record/rec_layer_s3.c:696:

real    0m6.599s
user    0m0.005s
sys 0m0.012s

When using the LLM from a static residential IP, no timeout occurs (I got impatient, so terminated with ^C):

residential$ H=kagi.com; time openssl s_client -quiet -servername $H $H:443
Connecting to 34.111.242.115
depth=2 C=US, O=Google Trust Services LLC, CN=GTS Root R1
verify return:1
depth=1 C=US, O=Google Trust Services, CN=WR3
verify return:1
depth=0 CN=kagi.com
verify return:1
^C
real    1m25.933s
user    0m0.016s
sys 0m0.007s

At first, I thought that maybe ProtonVPN ages out TCP connections after a couple of seconds of inactivity to free space in its NAT/PAT translation table. But that seemed like a draconian expiration, and also, it didn’t bear out:

vpn$ H=nos.nl; time openssl s_client -quiet -servername $H $H:443 
Connecting to 18.239.50.102
depth=2 C=GB, O=Sectigo Limited, CN=Sectigo Public Server Authentication Root
R46
verify return:1
depth=1 C=GB, O=Sectigo Limited, CN=Sectigo Public Server Authentication CA EV
R36
verify return:1
depth=0 serialNumber=32142157, jurisdictionC=NL, businessCategory=Private
Organization, C=NL, ST=Noord-Holland, O=Nederlandse Omroep Stichting, CN=nos.nl
verify return:1
^C
real    0m32.315s
user    0m0.007s
sys 0m0.009s

Oh, So It’s Google

OpenAI’s o3 model suggested to me that it might be Google’s Load Balancer. You see, every ProtonVPN server has a pool of addresses. Some or all of these addresses are marked by Google as high-risk, because a lot of simultaneous connections and/or attacks originate from them to Google properties. To prevent certain kind of attacks (Slowloris), resource depletion), Google tears down connections when the client is not sending data. Looking at a typical exchange with Wireshark, you will notice that if the model thinks more than 5 seconds, the connection will be torn down. If the model starts outputting its response before 5 seconds have elapsed, the connection will still be cut down exactly 5 seconds after the client sent its last TCP segment with a payload (i.e. the prompt). Although Wireshark shows my client sending many subsequent ACKs, none of these carry a payload, and they are thus not seen by Google as meaningful traffic:

I haven’t found any documentation to back up these claims, but the empirical data is there.

What Can You Do?

A workaround would be to try another egress IP address by switching to a different (Proton)VPN server. Anecdotally, this seems to work, which suggests that Google has not put the entire ProtonVPN IP range on its naughty list.

Seeing that Kagi is using Google infrastructure, they might also be able to do something about this by increasing this 5s teardown rule for high-risk IP addresses in their Google console. I do not know whether that is (1) possible or (2) desirable, though, as it might open them up to potential DoS.