Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save amgorb/5702b648f1c0ec4c9eb42f2dbe04b038 to your computer and use it in GitHub Desktop.
Save amgorb/5702b648f1c0ec4c9eb42f2dbe04b038 to your computer and use it in GitHub Desktop.
haproxy victoria metrics active tcp check handshake loadbalancer
If you are using Victoria Metrics with load balancer like haproxy with active healt checks, your vmstorage logs will be flooded
with messages like this:
{"ts":"2025-07-24T10:24:00.291Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33942\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33942: read: connection reset by peer; read only 0 bytes"}
{"ts":"2025-07-24T10:24:02.464Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33960\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33960: read: connection reset by peer; read only 0 bytes"}
{"ts":"2025-07-24T10:24:04.603Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33980\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33980: read: connection reset by peer; read only 0 bytes"}
{"ts":"2025-07-24T10:24:06.588Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33994\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33994: read: connection reset by peer; read only 0 bytes"}
{"ts":"2025-07-24T10:24:08.629Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:58416\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:58416: read: connection reset by peer; read only 0 bytes"}
this is because handshake is incomplete.
Handshake is: 'vmselect.01\nok'
38402 (= vmstorage port)
# echo -ne 'vmselect.01\nok' | nc -w 1 localhost 38402
Add this to haproxy backend section:
mode tcp
option tcp-check
tcp-check send vmselect.01\nok
tcp-check expect string ok
and your vmstorage logs will be nice and clean
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment