Created
July 24, 2025 10:28
-
-
Save amgorb/5702b648f1c0ec4c9eb42f2dbe04b038 to your computer and use it in GitHub Desktop.
haproxy victoria metrics active tcp check handshake loadbalancer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If you are using Victoria Metrics with load balancer like haproxy with active healt checks, your vmstorage logs will be flooded | |
with messages like this: | |
{"ts":"2025-07-24T10:24:00.291Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33942\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33942: read: connection reset by peer; read only 0 bytes"} | |
{"ts":"2025-07-24T10:24:02.464Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33960\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33960: read: connection reset by peer; read only 0 bytes"} | |
{"ts":"2025-07-24T10:24:04.603Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33980\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33980: read: connection reset by peer; read only 0 bytes"} | |
{"ts":"2025-07-24T10:24:06.588Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:33994\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:33994: read: connection reset by peer; read only 0 bytes"} | |
{"ts":"2025-07-24T10:24:08.629Z","level":"error","caller":"VictoriaMetrics/lib/vmselectapi/server.go:203","msg":"cannot perform vmselect handshake with client \"127.0.0.1:58416\": cannot read hello: cannot read message with size 11: read tcp4 127.0.0.1:8401->127.0.0.1:58416: read: connection reset by peer; read only 0 bytes"} | |
this is because handshake is incomplete. | |
Handshake is: 'vmselect.01\nok' | |
38402 (= vmstorage port) | |
# echo -ne 'vmselect.01\nok' | nc -w 1 localhost 38402 | |
Add this to haproxy backend section: | |
mode tcp | |
option tcp-check | |
tcp-check send vmselect.01\nok | |
tcp-check expect string ok | |
and your vmstorage logs will be nice and clean |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment