负载均衡测速报错 error: Connection refused (os error 111)

[pexcn]

我的配置文件如下：

{
    "locals": [
        {
            "local_address": "127.0.0.1",
            "local_port": 1234,
            "protocol": "redir",
            "tcp_redir": "tproxy",
            "udp_redir": "tproxy"
        },
        {
            "local_address": "127.0.0.1",
            "local_port": 5300,
            "protocol": "tunnel",
            "forward_address": "8.8.8.8",
            "forward_port": 53
        }
    ],

    "servers": [
        //
        // FASTEST
        //
        {
            "server": "xxx.xxx.xxx.xxx",
            "server_port": 2047,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "chacha20-ietf-poly1305",
            "mode": "tcp_only",
            "plugin": "qtun-client",
            "plugin-opts": "host=my.domain.com"
        },
        {
            "server": "127.0.0.1",
            "server_port": 2004,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "udp_only"
        },

        //
        // TCP
        //
        {
            "server": "111.111.111.111",
            "server_port": 1984,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "tcp_only"
        },
        {
            "server": "111.111.111.112",
            "server_port": 1984,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "tcp_only"
        },
        {
            "server": "111.111.111.113",
            "server_port": 1984,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "tcp_only"
        },

        //
        // UDP
        //
        {
            "server": "127.0.0.1",
            "server_port": 2001,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "udp_only"
        },
        {
            "server": "127.0.0.1",
            "server_port": 2002,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "udp_only"
        },
        {
            "server": "127.0.0.1",
            "server_port": 2003,
            "password": "${SHADOWSOCKS_PASSWORD}",
            "method": "2022-blake3-chacha8-poly1305",
            "mode": "udp_only"
        }
    ],

    "balancer": {
        "max_server_rtt": 5,
        "check_interval": 30,
        "check_best_interval": 15
    }
}

启动命令如下：

ssservice local --config ./local.json -vv 2>&1 | grep ping_balancer | grep -v "got AEAD"

其中日志第三行出现了 Connection refused (os error 111), 而实际是可以使用的。
我这里使用了 qtun 插件，服务器中只监听了 UDP 的 2047 端口，请问这是否和这个原因有关？

[shadowsocks_service::local::loadbalancing::ping_balancer] kept best TCP server 111.111.111.111:1984
[shadowsocks_service::local::loadbalancing::ping_balancer] kept best UDP server 127.0.0.1:2004
[shadowsocks_service::local::loadbalancing::ping_balancer] failed to check TCP server xxx.xxx.xxx.xxx:2047, error: Connection refused (os error 111)
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote TCP server xxx.xxx.xxx.xxx:2047 (score: 8000)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote UDP server 127.0.0.1:2004 latency with 16 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote UDP server 127.0.0.1:2004 (score: 7)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote UDP server 127.0.0.1:2003 latency with 174 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote UDP server 127.0.0.1:2003 (score: 1071)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote UDP server 127.0.0.1:2002 latency with 181 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote UDP server 127.0.0.1:2002 (score: 79)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote TCP server 111.111.111.113:1984 latency with 330 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote TCP server 111.111.111.113:1984 (score: 139)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote TCP server 111.111.111.112:1984 latency with 350 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote TCP server 111.111.111.112:1984 (score: 378)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote TCP server 111.111.111.111:1984 latency timeout, elapsed 5001 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote TCP server 111.111.111.111:1984 (score: 8000)
[shadowsocks_service::local::loadbalancing::ping_balancer] checked remote UDP server 127.0.0.1:2001 latency timeout, elapsed 5001 ms
[shadowsocks_service::local::loadbalancing::ping_balancer] updated remote UDP server 127.0.0.1:2001 (score: 8000)

[dev4u]

去掉qtun插件，或者换在一个稳定的网络环境下使用他。

[pexcn]

能兼容这个插件吗？我想让它搭配 load balancer 一起使用

[zonyitoo]

I don't think it has anything related to "compatibility".

Since the connect() call on a TCP socket returned Connection refused (os error 111), then maybe your qtun-client was not listening on the port that sslocal expected.

Is qtun-client supports the environment variables defined in SIP003?

[dev4u]

不是qtun不兼容，而是当你使用的网络发生了变化，ss client重用之前建立好的连接有可能已经broken，导致你看到的异常。

[zonyitoo]

Of course not. The connection in ping balancer is always new.

Since this server has plugin, so the Connection refused (os error 111) indicated that sslocal failed to connect to plugin's port that qtun-client should listen to.

If the qtun-client comes from this project: https://github.com/shadowsocks/qtun, then it should support SIP003. Hmm.

[dev4u]

Of course not. The connection in ping balancer is always a new.

我说qtun呢。

[zonyitoo]

Then it won't fail when sslocal tried to connect() to its listening port.

[pexcn]

不是qtun不兼容，而是当你使用的网络发生了变化，ss client重用之前建立好的连接有可能已经broken，导致你看到的异常。

网络是稳定的，服务端和客户端都是刚刚启动，并且另外再开了一个客户端测试过是可用的。

Since the connect() call on a TCP socket returned Connection refused (os error 111), then maybe your qtun-client was not listening on the port that sslocal expected.

也许这和 qtun 的 listen 参数有关？它默认是监听了 8138 端口

Is qtun-client supports the environment variables defined in SIP003?

应该是支持的，它在源码里也有体现 https://github.com/shadowsocks/qtun/blob/master/src/args/mod.rs#L6-L26

[zonyitoo]

https://github.com/shadowsocks/qtun/blob/9e1834e00f38d0424f5252fe6331533bfc50d207/src/client.rs#L49-L57

The listen IP and port should come from environment variables.

Did you see the line listening on ... from qtun-client? https://github.com/shadowsocks/qtun/blob/9e1834e00f38d0424f5252fe6331533bfc50d207/src/client.rs#L92-L94

[pexcn]

https://github.com/shadowsocks/qtun/blob/9e1834e00f38d0424f5252fe6331533bfc50d207/src/client.rs#L49-L57

The listen IP and port should come from environment variables.

Did you see the line listening on ... from qtun-client? https://github.com/shadowsocks/qtun/blob/9e1834e00f38d0424f5252fe6331533bfc50d207/src/client.rs#L92-L94

是的，我发现尽管配置了 ...;listen=127.0.0.1:2047, 日志里也会出现监听了 127.0.0.1:8138

[zonyitoo]

There is no listen= option in plugin_opts. The only option is host: https://github.com/shadowsocks/qtun/blob/9e1834e00f38d0424f5252fe6331533bfc50d207/src/client.rs#L54-L56 .

[pexcn]

也许我应该把这个 bug 反馈给 qtun 😄

[zonyitoo]

Did you really see the log listening on ... from qtun-client?

[pexcn]

Did you really see the log listening on ... from qtun-client?

是的，日志如下：

/tmp # ssservice local --config ./local.json -v
2023-02-02T14:34:24.548326221+08:00 INFO  [172:140030336721968] [shadowsocks_rust::service::local] shadowsocks local 1.15.2 build 2023-01-17T14:27:56.620285783+00:00
2023-02-02T14:34:24.550215301+08:00 DEBUG [172:140030336721968] [shadowsocks::plugin] started plugin "qtun-client" on 127.0.0.1:38649 <-> xxx.xxx.xxx.xxx:2047 (175)
[2023-02-02T06:34:24Z INFO  qtun_client] listening on 127.0.0.1:8138
2023-02-02T14:34:27.551698778+08:00 DEBUG [172:140030336721968] [shadowsocks_service::local::loadbalancing::ping_balancer] failed to check TCP server xxx.xxx.xxx.xxx:2047, error: Connection refused (os error 111)
2023-02-02T14:34:32.553126510+08:00 INFO  [172:140030336721968] [shadowsocks_service::local::loadbalancing::ping_balancer] chose best TCP server yyy.yyy.yyy.yyy:1984
2023-02-02T14:34:32.553231968+08:00 INFO  [172:140030336721968] [shadowsocks_service::local::loadbalancing::ping_balancer] chose best UDP server 127.0.0.1:2004

[pexcn]

我发现 qtun 只监听了 UDP 端口，会不会是这个原因？😮

root@HK ~/docker-compose/net/shadowsocks-rust/shadowsocks-rust-data # netstat -naop | grep qtun
udp        0      0 0.0.0.0:2047            0.0.0.0:*                           462031/qtun-server   off (0.00/0/0)
unix  3      [ ]         STREAM     CONNECTED     8615865  462031/qtun-server
unix  3      [ ]         STREAM     CONNECTED     8615866  462031/qtun-server