Doesn't work via VPN tunnel

[VladimirKravtsov]

Hi there!
First of all, huge thank you, developers, for so helpful tool!

Just recently I started to experience the trouble on macOS BigSur with accessing resources via bastion when a connection to a corporate VPN server is up.

sshuttle connects to bastion through VPN tunnel without visible issues, however, when I try to get connected to a resource via bastion, netcat throws "Operation timed out" error. Recently it was working for me on the same laptop, but after upgrade shuttle version and VPN client, it has broken.

sshutle - 1.0.4 (installed via homebrew)
Python - 3.9
MacOS - 11.0.1 BigSur
VPN - GlobalProtect

Also, I've tried sshuttle 1.0.3 on Python 3.8 - same behaviour. However, everything is working well without VPN.

When VPN connection is active, I run shuttle:

shuttle log

❯ sshuttle -vvv -r bastion 52.208.177.221/32
Starting sshuttle proxy.
firewall manager: Starting firewall with Python version 3.9.0
firewall manager: ready method name pf.
IPv6 enabled: True
UDP enabled: False
DNS enabled: False
User enabled: False
Binding redirector: 12300
TCP redirector listening on ('::1', 12300, 0, 0).
TCP redirector listening with <socket.socket fd=5, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, laddr=('::1', 12300, 0, 0)>.
TCP redirector listening on ('127.0.0.1', 12300).
TCP redirector listening with <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 12300)>.
Starting client with Python version 3.9.0
c : connecting to server...
c : executing: ['ssh', 'bastion', '--', '/bin/sh -c \'P=python3; $P -V 2>/dev/null || P=python; exec "$P" -c \'"\'"\'import sys, os; verbosity=3; sys.stdin = os.fdopen(0, "rb"); exec(compile(sys.stdin.read(1279), "assembler.py", "exec"))\'"\'"\'\'']
c :  > channel=0 cmd=PING len=7 (fullness=0)
server: assembling 'sshuttle' (88 bytes)
server: assembling 'sshuttle.cmdline_options' (65 bytes)
server: assembling 'sshuttle.helpers' (892 bytes)
server: assembling 'sshuttle.ssnet' (5789 bytes)
server: assembling 'sshuttle.hostwatch' (2466 bytes)
server: assembling 'sshuttle.server' (3675 bytes)
Starting server with Python version 3.5.3
 s: latency control setting = True
 s:  > channel=0 cmd=PING len=7 (fullness=0)
 s: auto-nets:False
 s:  > channel=0 cmd=ROUTES len=0 (fullness=7)
 s: Waiting: 1 r=[0] w=[1] x=[] (fullness=7/0)
c : Connected.
 s:   Ready: 1 r=[] w=[1] x=[]
 s: mux wrote: 15/15
 s: Waiting: 1 r=[0] w=[1] x=[] (fullness=7/0)
 s:   Ready: 1 r=[] w=[1] x=[]
c : Waiting: 2 r=[5, 7, 9] w=[9] x=[] (fullness=7/0)
c :   Ready: 2 r=[9] w=[9] x=[]
c : <  channel=0 cmd=PING len=7
c :  > channel=0 cmd=PONG len=7 (fullness=7)
c : mux wrote: 15/15
c : mux wrote: 15/15
c : Waiting: 2 r=[5, 7, 9] w=[] x=[] (fullness=14/0)
 s: mux wrote: 8/8
 s: Waiting: 1 r=[0] w=[] x=[] (fullness=7/0)
c :   Ready: 2 r=[9] w=[] x=[]
c : <  channel=0 cmd=ROUTES len=0
firewall manager: Got subnets: [(2, 32, False, '52.208.177.221', 0, 0), (2, 32, True, '127.0.0.1', 0, 0), (30, 128, True, '::1', 0, 0)]
firewall manager: Got nslist: []
firewall manager: Got ports: 12300,12300,0,0
firewall manager: Got udp: False, user: None
firewall manager: setting up.
firewall manager: setting up IPv6.
>> pfctl -s Interfaces -i lo -v
 s:   Ready: 1 r=[0] w=[] x=[]
 s: <  channel=0 cmd=PING len=7
 s:  > channel=0 cmd=PONG len=7 (fullness=7)
 s: <  channel=0 cmd=PONG len=7
 s: received PING response
 s: mux wrote: 15/15
 s: Waiting: 1 r=[0] w=[] x=[] (fullness=0/0)
>> pfctl -s all
rules:
---> pass out inet6 proto tcp to ::1/128
>> pfctl -a sshuttle6-12300 -f /dev/stdin
>> pfctl -E
firewall manager: setting up IPv4.
>> pfctl -s Interfaces -i lo -v
>> pfctl -s all
rules:
---> rdr pass on lo0 inet proto tcp from ! 127.0.0.1 to 52.208.177.221/32 -> 127.0.0.1 port 12300
---> pass out route-to lo0 inet proto tcp to 52.208.177.221/32 keep state
---> pass out inet proto tcp to 127.0.0.1/32
>> pfctl -a sshuttle-12300 -f /dev/stdin
>> pfctl -E
c : Waiting: 2 r=[5, 7, 9] w=[] x=[] (fullness=14/0)
c :   Ready: 2 r=[9] w=[] x=[]
c : <  channel=0 cmd=PONG len=7
c : received PING response
c : Waiting: 2 r=[5, 7, 9] w=[] x=[] (fullness=0/0)

Then try to reach out the resource which should be accessible over bastion:

nc output

❯ nc -vz 52.208.177.221 80
nc: connectx to 52.208.177.221 port 80 (tcp) failed: Operation timed out

tcpdump

❯ sudo tcpdump -nvvvi lo0 tcp and port 80 and dst 52.208.177.221
Password:
tcpdump: listening on lo0, link-type NULL (BSD loopback), capture size 262144 bytes
16:42:52.087303 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x5b76 (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906331310 ecr 0,sackOK,eol], length 0
16:42:53.089663 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x578e (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906332310 ecr 0,sackOK,eol], length 0
16:42:54.093105 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x53a6 (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906333310 ecr 0,sackOK,eol], length 0
16:42:55.097813 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x4fbe (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906334310 ecr 0,sackOK,eol], length 0
16:42:56.102782 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x4bd5 (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906335311 ecr 0,sackOK,eol], length 0
16:42:57.108287 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x47ed (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906336311 ecr 0,sackOK,eol], length 0
16:42:59.111677 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x401d (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906338311 ecr 0,sackOK,eol], length 0
16:43:03.122305 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x307c (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906342312 ecr 0,sackOK,eol], length 0
16:43:11.134725 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x113c (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906350312 ecr 0,sackOK,eol], length 0
16:43:27.176248 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->832a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0xd2bb (correct), seq 2041573538, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 906366312 ecr 0,sackOK,eol], length 0
16:43:59.242433 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 48, bad cksum 0 (->833a)!)
    100.64.108.160.65285 > 52.208.177.221.80: Flags [S], cksum 0x634e (correct), seq 2041573538, win 65535, options [mss 1360,sackOK,eol], length 0

routing table

❯ sudo netstat -nr
Password:
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            100.64.108.160     UGSc         utun5
default            192.168.1.1        UGScI          en0
10.6.0.6           100.64.108.160     UGHS         utun5
10.6.0.16          100.64.108.160     UGHS         utun5
86.57.255.94       192.168.1.1        UGHS           en0
100.64.108.160     100.64.108.160     UHr          utun5
100.64.108.160/32  link#16            UCS          utun5
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#4             UCS            en0      !
169.254.46.37      c:8b:fd:4b:c3:14   UHLSW          en0      !
192.168.1          link#4             UCS            en0      !
192.168.1.1/32     link#4             UCS            en0      !
192.168.1.1        b0:4e:26:f0:54:4f  UHLWIir        en0   1186
192.168.1.100/32   link#4             UCS            en0      !
192.168.1.100      ac:bc:32:c4:b8:df  UHLWI          lo0
192.168.1.102      ea:1f:5a:67:cc:31  UHLWIi         en0    303
192.168.1.106      2a:6f:ab:41:7c:da  UHLWIi         en0    771
224.0.0/4          link#16            UmCS         utun5
224.0.0/4          link#4             UmCSI          en0      !
224.0.0.251        1:0:5e:0:0:fb      UHmLWI         en0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
255.255.255.255/32 link#16            UCS          utun5
255.255.255.255/32 link#4             UCSI           en0      !

pf -s all

❯ sudo pfctl -s all
No ALTQ support in kernel
ALTQ related functions disabled
TRANSLATION RULES:
nat-anchor "com.apple/*" all
rdr-anchor "com.apple/*" all
rdr-anchor "sshuttle6-12300" all
rdr-anchor "sshuttle-12300" all

FILTER RULES:
scrub-anchor "com.apple/*" all fragment reassemble
anchor "com.apple/*" all
anchor "sshuttle6-12300" all
anchor "sshuttle-12300" all

DUMMYNET RULES:
dummynet-anchor "com.apple/*" all

STATES:
ALL tcp 192.168.1.100:52137 -> 52.49.35.215:80       ESTABLISHED:ESTABLISHED
ALL tcp 127.0.0.1:12300 <- 52.49.35.215:80 <- 192.168.1.100:52137       ESTABLISHED:ESTABLISHED
ALL tcp 100.64.108.160:65285 -> 52.208.177.221:80       SYN_SENT:CLOSED
ALL tcp 127.0.0.1:12300 <- 52.208.177.221:80 <- 100.64.108.160:65285       SYN_SENT:ESTABLISHED

INFO:
Status: Enabled for 0 days 00:07:40           Debug: Urgent

State Table                          Total             Rate
  current entries                        4
  searches                         4787932        10408.5/s
  inserts                              944            2.1/s
  removals                             940            2.0/s
Counters
  match                            2801941         6091.2/s
  bad-offset                             0            0.0/s
  fragment                               0            0.0/s
  short                                  0            0.0/s
  normalize                              0            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                          0            0.0/s
  congestion                             0            0.0/s
  ip-option                            745            1.6/s
  proto-cksum                            0            0.0/s
  state-mismatch                        58            0.1/s
  state-insert                           0            0.0/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s
  dummynet                               0            0.0/s

TIMEOUTS:
tcp.first                   120s
tcp.opening                  30s
tcp.established           86400s
tcp.closing                 900s
tcp.finwait                  45s
tcp.closed                   90s
tcp.tsdiff                   30s
udp.first                    60s
udp.single                   30s
udp.multiple                 60s
icmp.first                   20s
icmp.error                   10s
grev1.first                 120s
grev1.initiating             30s
grev1.estblished           1800s
esp.first                   120s
esp.estblished              900s
other.first                  60s
other.single                 30s
other.multiple               60s
frag                         30s
interval                     10s
adaptive.start             6000 states
adaptive.end              12000 states
src.track                     0s

LIMITS:
states        hard limit    10000
app-states    hard limit    10000
src-nodes     hard limit    10000
frags         hard limit     5000
tables        hard limit     1000
table-entries hard limit   200000

OS FINGERPRINTS:
696 fingerprints loaded

Looks like tcp traffic can't reach 127.0.0.1:12300, despite it has been routed to lo0 interface.

[cunnie]

I'm seeing the same issue.

• macOS Big Sur 11.1
• GlobalProtect Version 5.1.5-20
• sshuttle 1.0.4 (also tried 1.0.5 built from source)

[mschindl]

I'm seeing the same issue.

macOS Big Sur 11.1
Viscosity Version 1.9
sshuttle 1.0.4

Starting sshuttle proxy.
[local sudo] Password: 
firewall manager: Starting firewall with Python version 3.9.1
firewall manager: ready method name pf.
IPv6 enabled: True
UDP enabled: False
DNS enabled: True
User enabled: False
TCP redirector listening on ('::1', 12300, 0, 0).
TCP redirector listening on ('127.0.0.1', 12300).
DNS listening on ('::1', 12299, 0, 0).
DNS listening on ('127.0.0.1', 12299).
Starting client with Python version 3.9.1
c : connecting to server...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 18, in <module>
TypeError: module.__init__() argument 1 must be string, not unicode
c : fatal: expected server init string b'SSHUTTLE0001'; got b''

[jmehnle]

Same issue on …

macOS Catalina 10.15.7
Tunnelblick 3.8.4a
sshuttle 1.0.5 (installed via MacPorts)

sshuttle log

$ sshuttle -vvv -r dev-bastion --dns 10.1.0.0/16
Starting sshuttle proxy (version 1.0.5).
c : which() found 'sudo' at /usr/bin/sudo
fw: Starting firewall with Python version 3.8.7
fw: Selecting a method automatically...
fw: which() could not find 'iptables' in /Users/julian/source/data_management/.venv/bin:/opt/local/sbin:/opt/local/bin:/Users/julian/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Keybase.app/Contents/SharedSupport/bin:/Library/Apple/usr/bin
fw: nat method not supported because 'iptables' command is missing.
fw: which() could not find 'nft' in /Users/julian/source/data_management/.venv/bin:/opt/local/sbin:/opt/local/bin:/Users/julian/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Keybase.app/Contents/SharedSupport/bin:/Library/Apple/usr/bin
fw: nft method not supported because 'nft' command is missing.
fw: which() found 'pfctl' at /sbin/pfctl
fw: Method 'pf' was automatically selected.
fw: which() found 'pfctl' at /sbin/pfctl
fw: ready method name pf.
c : Found DNS servers in /etc/resolv.conf: ['192.168.86.1']
c : Failed to read /run/systemd/resolve/resolv.conf when looking for DNS servers: No such file or directory
c : IPv6 enabled: Using default IPv6 listen address ::1
c : Method: pf
c : IPv4: on
c : IPv6: on
c : UDP : off (not available with pf method)
c : DNS : on
c : User: off (not available with pf method)
c : Subnets to forward through remote host (type, IP, cidr mask width, startPort, endPort):
c :   (<AddressFamily.AF_INET: 2>, '10.1.0.0', 16, 0, 0)
c : Subnets to exclude from forwarding:
c :   (<AddressFamily.AF_INET: 2>, '127.0.0.1', 32, 0, 0)
c :   (<AddressFamily.AF_INET6: 30>, '::1', 128, 0, 0)
c : DNS requests normally directed at these servers will be redirected to remote:
c :   (<AddressFamily.AF_INET: 2>, '192.168.86.1')
c : Trying to bind redirector on port 12300
c : TCP redirector listening on ('::1', 12300, 0, 0).
c : TCP redirector listening with <socket.socket fd=5, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, laddr=('::1', 12300, 0, 0)>.
c : TCP redirector listening on ('127.0.0.1', 12300).
c : TCP redirector listening with <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 12300)>.
c : Trying to bind DNS redirector on port 12300
c : Trying to bind DNS redirector on port 12299
c : DNS listening on ('::1', 12299, 0, 0).
c : DNS listening with <socket.socket fd=8, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('::1', 12299, 0, 0)>.
c : DNS listening on ('127.0.0.1', 12299).
c : DNS listening with <socket.socket fd=9, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 12299)>.
c : Starting client with Python version 3.8.7
c : Connecting to server...
c : which() found 'ssh' at /usr/bin/ssh
c : executing: ['/usr/bin/ssh', 'dev-bastion', '--', '/bin/sh -c \'P=python3; $P -V 2>/dev/null || P=python; exec "$P" -c \'"\'"\'import sys, os; verbosity=3; sys.stdin = os.fdopen(0, "rb"); exec(compile(sys.stdin.read(1486), "assembler.py", "exec"))\'"\'"\'\'']
c :  > channel=0 cmd=PING len=7 (fullness=0)
 s: assembling 'sshuttle' (88 bytes)
 s: assembling 'sshuttle.cmdline_options' (65 bytes)
 s: assembling 'sshuttle.helpers' (2646 bytes)
 s: assembling 'sshuttle.ssnet' (5777 bytes)
 s: assembling 'sshuttle.hostwatch' (2436 bytes)
 s: assembling 'sshuttle.server' (3649 bytes)
 s: Starting server with Python version 3.6.9
 s: latency control setting = True
 s:  > channel=0 cmd=PING len=7 (fullness=0)
c : Connected to server.
 s: auto-nets:False
c : Waiting: 3 r=[5, 7, 8, 9, 11] w=[11] x=[] (fullness=7/0)
c :   Ready: 3 r=[] w=[11] x=[]
c : mux wrote: 15/15
c : Waiting: 3 r=[5, 7, 8, 9, 11] w=[] x=[] (fullness=7/0)
 s:  > channel=0 cmd=ROUTES len=0 (fullness=7)
 s: Waiting: 1 r=[0] w=[1] x=[] (fullness=7/0)
 s:   Ready: 1 r=[] w=[1] x=[]
 s: mux wrote: 15/15
c :   Ready: 3 r=[11] w=[] x=[]
c : <  channel=0 cmd=PING len=7
c :  > channel=0 cmd=PONG len=7 (fullness=7)
c : mux wrote: 15/15
c : Waiting: 3 r=[5, 7, 8, 9, 11] w=[] x=[] (fullness=14/0)
 s: Waiting: 1 r=[0] w=[1] x=[] (fullness=7/0)
 s:   Ready: 1 r=[] w=[1] x=[]
 s: mux wrote: 8/8
 s: Waiting: 1 r=[0] w=[] x=[] (fullness=7/0)
 s:   Ready: 1 r=[0] w=[] x=[]
 s: <  channel=0 cmd=PING len=7
 s:  > channel=0 cmd=PONG len=7 (fullness=7)
 s: mux wrote: 15/15
c :   Ready: 3 r=[11] w=[] x=[]
c : <  channel=0 cmd=ROUTES len=0
fw: Got subnets: [(2, 16, False, '10.1.0.0', 0, 0), (2, 32, True, '127.0.0.1', 0, 0), (30, 128, True, '::1', 0, 0)]
fw: Got partial nslist: [(2, '192.168.86.1')]
fw: Got nslist: [(2, '192.168.86.1')]
fw: Got ports: 12300,12300,12299,12299
fw: Got udp: False, user: None
fw: setting up.
fw: setting up IPv6.
fw: >> pfctl -s Interfaces -i lo -v
 s: Waiting: 1 r=[0] w=[] x=[] (fullness=14/0)
 s:   Ready: 1 r=[0] w=[] x=[]
fw: >> pfctl -s all
 s: <  channel=0 cmd=PONG len=7
 s: received PING response
 s: Waiting: 1 r=[0] w=[] x=[] (fullness=0/0)
fw: rules:
---> pass out inet6 proto tcp to ::1/128
fw: >> pfctl -a sshuttle6-12300 -f /dev/stdin
fw: >> pfctl -E
fw: setting up IPv4.
fw: >> pfctl -s Interfaces -i lo -v
fw: >> pfctl -s all
fw: rules:
---> table <dns_servers> {192.168.86.1}
---> rdr pass on lo0 inet proto tcp from ! 127.0.0.1 to 10.1.0.0/16 -> 127.0.0.1 port 12300
---> rdr pass on lo0 inet proto udp to <dns_servers> port 53 -> 127.0.0.1 port 12299
---> pass out route-to lo0 inet proto tcp to 10.1.0.0/16 keep state
---> pass out inet proto tcp to 127.0.0.1/32
---> pass out route-to lo0 inet proto udp to <dns_servers> port 53 keep state
fw: >> pfctl -a sshuttle-12300 -f /dev/stdin
fw: >> pfctl -E
c : <  channel=0 cmd=PONG len=7
c : received PING response
c : Waiting: 3 r=[5, 7, 8, 9, 11] w=[] x=[] (fullness=0/0)

[skuhl]

There are at least two issues here. This response is directed at @mschindl and those getting: expected server init string b'SSHUTTLE0001'; got b''

Does something like this print the python version on the server? ssh dev-bastion "python3 -V"

If you don't have python3 on the server, using the recently released sshuttle 1.0.5 should work with either python2 or python3 on the server.

If you can get the ssh command above to print the ssh version, sshuttle should then work if you take python3 from the previous command (or whatever it is that you need to use to make it work) and pass it into sshuttle "--python python3".

[skuhl]

For the other comments about sshuttle not working with VPN in MacOS Big Sur. This isn't something that I've tried before. But, I can confirm that when I enable a wireguard vpn connection, then sshuttle (1.0.5) doesn't work in a way similar to the original post. I didn't try other versions of sshuttle or investigate what was going on though.

It isn't terribly surprising to me that a sshuttle VPN running inside of SSH inside of another VPN might be a problem. I also wonder if it is sshuttle that broke something or a MacOS update? If it was a sshuttle change that might have broken it, then it seems likely the problem would be linked to one of these commits: https://github.com/sshuttle/sshuttle/commits/master/sshuttle/methods/pf.py

[skuhl]

Are people also encountering this problem on other platforms? All reports here are related to macOS. If it is limited to macOS, it might make sense to modify the issue title to make that clear.

[cunnie]

@skuhl : Thanks for untangling the different errors. Let me clarify my error:

• sshuttle does not work for me on macOS Big Sur with the Global Protect 5.1.5-20 client when connecting to linux jumpboxes on the corporate VPN. Using sshuttle 1.0.5 also does not work (edit: installing from Homebrew does work, but the pre-release didn't work).
• sshuttle works fine on macOS Catalina with an earlier Global Protect 5.1.1-12 client when connecting to the same linux jumpboxes on the corporate VPN. My teammates have not updated to macOS Big Sur, so sshuttle works for them.
• I am not getting the error expected server init string b'SSHUTTLE0001'; as you pointed out, that's a different class of error.

[mschindl]

There are at least two issues here. This response is directed at @mschindl and those getting: expected server init string b'SSHUTTLE0001'; got b''

Does something like this print the python version on the server? ssh dev-bastion "python3 -V"

If you don't have python3 on the server, using the recently released sshuttle 1.0.5 should work with either python2 or python3 on the server.

It's now working with this version. You are right, python3 is not installed. Netstat is also not installed, but it seems not to be an issue, except the error message "hostwatch: ['netstat', '-n'] failed: OSError(2, 'No such file or directory')"

Thank you

[skuhl]

@mschindl You must also be using the --auto-host option? As the message indicates, sshuttle cannot find netstat on the server. Is it in the default PATH when you log into the remote server? Do you get this error with 1.0.5 too (1.0.5 includes additional paths to search for binaries in hopes of resolving problems just like this one). Although the --auto-host man page only indicates that --auto-host only has to do with /etc/hosts, it also uses a few other tricks to get IPs and their matching hostnames (such as using netstat).

[mschindl]

@skuhl : I'm using following command and got the message with version 1.0.5
sshuttle --dns -r root@<FGDN>:5050 -v 172.20.40.0/24 --auto-hosts
netstat is not installed, because it was not necessary on this server (RHEL7.7) up to now.
Currently it's working for me without installing python3 or netstat.

Thank you!

[cunnie]

Thanks for fixing

sshuttle version 1.0.5 works fine with the GlobalProtect VPN on macOS Big Sur. Version 1.0.4 didn't work.

[jswainston]

I'm also seeing the same issue

macOS Big Sur 11.1
GlobalProtect Version 5.1.6-16 also tried openconnnect with same result
sshuttle 1.0.5 (installed using brew and also tried with pip)

@cunnie you say its started working for you. any pointers?