Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Remote-SSH Bug]: reconnection logic is flaky after host sleep #9602

Open
3 tasks done
JamyDev opened this issue Feb 27, 2024 · 4 comments
Open
3 tasks done

[Remote-SSH Bug]: reconnection logic is flaky after host sleep #9602

JamyDev opened this issue Feb 27, 2024 · 4 comments
Assignees
@connor4312
Labels
bug Issue identified by VS Code Team member as probable bug ssh Issue in vscode-remote SSH

Comments

@JamyDev
Copy link

JamyDev commented Feb 27, 2024 •
edited
Loading

Is there an existing issue for this bug?

  • I have searched the existing issues

Required Troubleshooting Steps

  • I have followed these troubleshooting steps
  • I have tried both values of the remote.SSH.useLocalServer setting
    Repro case only when:
"remote.SSH.useExecServer": false,
"remote.SSH.useLocalServer": true,

Connect Locally

It connects successfully

->

VS Code reconnects, but warns that it failed to forward a port (I think the error message is stale after the reconnect succeeded though)
image

Expected Behavior

VS Code reconnects without errors

Steps To Reproduce

useExecServer enabled, after laptop sleep (on Mac OS 14.3.1), when VS Code tries to reconnect it hits some initial connection trouble, and throws an error message, but seems to have reconnected fine.

Remote-SSH Log

Remote-SSH Log 

[07:24:19.570] Log Level: 2
[07:24:19.576] VS Code version: 1.86.2
[07:24:19.576] Remote-SSH version: [email protected]
[07:24:19.576] darwin arm64
[07:24:19.577] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 1
[07:24:19.577] "remote.SSH.useLocalServer": true
[07:24:19.577] "remote.SSH.useExecServer": false
[07:24:19.578] "remote.SSH.path": undefined
[07:24:19.578] "remote.SSH.configFile": undefined
[07:24:19.578] "remote.SSH.useFlock": true
[07:24:19.578] "remote.SSH.lockfilesInTmp": false
[07:24:19.578] "remote.SSH.localServerDownload": auto
[07:24:19.578] "remote.SSH.remoteServerListenOnSocket": false
[07:24:19.578] "remote.SSH.showLoginTerminal": false
[07:24:19.578] "remote.SSH.defaultExtensions": []
[07:24:19.578] "remote.SSH.loglevel": 2
[07:24:19.579] "remote.SSH.maxReconnectionAttempts": 2
[07:24:19.579] "remote.SSH.enableDynamicForwarding": true
[07:24:19.579] "remote.SSH.enableRemoteCommand": false
[07:24:19.579] "remote.SSH.serverPickPortsFromRange": {}
[07:24:19.579] "remote.SSH.serverInstallPath": {}
[07:24:19.586] SSH Resolver called for host: my-secret-host.example.com
[07:24:19.586] Setting up SSH remote "my-secret-host.example.com"
[07:24:19.587] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[07:24:19.588] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:24:19.588] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[07:24:19.590] Install and start server if needed
[07:24:19.593] PATH: /Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin:/Users/jamy/Downloads/google-cloud-sdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/X11/bin:/usr/local/munki:/Users/jamy/Library/Application Support/JetBrains/Toolbox/scripts
[07:24:19.594] Checking ssh with "ssh -V"
[07:24:19.606] > OpenSSH_9.4p1, LibreSSL 3.3.6

[07:24:19.609] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-fb3e0c6f8aeee03fb15300ac5a89921fb06b71a3.sock
[07:24:19.609] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-f38407e9ef27dcbd8befa5a669cad99255cf8a5a.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","56576","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[07:24:19.609] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-fb3e0c6f8aeee03fb15300ac5a89921fb06b71a3.sock"}
[07:24:19.609] Spawned 98339
[07:24:19.716] > local-server-1> Running ssh connection command: ssh -v -T -D 56576 -o ConnectTimeout=45 my-secret-host.example.com bash
[07:24:19.717] > local-server-1> Spawned ssh, pid=98346
[07:24:19.725] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[07:24:19.909] stderr> found cert valid for 19h52m19s
[07:24:20.874] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[07:24:20.875] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[07:24:21.193] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[07:24:21.563] > ready: 22c625997811
[07:24:21.673] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[07:24:21.674] Platform: linux
[07:24:21.785] > /bin/zsh
[07:24:21.785] Shell: zsh
[07:24:21.785] Creating bash subshell inside "zsh"
[07:24:21.901] > bash version:  5.0.3(1)-release
[07:24:21.956] > bash version:  5.0.3(1)-release
[07:24:22.002] > bash version:  5.0.3(1)-release
> 22c625997811: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 324597
[07:24:22.010] > Starting VS Code Server... "/home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/bin/code-server" --start-server --host=127.0.0.1 --accept-server-license-terms --enable-remote-auto-shutdown --port=0 --telemetry-level all   &> "/home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log" < /dev/null
> The VS Code Server log file is /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log
> printenv:
>     SHELL=/bin/zsh
>     SSH_AUTH_SOCK=/tmp/ssh-jDMjGGE5uU/agent.2541249
>     PWD=/home/user
>     LOGNAME=jamy
>     HOME=/home/user
>     DEVPOD_ENVIRONMENT=kubernetes
>     VSCODE_AGENT_FOLDER=/home/user/.vscode-server
>     SSH_CONNECTION=some-ip 34774 some-other-ip 22
>     USER=jamy
>     SHLVL=1
>     SSH_CLIENT=some-ip 34774 22
>     PATH=/usr/local/bin:/usr/bin:/bin:/usr/games
>     MAIL=/var/mail/jamy
>     OLDPWD=/home/user
>     _=/usr/bin/printenv
> Removing old logfile at /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log
[07:24:22.086] > Spawned remote server: 2541306
> Waiting for server log...
> Waiting for server log...
[07:24:22.122] > Waiting for server log...
>  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
> 22c625997811: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime==113==
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 22c625997811: end
[07:24:22.122] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime==113==
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[07:24:22.123] Remote server is listening on port 33771
[07:24:22.123] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"serverStartTime":113,"installUnpackCode":""}
[07:24:22.123] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:24:22.126] Starting forwarding server. local port 56592 -> socksPort 56576 -> remotePort 33771
[07:24:22.126] Forwarding server listening on port 56592
[07:24:22.126] Waiting for ssh tunnel to be ready
[07:24:22.127] Tunneled port 33771 to local port 56592
[07:24:22.127] Resolved "ssh-remote+my-secret-host.example.com" to "port 56592"
[07:24:22.127] [Forwarding server port 56592] Got connection 0
[07:24:22.133] ------




[07:24:22.142] [Forwarding server port 56592] Got connection 1
[07:24:22.365] [Forwarding server port 56592] Got connection 2
[07:24:23.172] [Forwarding server port 56592] Got connection 3
[07:24:23.178] [Forwarding server port 56592] Got connection 4
[07:24:23.184] [Forwarding server port 56592] Got connection 5
[07:24:23.247] [Forwarding server port 56592] Got connection 6
[07:24:28.465] [Forwarding server port 56592] Got connection 7
[07:28:37.561] [Forwarding server port 56592] Got connection 8
[07:36:55.067] [Forwarding server port 56592] Got connection 9
[07:36:55.068] ------




[07:36:55.069] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 2, (Reconnection)
[07:36:55.071] SSH Resolver called for host: my-secret-host.example.com
[07:36:55.072] Setting up SSH remote "my-secret-host.example.com"
[07:36:55.072] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[07:36:55.073] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:36:55.073] Found local server running: {"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa","pid":98339,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-f38407e9ef27dcbd8befa5a669cad99255cf8a5a.sock","socksPort":56576,"startupTime":1709047459609}
[07:36:55.073] Running server is stale. Ignoring
[07:36:55.074] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[07:36:55.076] Install and start server if needed
[07:36:55.080] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-bbe76bb4f0a6b751f44a92b2c31218d3f03731ad.sock
[07:36:55.080] Spawning local server with {"serverId":2,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-7e0206a8a53e20adf76cbc465f118549f3d8840f.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","57472","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[07:36:55.081] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-bbe76bb4f0a6b751f44a92b2c31218d3f03731ad.sock"}
[07:36:55.081] Spawned 98567
[07:36:55.193] > local-server-2> Running ssh connection command: ssh -v -T -D 57472 -o ConnectTimeout=45 my-secret-host.example.com bash
[07:36:55.195] > local-server-2> Spawned ssh, pid=98574
[07:36:55.203] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[07:36:55.363] stderr> found cert valid for 19h39m43s
[07:36:56.241] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[07:36:56.241] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[07:36:56.556] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[07:36:56.901] > ready: 2f4f894de11d
[07:36:57.003] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[07:36:57.004] Platform: linux
[07:36:57.107] > /bin/zsh
[07:36:57.107] Shell: zsh
[07:36:57.107] Creating bash subshell inside "zsh"
[07:36:57.212] > bash version:  5.0.3(1)-release
[07:36:57.262] > bash version:  5.0.3(1)-release
[07:36:57.304] > bash version:  5.0.3(1)-release
> 2f4f894de11d: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
[07:36:57.315] >  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
[07:36:57.364] > Checking server status on port 33771 with wget
> 2f4f894de11d: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 2f4f894de11d: end
[07:36:57.365] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[07:36:57.366] Remote server is listening on port 33771
[07:36:57.367] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[07:36:57.369] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:36:57.382] Starting forwarding server. local port 57497 -> socksPort 57472 -> remotePort 33771
[07:36:57.382] Forwarding server listening on port 57497
[07:36:57.383] Waiting for ssh tunnel to be ready
[07:36:57.387] [Forwarding server port 57497] Got connection 0
[07:36:57.389] Tunneled port 33771 to local port 57497
[07:36:57.389] Resolved "ssh-remote+my-secret-host.example.com" to "port 57497"
[07:36:57.396] ------




[07:36:57.426] [Forwarding server port 57497] Got connection 1
[07:36:57.862] [Forwarding server port 57497] Got connection 2
[07:36:58.411] > local-server-1> Timed out
[07:36:58.429] Local server exit: 0
[07:36:58.430] [Forwarding server port 57497] Got connection 3
[07:37:19.211] Server delay-shutdown request failed: connect ECONNREFUSED /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-7e0206a8a53e20adf76cbc465f118549f3d8840f.sock
[07:37:19.212] > local-server-2> Timed out
[07:37:19.213] Local server exit: 0
[07:37:19.238] ------




[07:37:19.238] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 3, (Reconnection)
[07:37:19.240] SSH Resolver called for host: my-secret-host.example.com
[07:37:19.241] Setting up SSH remote "my-secret-host.example.com"
[07:37:19.241] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[07:37:19.242] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:37:19.242] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[07:37:19.245] Install and start server if needed
[07:37:19.249] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-5c44459df760077451c9700b8adc155b5f209a87.sock
[07:37:19.249] Spawning local server with {"serverId":3,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-9f89fa9020197af469c611c65924545d43cedd70.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","57512","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[07:37:19.249] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-5c44459df760077451c9700b8adc155b5f209a87.sock"}
[07:37:19.250] Spawned 98591
[07:37:19.519] > local-server-3> Running ssh connection command: ssh -v -T -D 57512 -o ConnectTimeout=45 my-secret-host.example.com bash
[07:37:19.520] > local-server-3> Spawned ssh, pid=98601
[07:37:19.534] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[07:37:19.821] stderr> found cert valid for 19h39m19s
[07:37:20.892] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[07:37:20.892] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[07:37:21.232] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[07:37:21.598] > ready: a2c88a15bd68
[07:37:21.706] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[07:37:21.706] Platform: linux
[07:37:21.816] > /bin/zsh
[07:37:21.820] Shell: zsh
[07:37:21.820] Creating bash subshell inside "zsh"
[07:37:21.926] > bash version:  5.0.3(1)-release
[07:37:21.983] > bash version:  5.0.3(1)-release
[07:37:22.024] > bash version:  5.0.3(1)-release
> a2c88a15bd68: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
[07:37:22.047] >  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
> Checking server status on port 33771 with wget
> a2c88a15bd68: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> a2c88a15bd68: end
[07:37:22.049] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[07:37:22.049] Remote server is listening on port 33771
[07:37:22.050] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[07:37:22.055] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[07:37:22.070] Starting forwarding server. local port 57527 -> socksPort 57512 -> remotePort 33771
[07:37:22.070] Forwarding server listening on port 57527
[07:37:22.070] Waiting for ssh tunnel to be ready
[07:37:22.071] Tunneled port 33771 to local port 57527
[07:37:22.071] Resolved "ssh-remote+my-secret-host.example.com" to "port 57527"
[07:37:22.072] [Forwarding server port 57527] Got connection 0
[07:37:22.075] ------




[07:37:22.092] [Forwarding server port 57527] Got connection 1
[07:37:23.066] [Forwarding server port 57527] Got connection 2
[07:37:41.087] [Forwarding server port 57527] Got connection 3
[08:01:18.914] [Forwarding server port 57527] Got connection 4
[08:01:18.917] ------




[08:01:18.917] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 4, (Reconnection)
[08:01:18.922] SSH Resolver called for host: my-secret-host.example.com
[08:01:18.922] Setting up SSH remote "my-secret-host.example.com"
[08:01:18.923] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[08:01:18.924] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:01:18.925] Found local server running: {"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa","pid":98591,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-9f89fa9020197af469c611c65924545d43cedd70.sock","socksPort":57512,"startupTime":1709048239250}
[08:01:18.925] Running server is stale. Ignoring
[08:01:18.926] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[08:01:18.928] Install and start server if needed
[08:01:18.934] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-812cca84e66854fee1e72e39a438b91cdc6b3f39.sock
[08:01:18.934] Spawning local server with {"serverId":4,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-35cbe4fb97c532547f8327e71da5bb9f6b201032.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","59362","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[08:01:18.934] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-812cca84e66854fee1e72e39a438b91cdc6b3f39.sock"}
[08:01:18.936] Spawned 99253
[08:01:19.132] > local-server-4> Running ssh connection command: ssh -v -T -D 59362 -o ConnectTimeout=45 my-secret-host.example.com bash
[08:01:19.133] > local-server-4> Spawned ssh, pid=99261
[08:01:19.143] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[08:01:19.335] stderr> found cert valid for 19h15m19s
[08:01:20.234] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[08:01:20.235] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[08:01:20.547] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[08:01:20.887] > ready: 2e879f66ff4e
[08:01:20.999] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[08:01:21.000] Platform: linux
[08:01:21.098] > /bin/zsh
[08:01:21.098] Shell: zsh
[08:01:21.098] Creating bash subshell inside "zsh"
[08:01:21.202] > bash version:  5.0.3(1)-release
[08:01:21.254] > bash version:  5.0.3(1)-release
[08:01:21.307] > bash version:  5.0.3(1)-release
> 2e879f66ff4e: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
>  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
> Checking server status on port 33771 with wget
[08:01:21.356] > 2e879f66ff4e: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 2e879f66ff4e: end
[08:01:21.357] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[08:01:21.357] Remote server is listening on port 33771
[08:01:21.358] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[08:01:21.358] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:01:21.362] Starting forwarding server. local port 59389 -> socksPort 59362 -> remotePort 33771
[08:01:21.363] Forwarding server listening on port 59389
[08:01:21.363] Waiting for ssh tunnel to be ready
[08:01:21.364] Tunneled port 33771 to local port 59389
[08:01:21.364] Resolved "ssh-remote+my-secret-host.example.com" to "port 59389"
[08:01:21.365] [Forwarding server port 59389] Got connection 0
[08:01:21.369] ------




[08:01:21.385] [Forwarding server port 59389] Got connection 1
[08:01:21.388] [Forwarding server port 59389] Got connection 2
[08:01:23.233] > local-server-3> Timed out
[08:01:23.242] [Forwarding server port 59389] Got connection 3
[08:01:23.247] Local server exit: 0
[08:03:03.897] Server delay-shutdown request failed: connect ECONNREFUSED /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-35cbe4fb97c532547f8327e71da5bb9f6b201032.sock
[08:03:03.898] > local-server-4> Timed out
[08:03:03.903] ------




[08:03:03.903] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 5, (Reconnection)
[08:03:03.903] Local server exit: 0
[08:03:03.904] SSH Resolver called for host: my-secret-host.example.com
[08:03:03.904] Setting up SSH remote "my-secret-host.example.com"
[08:03:03.905] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[08:03:03.905] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:03:03.906] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[08:03:03.907] Install and start server if needed
[08:03:03.910] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-3ebca69dc315f8d4be786a7e8a1f4dd968f043bf.sock
[08:03:03.910] Spawning local server with {"serverId":5,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-e5eb00b37eb1ab7d102f5f4dc126618968b7931d.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","59405","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[08:03:03.910] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-3ebca69dc315f8d4be786a7e8a1f4dd968f043bf.sock"}
[08:03:03.911] Spawned 99278
[08:03:04.024] > local-server-5> Running ssh connection command: ssh -v -T -D 59405 -o ConnectTimeout=45 my-secret-host.example.com bash
[08:03:04.025] > local-server-5> Spawned ssh, pid=99285
[08:03:04.033] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[08:03:04.459] stderr> found cert valid for 19h13m34s
[08:03:05.416] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[08:03:05.416] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[08:03:05.782] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[08:03:06.113] > ready: 66500bc73bd4
[08:03:06.217] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[08:03:06.217] Platform: linux
[08:03:06.318] > /bin/zsh
[08:03:06.319] Shell: zsh
[08:03:06.319] Creating bash subshell inside "zsh"
[08:03:06.422] > bash version:  5.0.3(1)-release
[08:03:06.473] > bash version:  5.0.3(1)-release
[08:03:06.523] > 66500bc73bd4: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
>  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
[08:03:06.573] > Checking server status on port 33771 with wget
> 66500bc73bd4: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 66500bc73bd4: end
[08:03:06.574] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[08:03:06.575] Remote server is listening on port 33771
[08:03:06.575] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[08:03:06.576] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:03:06.581] Starting forwarding server. local port 59447 -> socksPort 59405 -> remotePort 33771
[08:03:06.581] Forwarding server listening on port 59447
[08:03:06.581] Waiting for ssh tunnel to be ready
[08:03:06.582] Tunneled port 33771 to local port 59447
[08:03:06.582] Resolved "ssh-remote+my-secret-host.example.com" to "port 59447"
[08:03:06.583] [Forwarding server port 59447] Got connection 0
[08:03:06.599] ------




[08:03:06.622] [Forwarding server port 59447] Got connection 1
[08:03:06.826] [Forwarding server port 59447] Got connection 2
[08:03:06.945] [Forwarding server port 59447] Got connection 3
[08:22:31.951] [Forwarding server port 59447] Got connection 4
[08:26:28.545] [Forwarding server port 59447] Got connection 5
[08:26:28.548] ------




[08:26:28.548] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 6, (Reconnection)
[08:26:28.551] SSH Resolver called for host: my-secret-host.example.com
[08:26:28.551] Setting up SSH remote "my-secret-host.example.com"
[08:26:28.552] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[08:26:28.553] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:26:28.555] Found local server running: {"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa","pid":99278,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-e5eb00b37eb1ab7d102f5f4dc126618968b7931d.sock","socksPort":59405,"startupTime":1709049783911}
[08:26:28.555] Running server is stale. Ignoring
[08:26:28.555] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[08:26:28.557] Install and start server if needed
[08:26:28.561] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-36ac821a8f3bb882e9a0e22f01e200d8ea65e42e.sock
[08:26:28.561] Spawning local server with {"serverId":6,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-44326bb7909f3e04a4aa49f991da2a3f2aaf41ca.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","61264","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[08:26:28.561] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-36ac821a8f3bb882e9a0e22f01e200d8ea65e42e.sock"}
[08:26:28.562] Spawned 394
[08:26:28.681] > local-server-6> Running ssh connection command: ssh -v -T -D 61264 -o ConnectTimeout=45 my-secret-host.example.com bash
[08:26:28.682] > local-server-6> Spawned ssh, pid=401
[08:26:28.689] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[08:26:28.872] stderr> found cert valid for 18h50m10s
[08:26:29.849] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[08:26:29.849] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[08:26:30.185] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[08:26:30.532] > ready: 598c6e8688ca
[08:26:30.643] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[08:26:30.644] Platform: linux
[08:26:30.755] > /bin/zsh
[08:26:30.756] Shell: zsh
[08:26:30.757] Creating bash subshell inside "zsh"
[08:26:30.870] > bash version:  5.0.3(1)-release
[08:26:30.930] > bash version:  5.0.3(1)-release
[08:26:30.971] > bash version:  5.0.3(1)-release
> 598c6e8688ca: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
[08:26:30.978] >  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
[08:26:31.033] > Checking server status on port 33771 with wget
> 598c6e8688ca: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 598c6e8688ca: end
[08:26:31.033] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[08:26:31.034] Remote server is listening on port 33771
[08:26:31.034] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[08:26:31.036] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:26:31.043] Starting forwarding server. local port 61294 -> socksPort 61264 -> remotePort 33771
[08:26:31.044] Forwarding server listening on port 61294
[08:26:31.044] Waiting for ssh tunnel to be ready
[08:26:31.046] Tunneled port 33771 to local port 61294
[08:26:31.046] Resolved "ssh-remote+my-secret-host.example.com" to "port 61294"
[08:26:31.048] [Forwarding server port 61294] Got connection 0
[08:26:31.064] ------




[08:26:31.099] [Forwarding server port 61294] Got connection 1
[08:26:31.892] > local-server-5> Timed out
[08:26:31.908] Local server exit: 0
[08:26:31.909] [Forwarding server port 61294] Got connection 2
[08:26:31.943] [Forwarding server port 61294] Got connection 3
[08:26:55.941] Server delay-shutdown request failed: connect ECONNREFUSED /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-44326bb7909f3e04a4aa49f991da2a3f2aaf41ca.sock
[08:26:55.941] > local-server-6> Timed out
[08:26:55.943] Local server exit: 0
[08:26:55.946] [Forwarding server port 61294] Got connection 4
[08:26:55.947] ------




[08:26:55.947] SSH Resolver called for "ssh-remote+my-secret-host.example.com", attempt 7, (Reconnection)
[08:26:55.948] Failed to set up socket for dynamic port forward to remote port 33771: connect ECONNREFUSED 127.0.0.1:61264. TCP port forwarding may be disabled, or the remote server may have crashed. See the VS Code Server log above for details.
[08:26:55.949] SSH Resolver called for host: my-secret-host.example.com
[08:26:55.949] Setting up SSH remote "my-secret-host.example.com"
[08:26:55.949] Acquiring local install lock: /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-remote-ssh-1e5873a0-install.lock
[08:26:55.950] Looking for existing server data file at /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:26:55.950] Using commit id "903b1e9d8990623e3d7da1df3d33db3e42d80eda" and quality "stable" for server
[08:26:55.951] Install and start server if needed
[08:26:55.953] askpass server listening on /var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-2ec6a1ad93802bdf78fc73625ebdb97def68272e.sock
[08:26:55.953] Spawning local server with {"serverId":7,"ipcHandlePath":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-fe7e676f58f4d223369ea05a63cfdbfaa0b72661.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","61315","-o","ConnectTimeout=45","my-secret-host.example.com","bash"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json"}
[08:26:55.953] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.XvhR14Wtc3/Listeners","SHELL":"/bin/zsh","DISPLAY":"/private/tmp/com.apple.launchd.1yJ7ichYWx/org.xquartz:0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/jamy/.vscode/extensions/ms-vscode-remote.remote-ssh-0.108.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/5m/rn5yhlzx6n3d0b8h2zkg4gvw0000gn/T/vscode-ssh-askpass-2ec6a1ad93802bdf78fc73625ebdb97def68272e.sock"}
[08:26:55.954] Spawned 421
[08:26:56.092] > local-server-7> Running ssh connection command: ssh -v -T -D 61315 -o ConnectTimeout=45 my-secret-host.example.com bash
[08:26:56.093] > local-server-7> Spawned ssh, pid=428
[08:26:56.107] stderr> OpenSSH_9.4p1, LibreSSL 3.3.6
[08:26:56.431] stderr> found cert valid for 18h49m42s
[08:26:57.478] stderr> debug1: Server host key: ssh-ed25519 SHA256:XXXXXXXX
[08:26:57.479] stderr> Warning: Permanently added 'my-secret-host.example.com' (ED25519) to the list of known hosts.
[08:26:57.818] stderr> Authenticated to my-secret-host.example.com (via proxy) using "publickey".
[08:26:58.171] > ready: 4e4af346bebb
[08:26:58.290] > Linux 5.15.133+ #1 SMP Sat Nov 4 10:20:54 UTC 2023
[08:26:58.291] Platform: linux
[08:26:58.404] > /bin/zsh
[08:26:58.404] Shell: zsh
[08:26:58.405] Creating bash subshell inside "zsh"
[08:26:58.515] > bash version:  5.0.3(1)-release
> bash version:  5.0.3(1)-release
[08:26:58.572] > bash version:  5.0.3(1)-release
[08:26:58.623] > 4e4af346bebb: running
> Acquiring lock on /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda/vscode-remote-lock.jamy.903b1e9d8990623e3d7da1df3d33db3e42d80eda
> Found existing installation at /home/user/.vscode-server/bin/903b1e9d8990623e3d7da1df3d33db3e42d80eda...
> Checking /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.log and /home/user/.vscode-server/.903b1e9d8990623e3d7da1df3d33db3e42d80eda.pid for a running server
> Looking for server with pid: 2541306
> Found running server...
>  
> *
> * Visual Studio Code Server
> *
> * By using the software, you agree to
> * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
> * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
> *
>  
[08:26:58.627] > Checking server status on port 33771 with wget
[08:26:58.683] > 4e4af346bebb: start
> DISPLAY====
> webUiAccessToken====
> listeningOn==33771==
> osReleaseId==debian==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/tmp==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==
> 4e4af346bebb: end
[08:26:58.689] Received install output: 
DISPLAY====
webUiAccessToken====
listeningOn==33771==
osReleaseId==debian==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/tmp==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==11a11111-1aaa-1aa1-aa11-11111111a1aa==

[08:26:58.689] Remote server is listening on port 33771
[08:26:58.690] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":33771},"osReleaseId":"debian","arch":"x86_64","webUiAccessToken":"","display":"","tmpDir":"/tmp","platform":"linux","connectionToken":"11a11111-1aaa-1aa1-aa11-11111111a1aa"},"installUnpackCode":""}
[08:26:58.694] Persisting server connection details to /Users/jamy/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-1e5873a0-903b1e9d8990623e3d7da1df3d33db3e42d80eda-0.108.0-tr/data.json
[08:26:58.700] Starting forwarding server. local port 61322 -> socksPort 61315 -> remotePort 33771
[08:26:58.704] Forwarding server listening on port 61322
[08:26:58.706] Waiting for ssh tunnel to be ready
[08:26:58.710] [Forwarding server port 61322] Got connection 0
[08:26:58.713] Tunneled port 33771 to local port 61322
[08:26:58.713] Resolved "ssh-remote+my-secret-host.example.com" to "port 61322"
[08:26:58.728] ------




[08:26:58.747] [Forwarding server port 61322] Got connection 1
[08:26:58.882] [Forwarding server port 61322] Got connection 2
[08:27:31.007] [Forwarding server port 61322] Got connection 3

Anything else?

I'm still digging around to see if any of the network monitoring tooling is interacting with anything here, but this was at least somewhat reproducible with useExecServer -> true, but not with it false.
@JamyDev JamyDev added the ssh Issue in vscode-remote SSH label Feb 27, 2024
@roblourens roblourens assigned connor4312 and unassigned roblourens Feb 27, 2024
@connor4312
Copy link
Member

Thanks for the issue -- you checked that you tried out both versions of remote.SSH.useLocalServer, does turning it off help this behavior? With it off you should not see that specific notification, but I'm wondering if the connection logic is still flakey without that.

@connor4312 connor4312 added the bug Issue identified by VS Code Team member as probable bug label Feb 27, 2024
@JamyDev
Copy link
Author

JamyDev commented Feb 29, 2024

Looks like I was toggling the wrong setting, repro case is:

"remote.SSH.useExecServer": false,
"remote.SSH.useLocalServer": true,

@connor4312
Copy link
Member

Ah, okay, so this is independent of the exec server. That makes sense off what I was seeing in the logs.

@JamyDev JamyDev changed the title [Remote-SSH Bug]: useExecServer reconnection logic is flaky after host sleep [Remote-SSH Bug]: reconnection logic is flaky after host sleep Feb 29, 2024
@connor4312 connor4312 mentioned this issue Apr 30, 2024
Closed
@glenn2wang
Copy link

in my case , when the network is broken, after the network recover, the forward port is not work。i can not visit the forwarded address in my location

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@connor4312 connor4312

Labels
bug Issue identified by VS Code Team member as probable bug ssh Issue in vscode-remote SSH
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@roblourens @JamyDev @connor4312 @glenn2wang