Add "ssh server" command

[ilia-db]

Changes

1. "ssh" and "ssh setup" commands PR: Add "ssh" and "ssh setup" commands #3470
2. "ssh connect" PR: Add "ssh connect" command #3471

This PR is based on the above, it adds "ssh server" command and related logic and finishes the "ssh" implementation by adding ssh command group to the root cmd (it's still hidden from the --help output).

Server overview:

• The server itself is started by the client running the ssh-server-bootsrap.py job
  • The client passes inputs to this file as jobs API args, and the python logic gets the values using dbutils widgets
  • One interesting part is setup_subreaper method, which ensures no child process leave the parent process group. This is necessary in order to prevent background processes being reparented to pid 1 when their parent ssh server session terminates, after which they loose access to wsfs and dbfs fuse mounts (as access is based on process groups). All major IDEs usually try to spawn their server processes as a background processes (by double forking for example).
• Server creates one sshd config for all future sshd processes in the home folder
• It spawns sshd with -i flag, making it run as in non-daemon mode and communicating with it through stdin and our (and so sshd doesn't listen on any ports)
• Separate sshd process is spawned for each new websocket connection (but the sshd config is re-used), there are limit of 10 connections (for no particular reason)
• The server start a shutdown timer when there are not active connections, after which it terminates itself, which terminates the job, and the cluster idle timeout can start ticking after that
• The server also places jupyter-init.py file to the $HOME/.ipython/profile_default/startup. This file makes jupyter kernels a bit more similar to webapp notebooks - it provides spark and dbutils globals, plus handles %sql and %pip magic commands

WIP: unit tests

Tests

Manual and unit

[eng-dev-ecosystem-bot]

Run: 17551494331

	Env	✅​pass	🙈​skip
✅​	aws linux	308	519
✅​	aws windows	309	518
✅​	aws-ucws linux	420	417
✅​	aws-ucws windows	421	416
✅​	azure linux	308	518
✅​	azure windows	309	517
✅​	azure-ucws linux	420	416
✅​	azure-ucws windows	421	415
✅​	gcp linux	307	520
✅​	gcp windows	308	519

[pietern]

Stamp to unblock review of the #3544.

[pietern]

Ack. The failure happens somewhere deep in the stack. It would be useful to make this error more explicit. If a user runs it and gets some error about a secret missing, that's confusing. If instead they see "This command is supposed to be run on Databricks compute. Please find usage instructions at <>.", that's better.