Ianjennings/self hosted #466
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR implements comprehensive self-hosting infrastructure for TestDriver, enabling users to run tests on their own AWS EC2 instances instead of relying solely on the hosted sandbox service.
- Adds complete AWS infrastructure provisioning via CloudFormation template with VPC, security groups, and launch templates
- Introduces instance management automation through
aws-setup.shscript for launching, configuring, and validating EC2 instances - Implements direct IP connection capability in the CLI with new
--ipflag to bypass sandbox service
Reviewed Changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
docs/getting-started/self-hosting.mdx |
Complete documentation guide covering infrastructure setup, GitHub Actions integration, and troubleshooting |
docs/docs.json |
Adds self-hosting documentation to navigation structure |
cloudformation.yaml |
AWS CloudFormation template for VPC, security groups, IAM roles, and EC2 launch template creation |
aws-setup.sh |
Shell script for automated EC2 instance lifecycle management with SSM connectivity validation |
agent/lib/sandbox.js |
Adds direct() method for connecting to IP addresses without sandbox service |
agent/interface.js |
Adds --ip flag definition to run and explore commands |
agent/index.js |
Implements direct IP connection logic bypassing normal sandbox workflow |
.github/workflows/self-hosted.yml |
GitHub Actions workflow demonstrating complete self-hosted testing pipeline |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
agent/lib/sandbox.js
Outdated
| async direct(ip) { | ||
|
|
||
| let reply = await this.send({ | ||
| type: "direct" |
There was a problem hiding this comment.
The ip parameter is not being used in the message sent to the server. The function accepts an ip parameter but doesn't include it in the message payload, which will likely cause the direct connection to fail.
| type: "direct" | |
| type: "direct", | |
| ip |
Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <[email protected]>
…tdriverai into ianjennings/self-hosted
There was a problem hiding this comment.
This is looking pretty good!
Admittedly, almost all of the edits are docs/cosmetic.
The one thing I'm curious about is if it's possible to default to the runner IP by default. Secure by default should be our recommendation.: https://github.com/marketplace/actions/public-ip
| ## Prerequisites | ||
|
|
||
| - AWS account with appropriate permissions | ||
| - AWS CLI installed and configured |
There was a problem hiding this comment.
This is already installed on GitHub Runners, so maybe it's optional?
There was a problem hiding this comment.
needed for local dev ?
| --output text | ||
| ``` | ||
|
|
||
| Save this ID - you'll need it for the next step. |
There was a problem hiding this comment.
This happens in CI, so is this actually a step for the user?
There was a problem hiding this comment.
This doesn't happen in CI. The cloudformation is one-time use to setup security groups. Will clarify in docs.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Eric Clemmons <[email protected]>
Oh that would be cool! This + OIDC would be pretty dang secure. It probably is possible with aws cli, but let's do it later. |
Co-authored-by: Eric Clemmons <[email protected]>
… into ianjennings/self-hosted
Implemented server side with https://github.com/testdriverai/api/pull/44/files
cloudformation.yamlcontains a VPC formation that (primarily) creates a launch template with the proper open portsaws-setup.shprovisions our instances so they are ready for testdriver--ipflag enables cli to connect directly to an instance that has been provisionedself-hosted.ymlruns our acceptance tests using a recipe combining the above improvements