Skip to content

Commit e194e86

Browse files
committed
docs: add comprehensive testing documentation
- Add TESTING.md with complete overview of GitHub Actions testing suite - Document all 4 new workflows and their coverage areas - Detail v1.5.0 specific testing including version commands, cross-platform compatibility, and enhanced error handling - Explain platform coverage (macOS, Ubuntu, Linux containers) and test categories - Provide maintenance guidelines and usage instructions for contributors and maintainers - Document 823 lines of workflow configuration ensuring comprehensive quality assurance
1 parent 6fa1265 commit e194e86

File tree

1 file changed

+159
-0
lines changed

1 file changed

+159
-0
lines changed

TESTING.md

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# PHPVM GitHub Actions Testing Suite
2+
3+
## Overview
4+
5+
This document describes the comprehensive GitHub Actions testing suite set up for phpvm to ensure reliable functionality across multiple platforms and environments.
6+
7+
## Workflows Created
8+
9+
### 1. **Comprehensive Test** (`comprehensive-test.yml`)
10+
**Purpose**: Main testing workflow covering all core functionality
11+
**Triggers**: Push to main/development/fix/feature branches, PRs, weekly schedule
12+
**Platforms**: Ubuntu (latest, 20.04), macOS (latest, 12)
13+
14+
**Test Coverage**:
15+
- ✅ Syntax validation and ShellCheck analysis
16+
- ✅ All version commands (`phpvm version`, `phpvm --version`, `phpvm -v`)
17+
- ✅ Help and list commands
18+
- ✅ Integrated self-tests
19+
- ✅ PHP installation and version switching
20+
- ✅ System PHP switching with improved messaging
21+
-`.phpvmrc` auto-switching functionality
22+
- ✅ Error handling for invalid commands/versions
23+
- ✅ Cross-platform compatibility (macOS Homebrew + Linux package managers)
24+
- ✅ Shell compatibility testing (bash, zsh)
25+
- ✅ Performance benchmarking and memory usage
26+
27+
### 2. **Edge Cases** (`edge-cases.yml`)
28+
**Purpose**: Test edge cases and cross-platform compatibility issues
29+
**Triggers**: Push to main, PRs, manual dispatch
30+
31+
**Test Coverage**:
32+
- ✅ WSL simulation and bash-specific features
33+
- ✅ Multiple Linux distribution containers (Ubuntu, Debian, CentOS)
34+
- ✅ Package manager detection across different systems
35+
- ✅ Version command consistency verification
36+
- ✅ Enhanced error handling validation
37+
- ✅ Homebrew integration edge cases on macOS
38+
-`.phpvmrc` edge cases (missing, empty, invalid, whitespace)
39+
- ✅ Self-test robustness across multiple runs
40+
- ✅ Resource usage monitoring
41+
42+
### 3. **Installation Testing** (`installation-test.yml`)
43+
**Purpose**: Verify installation processes and scripts
44+
**Triggers**: Push to main, PRs, manual dispatch, releases
45+
46+
**Test Coverage**:
47+
- ✅ Installation script syntax validation
48+
- ✅ cURL-based installation testing
49+
- ✅ Installation verification and functionality checks
50+
- ✅ Uninstallation process testing
51+
- ✅ Cross-platform installation compatibility
52+
53+
### 4. **Release Testing** (`release-test.yml`)
54+
**Purpose**: Final verification for releases
55+
**Triggers**: Published releases, pre-releases, manual dispatch
56+
57+
**Test Coverage**:
58+
- ✅ Release version verification
59+
- ✅ All core commands functionality
60+
- ✅ Installation from release tags
61+
- ✅ Performance regression testing
62+
- ✅ Documentation verification (README, commands)
63+
- ✅ Changelog verification
64+
65+
### 5. **Existing Workflows** (Enhanced)
66+
- `test.yml`: Basic integrated self-tests
67+
- `use.yml`: PHP installation and usage workflow
68+
69+
## Key Testing Features
70+
71+
### **v1.5.0 Specific Testing**
72+
-**New Version Commands**: All three variants (`version`, `--version`, `-v`)
73+
-**Enhanced Cross-Platform Support**: WSL, Linux distributions
74+
-**Improved Error Handling**: Homebrew link failures, missing binaries
75+
-**System PHP Messaging**: Updated "Homebrew default PHP" messaging
76+
-**Bash Shebang Compatibility**: WSL and Linux shell compatibility
77+
-**Post-Install Validation**: Binary availability checks
78+
79+
### **Platform Coverage**
80+
- **macOS**: Latest and macOS-12 with Homebrew integration
81+
- **Ubuntu**: Latest and 20.04 LTS versions
82+
- **Linux Containers**: Debian, CentOS for broad compatibility
83+
- **Package Managers**: brew, apt, dnf, yum, pacman detection
84+
- **Shell Environments**: bash, zsh, dash (POSIX) compatibility
85+
86+
### **Test Categories**
87+
1. **Functional Tests**: All commands work as expected
88+
2. **Integration Tests**: Installation, switching, auto-switching
89+
3. **Compatibility Tests**: Cross-platform, shell, distribution
90+
4. **Edge Case Tests**: Error conditions, malformed inputs
91+
5. **Performance Tests**: Speed, memory usage, regression
92+
6. **Installation Tests**: cURL, wget, verification, cleanup
93+
94+
## Workflow Execution
95+
96+
### **Automatic Triggers**
97+
- **Push** to main, development, fix/*, feature/* branches
98+
- **Pull Requests** to main, development
99+
- **Releases** (published, pre-released)
100+
- **Scheduled** (weekly on Sundays at 2 AM UTC)
101+
102+
### **Manual Triggers**
103+
- All workflows support `workflow_dispatch` for manual execution
104+
- Release testing can be triggered with custom version input
105+
106+
## Benefits
107+
108+
### **Quality Assurance**
109+
-**Multi-Platform Validation**: Ensures phpvm works on all supported systems
110+
-**Regression Prevention**: Catches issues before they reach users
111+
-**Edge Case Coverage**: Tests unusual scenarios and error conditions
112+
-**Performance Monitoring**: Tracks resource usage and execution speed
113+
114+
### **Development Confidence**
115+
-**Automated Validation**: Every change is thoroughly tested
116+
-**Cross-Platform Verification**: No platform-specific issues slip through
117+
-**Release Readiness**: Comprehensive validation before releases
118+
-**Documentation Sync**: Ensures docs match functionality
119+
120+
### **User Experience**
121+
-**Reliable Installation**: Installation methods are verified to work
122+
-**Cross-Platform Consistency**: Same experience across all platforms
123+
-**Error Handling**: Users get helpful error messages
124+
-**Performance**: Commands execute efficiently
125+
126+
## Maintenance
127+
128+
### **Workflow Updates**
129+
- Workflows are version-controlled and can be updated with phpvm changes
130+
- New features should add corresponding tests to relevant workflows
131+
- Platform versions should be updated periodically (Ubuntu LTS, macOS versions)
132+
133+
### **Monitoring**
134+
- GitHub Actions dashboard shows workflow status
135+
- Failed workflows require investigation and fixes
136+
- Performance trends can be monitored over time
137+
138+
## Usage
139+
140+
### **For Contributors**
141+
- All PRs automatically trigger comprehensive testing
142+
- Check workflow results before merging
143+
- Add new tests when adding new features
144+
145+
### **For Maintainers**
146+
- Monitor workflow health and update as needed
147+
- Use release workflows for final verification
148+
- Review performance trends and address regressions
149+
150+
### **For Users**
151+
- Badge status in README shows current test status
152+
- Release workflows ensure stable releases
153+
- Installation methods are continuously verified
154+
155+
## Conclusion
156+
157+
This comprehensive testing suite ensures phpvm v1.5.0 and future versions maintain high quality, reliability, and cross-platform compatibility. The automated workflows catch issues early, provide confidence in releases, and ensure a consistent user experience across all supported platforms.
158+
159+
The 823 lines of workflow configuration provide extensive coverage that would be impractical to test manually, making this an essential part of the phpvm development process.

0 commit comments

Comments
 (0)