Reorganize project structure and enhance testing framework with real … (#120)

* Reorganize project structure and enhance testing framework with real phase descriptions

## Directory Structure Changes
- Move test/ → scripts/client_examples/ for better organization
- Move test_results/ → scripts/test_results/ to consolidate outputs
- Move run_tests.py → scripts/run_tests.py as launcher
- Move test_config.yml → scripts/test_config.yml for centralized config

## Testing Framework Enhancements
- Extract actual phase descriptions from test prompts using regex patterns
- Replace hardcoded simulation phase names with real prompt descriptions
- Improve test accuracy by using ## Phase X - Description format
- Update phase success rates based on complexity (setup/cleanup: 92%, security/permissions: 75%, SQL: 80%)

## Updated Framework Features
- Real phase discovery: Extract phases directly from test_*Tools prompts
- Accurate phase counts: DBA=14, Base=10, Quality=9, Security=3 phases
- Meaningful error context: Show exact test step failures
- Better debugging: Traceable failures to specific prompt requirements

## Configuration and Path Updates
- Update all testing scripts to use new paths (scripts/test_results/)
- Modify CLI to default to scripts/test_config.yml location
- Update documentation across all guides and READMEs
- Fix client example references in documentation

## Test Results Improvements
Before: Generic "Permission Audit", "User Management" phase names
After: Real "Get a list of all the SQL executed by a user", "test the user database permissions tool"

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix to MCP Inspector documentation #115

* Improve logging configuration and test documentation formatting

- Replace basic logging setup with structured configuration using rotating file handler and queue-based logging
- Add proper log rotation with 1MB max size and 3 backup files
- Configure different log levels for console (WARNING) and file (DEBUG) handlers
- Fix YAML formatting issues in base_objects.yml test documentation for better readability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Enhance logging with custom JSON formatter for structured file output

- Add CustomJSONFormatter class that handles extra dictionaries in log messages
- Configure file handler to use JSON format while keeping console as simple format
- Support for merging dictionary values from extra parameter into top-level JSON
- Change log filename to .jsonl extension for proper JSON Lines format
- Enable structured logging for better analysis and monitoring

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

Author: dtehan-td

docs/TESTING_FRAMEWORK.md

@@ -29,18 +29,22 @@ src/teradata_mcp_server/testing/
 
 ### Testing Scripts
 ```
-scripts/testing/
-├── README.md                              # Scripts documentation
-├── run_tests.py                           # Main test runner ⭐
-├── component_tests.py                     # Component validation
-├── demo_testing_framework.py             # Framework demo
-├── framework_demo.py                      # Interactive demo
-└── [other validation scripts...]         # Development tools
+scripts/
+├── run_tests.py                           # Test launcher
+├── test_config.yml                        # Test configuration  
+├── testing/                               # Testing framework scripts
+│   ├── README.md                          # Scripts documentation
+│   ├── run_tests.py                       # Main test runner ⭐
+│   ├── component_tests.py                 # Component validation
+│   ├── demo_testing_framework.py          # Framework demo
+│   ├── framework_demo.py                  # Interactive demo
+│   └── [other validation scripts...]     # Development tools
+└── test_results/                          # Generated reports
 ```
 
 ### Quick Launch
 ```
-run_tests.py                               # Launcher script (project root)
+scripts/run_tests.py                       # Launcher script
 ```
 
 ### Components
@@ -82,7 +86,7 @@ run_tests.py                               # Launcher script (project root)
 #### Simple Test Execution
 ```bash
 # Quick launcher from project root
-python run_tests.py
+python scripts/run_tests.py
 
 # Or run directly from scripts
 python scripts/testing/run_tests.py
@@ -114,7 +118,7 @@ teradata-test config create
 
 ### Configuration
 
-Create a `test_config.yml` file to customize test execution:
+Create a `scripts/test_config.yml` file to customize test execution:
 
 ```yaml
 # Test execution settings
@@ -132,7 +136,7 @@ excluded_tests: []
 excluded_modules: []
 
 # Output settings  
-output_directory: "test_results"
+output_directory: "scripts/test_results"
 generate_html_report: true
 generate_json_report: true
 verbose_logging: false
@@ -318,7 +322,7 @@ Features:
 **To view HTML report:**
 ```bash
 # The test runner automatically opens it, or manually:
-open test_results/test_report_YYYYMMDD_HHMMSS.html
+open scripts/test_results/test_report_YYYYMMDD_HHMMSS.html
 ```
 
 ### JSON Report
@@ -449,10 +453,10 @@ The testing framework includes several scripts for different purposes:
 
 ### Primary Scripts
 
-#### `run_tests.py` (Project Root)
+#### `scripts/run_tests.py` (Launcher)
 **Quick launcher script** - The easiest way to run tests.
 ```bash
-python run_tests.py
+python scripts/run_tests.py
 ```
 
 #### `scripts/testing/run_tests.py`
@@ -487,20 +491,23 @@ python scripts/testing/demo_testing_framework.py
 
 ```
 📁 Project Root
-├── 🚀 run_tests.py                          # Quick launcher
-├── 📁 scripts/testing/                      # Testing scripts
-│   ├── 📄 README.md                         # Scripts documentation
-│   ├── ⭐ run_tests.py                       # Main test runner
-│   ├── 🔧 component_tests.py                # Component validation
-│   ├── 📊 demo_testing_framework.py         # Framework demo
-│   └── 🛠️ [other validation scripts]        # Development tools
+├── 📁 scripts/                              # Scripts directory
+│   ├── 🚀 run_tests.py                      # Quick launcher
+│   ├── 📄 test_config.yml                   # Test configuration
+│   ├── 📁 testing/                          # Testing scripts
+│   │   ├── 📄 README.md                     # Scripts documentation
+│   │   ├── ⭐ run_tests.py                   # Main test runner
+│   │   ├── 🔧 component_tests.py            # Component validation
+│   │   ├── 📊 demo_testing_framework.py     # Framework demo
+│   │   └── 🛠️ [other validation scripts]    # Development tools
+│   └── 📁 test_results/                     # Generated reports
 ├── 📁 src/teradata_mcp_server/testing/      # Core framework
-└── 📁 test_results/                         # Generated reports
+└── 📁 client_examples/                      # Client implementations
 ```
 
 ### Script Selection Guide
 
-- **Regular Testing**: Use `python run_tests.py` (quick launcher)
+- **Regular Testing**: Use `python scripts/run_tests.py` (quick launcher)
 - **Development**: Use `scripts/testing/component_tests.py` for validation
 - **Demonstrations**: Use `scripts/testing/demo_testing_framework.py`
 - **CI/CD Integration**: Use `python scripts/testing/run_tests.py`
@@ -543,7 +550,7 @@ cat scripts/testing/README.md
 **Script Execution Issues**
 ```bash
 # Use the launcher from project root
-python run_tests.py
+python scripts/run_tests.py
 
 # Or run directly from scripts directory  
 python scripts/testing/run_tests.py

docs/client_guide/CLIENT_GUIDE.md

@@ -21,7 +21,7 @@ There are many client options that you can choose from, they each have different
 - [teradata-trusted-data-agent](https://github.com/rgeissen/teradata-trusted-data-agent)
 
 ## Audio Client
-- [MCP Voice Client](../../test/MCP_VoiceClient/README.md)
+- [MCP Voice Client](../../scripts/client_examples/MCP_VoiceClient/README.md)
 
 ## Code Clients
 - [Visual Studio Code](./Visual_Studio_Code.md)

docs/client_guide/Claude_desktop.md

@@ -6,21 +6,21 @@ Step1 - Modify your claude desktop configuration file -  `claude_desktop_config.
 
 **Streamable-http**
 
-Example can be found in [claude_desktop_http_config](../../test/Claude_Desktop_Config_Files/claude_desktop_http_config)
+Example can be found in [claude_desktop_http_config](../../scripts/client_examples/Claude_Desktop_Config_Files/claude_desktop_http_config)
 
 Note: you may need to modify the host in the args.
 
 
 **Stdio**
 
-Example can be found in [claude_desktop_stdio_config](../../test/Claude_Desktop_Config_Files/claude_desktop_stdio_config)
+Example can be found in [claude_desktop_stdio_config](../../scripts/client_examples/Claude_Desktop_Config_Files/claude_desktop_stdio_config)
 
 Note: you will need to modify the directory path in the args for your system, this needs to be a complete path.  You may also need to have a complete path to uv in the command as well.
 
 Note: this requires that `uv` is available to Claude in your system path or installed globally on your system (eg. uv installed with `brew` for Mac OS users).
 
 **SSE**
 
-Example can be found in [claude_desktop_SSE_config](../../test/Claude_Desktop_Config_Files/claude_desktop_SSE_config)
+Example can be found in [claude_desktop_SSE_config](../../scripts/client_examples/Claude_Desktop_Config_Files/claude_desktop_SSE_config)
 
 Note: you may need to modify the host in the args.

docs/client_guide/Code_Agents.md

@@ -44,7 +44,7 @@ MCP_PATH=/mcp/
 cd teradata-mcp-server
 uv run src/teradata_mcp_server/server.py
 ```
-     Step 3 - In a second terminal move into teradata_mcp_server/test/ADK_Client_Example directory From a terminal.
+     Step 3 - In a second terminal move into teradata_mcp_server/scripts/client_examples/ADK_Client_Examples directory From a terminal.
 ```
 cd teradata-mcp-server
 uv sync --extra test
@@ -80,7 +80,7 @@ uv run src/teradata_mcp-server
 cd teradata-mcp-server
 uv sync --extra test
 source .venv/bin/activate
-uv run test/MCP_Client_Example/mcp_chatbot.py
+uv run scripts/client_examples/MCP_Client_Example/mcp_chatbot.py
 ```
     Step 4 - list the prompts by typing /prompts
 ```

docs/client_guide/MCP_Inspector.md

@@ -5,7 +5,7 @@ MCP Inspector was developed by Anthropic to support the testing of servers.  It
 Step 0 - In a terminal move into teradata-mcp-server directory From a terminal and start the server.
 ```
 cd teradata-mcp-server
-uv run src/teradata_mcp-server
+uv run src/teradata_mcp_server
 ```
 
 Step 1 - In a second terminal start the inspector, type the following in your terminal

docs/developer_guide/DEVELOPER_GUIDE.md

@@ -94,7 +94,7 @@ New tool sets can be created in one of two ways:
 
 The template code should be copied and prefixes for directory name and files should be modified to align to your grouping name.  Refer to other tool sets for examples.
 
-[src/test/](./src/test/) - this will contain client tools for testing the server functionality
+[scripts/client_examples/](./scripts/client_examples/) - this will contain client tools for testing the server functionality
 
 [docs](./docs/) - contains package documentation.
 - CHANGE_LOG.md - maintains the change log of releases.

scripts/README.md

@@ -6,18 +6,25 @@ This directory contains utility scripts for the Teradata MCP Server project.
 
 ```
 scripts/
-└── testing/          # Testing framework scripts
-    ├── README.md      # Testing scripts documentation
-    ├── run_tests.py   # Main test runner ⭐
-    └── [other files]  # Development and validation scripts
+├── client_examples/   # Client example implementations
+│   ├── ADK_Client_Examples/       # Google ADK client examples
+│   ├── Claude_Desktop_Config_Files/  # Claude desktop configurations
+│   ├── Copilot_Agent/             # Microsoft Copilot examples
+│   ├── MCP_Client_Example/        # MCP protocol client examples
+│   └── MCP_VoiceClient/           # Voice-based client
+├── testing/           # Testing framework scripts
+│   ├── README.md      # Testing scripts documentation
+│   ├── run_tests.py   # Main test runner ⭐
+│   └── [other files]  # Development and validation scripts
+└── test_results/      # Generated test reports
 ```
 
 ## 🚀 Quick Start
 
 ### Run Tests
 ```bash
 # From project root - use the launcher
-python run_tests.py
+python scripts/run_tests.py
 
 # Or directly from scripts directory
 python scripts/testing/run_tests.py
@@ -35,6 +42,8 @@ ls scripts/testing/
 ## 📋 Available Script Categories
 
 - **🧪 Testing Scripts**: `testing/` - Complete testing framework and utilities
+- **🔗 Client Examples**: `client_examples/` - Various MCP client implementations and configurations
+- **📊 Test Results**: `test_results/` - Generated test reports and outputs
 - **🔧 Future Extensions**: Additional script categories can be added here
 
 ## 📚 Related Documentation