feat: Add auth_tools support and fix test infrastructure (#66)

* Add auth_tools field for selective authentication in OpenAPI tool generation

- Add auth_tools field to HttpCallTemplate for tool-specific authentication
- Implement compatibility checking between OpenAPI security schemes and auth_tools
- Apply real credentials when compatible, use placeholders when incompatible
- Preserve existing behavior for public endpoints (no auth required)
- Add comprehensive test coverage for all authentication scenarios
- Update documentation with auth_tools examples and usage
- Maintain full backward compatibility

* Update implementation files and documentation for auth_tools feature

- Update HttpCallTemplate, HttpCommunicationProtocol, and OpenApiConverter
- Add auth_tools examples to README.md
- Update existing tests for new auth_tools parameter
- Add integration test for auth_tools field functionality

* fix: resolve pytest fixture dependency issue in HTTP tests

- Fix aiohttp_client fixture usage by properly injecting app dependency
- Ensure all test fixtures receive required parameters correctly
- All 153 tests now pass without fixture conflicts

* feat: add auth_tools support to text plugin

- Add auth_tools field to TextCallTemplate for OpenAPI-generated tools
- Pass auth_tools to OpenApiConverter when processing local OpenAPI specs
- Update documentation to reflect new authentication capabilities
- Add test coverage for auth_tools functionality
- Maintains backward compatibility (auth_tools is optional)

This allows text plugin to apply authentication to tools generated from
local OpenAPI specifications, enabling secure API calls while keeping
file access authentication-free.

* fix: add proper serialization/validation for auth and auth_tools fields

- Add field_serializer and field_validator for auth_tools in TextCallTemplate
- Add field_serializer and field_validator for both auth and auth_tools in HttpCallTemplate
- Use AuthSerializer.validate_dict() for proper dict-to-Auth conversion
- Add comprehensive test coverage for auth_tools serialization
- Ensures dict configurations preserve all critical authentication fields
- All 155 tests pass with proper field validation

* Update README.md

---------

Co-authored-by: Razvan Radulescu <[email protected]>

Author: perrozzi

README.md

@@ -376,12 +376,18 @@ Configuration examples for each protocol. Remember to replace `provider_type` wi
   "url": "https://api.example.com/users/{user_id}", // Required
   "http_method": "POST", // Required, default: "GET"
   "content_type": "application/json", // Optional, default: "application/json"
-  "auth": { // Optional, example using ApiKeyAuth for a Bearer token. The client must prepend "Bearer " to the token.
+  "auth": { // Optional, authentication for the HTTP request (example using ApiKeyAuth for Bearer token)
     "auth_type": "api_key",
     "api_key": "Bearer $API_KEY", // Required
     "var_name": "Authorization", // Optional, default: "X-Api-Key"
     "location": "header" // Optional, default: "header"
   },
+  "auth_tools": { // Optional, authentication for converted tools, if this call template points to an openapi spec that should be automatically converted to a utcp manual (applied only to endpoints requiring auth per OpenAPI spec)
+    "auth_type": "api_key",
+    "api_key": "Bearer $TOOL_API_KEY", // Required
+    "var_name": "Authorization", // Optional, default: "X-Api-Key"
+    "location": "header" // Optional, default: "header"
+  },
   "headers": { // Optional
     "X-Custom-Header": "value"
   },
@@ -473,7 +479,13 @@ Note the name change from `http_stream` to `streamable_http`.
   "name": "my_text_manual",
   "call_template_type": "text", // Required
   "file_path": "./manuals/my_manual.json", // Required
-  "auth": null // Optional (always null for Text)
+  "auth": null, // Optional (always null for Text)
+  "auth_tools": { // Optional, authentication for generated tools from OpenAPI specs
+    "auth_type": "api_key",
+    "api_key": "Bearer ${API_TOKEN}",
+    "var_name": "Authorization",
+    "location": "header"
+  }
 }
 ```
 
@@ -569,7 +581,13 @@ client = await UtcpClient.create(config={
     "manual_call_templates": [{
         "name": "github",
         "call_template_type": "http", 
-        "url": "https://api.github.com/openapi.json"
+        "url": "https://api.github.com/openapi.json",
+        "auth_tools": {  # Authentication for generated tools requiring auth
+            "auth_type": "api_key",
+            "api_key": "Bearer ${GITHUB_TOKEN}",
+            "var_name": "Authorization",
+            "location": "header"
+        }
     }]
 })
 ```
@@ -579,6 +597,7 @@ client = await UtcpClient.create(config={
 - ✅ **Zero Infrastructure**: No servers to deploy or maintain
 - ✅ **Direct API Calls**: Native performance, no proxy overhead  
 - ✅ **Automatic Conversion**: OpenAPI schemas → UTCP tools
+- ✅ **Selective Authentication**: Only protected endpoints get auth, public endpoints remain accessible
 - ✅ **Authentication Preserved**: API keys, OAuth2, Basic auth supported
 - ✅ **Multi-format Support**: JSON, YAML, OpenAPI 2.0/3.0
 - ✅ **Batch Processing**: Convert multiple APIs simultaneously

plugins/communication_protocols/http/src/utcp_http/http_call_template.py

@@ -1,10 +1,10 @@
 from utcp.data.call_template import CallTemplate, CallTemplateSerializer
-from utcp.data.auth import Auth
+from utcp.data.auth import Auth, AuthSerializer
 from utcp.interfaces.serializer import Serializer
 from utcp.exceptions import UtcpSerializerValidationError
 import traceback
-from typing import Optional, Dict, List, Literal
-from pydantic import Field
+from typing import Optional, Dict, List, Literal, Any
+from pydantic import Field, field_serializer, field_validator
 
 class HttpCallTemplate(CallTemplate):
     """REQUIRED
@@ -40,6 +40,12 @@ class HttpCallTemplate(CallTemplate):
             "var_name": "Authorization",
             "location": "header"
           },
+          "auth_tools": {
+            "auth_type": "api_key",
+            "api_key": "Bearer ${TOOL_API_KEY}",
+            "var_name": "Authorization",
+            "location": "header"
+          },
           "headers": {
             "X-Custom-Header": "value"
           },
@@ -85,7 +91,8 @@ class HttpCallTemplate(CallTemplate):
         url: The base URL for the HTTP endpoint. Supports path parameters like
             "https://api.example.com/users/{user_id}/posts/{post_id}".
         content_type: The Content-Type header for requests.
-        auth: Optional authentication configuration.
+        auth: Optional authentication configuration for accessing the OpenAPI spec URL.
+        auth_tools: Optional authentication configuration for generated tools. Applied only to endpoints requiring auth per OpenAPI spec.
         headers: Optional static headers to include in all requests.
         body_field: Name of the tool argument to map to the HTTP request body.
         header_fields: List of tool argument names to map to HTTP request headers.
@@ -96,10 +103,49 @@ class HttpCallTemplate(CallTemplate):
     url: str
     content_type: str = Field(default="application/json")
     auth: Optional[Auth] = None
+    auth_tools: Optional[Auth] = Field(default=None, description="Authentication configuration for generated tools (applied only to endpoints requiring auth per OpenAPI spec)")
     headers: Optional[Dict[str, str]] = None
     body_field: Optional[str] = Field(default="body", description="The name of the single input field to be sent as the request body.")
     header_fields: Optional[List[str]] = Field(default=None, description="List of input fields to be sent as request headers.")
 
+    @field_serializer('auth')
+    def serialize_auth(self, auth: Optional[Auth]) -> Optional[dict]:
+        """Serialize auth to dictionary."""
+        if auth is None:
+            return None
+        return AuthSerializer().to_dict(auth)
+
+    @field_validator('auth', mode='before')
+    @classmethod
+    def validate_auth(cls, v: Any) -> Optional[Auth]:
+        """Validate and deserialize auth from dictionary."""
+        if v is None:
+            return None
+        if isinstance(v, Auth):
+            return v
+        if isinstance(v, dict):
+            return AuthSerializer().validate_dict(v)
+        raise ValueError(f"auth must be None, Auth instance, or dict, got {type(v)}")
+
+    @field_serializer('auth_tools')
+    def serialize_auth_tools(self, auth_tools: Optional[Auth]) -> Optional[dict]:
+        """Serialize auth_tools to dictionary."""
+        if auth_tools is None:
+            return None
+        return AuthSerializer().to_dict(auth_tools)
+
+    @field_validator('auth_tools', mode='before')
+    @classmethod
+    def validate_auth_tools(cls, v: Any) -> Optional[Auth]:
+        """Validate and deserialize auth_tools from dictionary."""
+        if v is None:
+            return None
+        if isinstance(v, Auth):
+            return v
+        if isinstance(v, dict):
+            return AuthSerializer().validate_dict(v)
+        raise ValueError(f"auth_tools must be None, Auth instance, or dict, got {type(v)}")
+
 
 class HttpCallTemplateSerializer(Serializer[HttpCallTemplate]):
     """REQUIRED

plugins/communication_protocols/http/src/utcp_http/http_communication_protocol.py

@@ -197,7 +197,7 @@ async def register_manual(self, caller, manual_call_template: CallTemplate) -> R
                             utcp_manual = UtcpManualSerializer().validate_dict(response_data)
                         else:
                             logger.info(f"Assuming OpenAPI spec from '{manual_call_template.name}'. Converting to UTCP manual.")
-                            converter = OpenApiConverter(response_data, spec_url=manual_call_template.url, call_template_name=manual_call_template.name)
+                            converter = OpenApiConverter(response_data, spec_url=manual_call_template.url, call_template_name=manual_call_template.name, auth_tools=manual_call_template.auth_tools)
                             utcp_manual = converter.convert()
 
                         return RegisterManualResult(

plugins/communication_protocols/http/src/utcp_http/openapi_converter.py

@@ -87,7 +87,7 @@ class OpenApiConverter:
         call_template_name: Normalized name for the call_template derived from the spec.
     """
 
-    def __init__(self, openapi_spec: Dict[str, Any], spec_url: Optional[str] = None, call_template_name: Optional[str] = None):
+    def __init__(self, openapi_spec: Dict[str, Any], spec_url: Optional[str] = None, call_template_name: Optional[str] = None, auth_tools: Optional[Auth] = None):
         """Initializes the OpenAPI converter.
 
         Args:
@@ -96,9 +96,12 @@ def __init__(self, openapi_spec: Dict[str, Any], spec_url: Optional[str] = None,
                 Used for base URL determination if servers are not specified.
             call_template_name: Optional custom name for the call_template if
                 the specification title is not provided.
+            auth_tools: Optional auth configuration for generated tools.
+                Applied only to endpoints that require authentication per OpenAPI spec.
         """
         self.spec = openapi_spec
         self.spec_url = spec_url
+        self.auth_tools = auth_tools
         # Single counter for all placeholder variables
         self.placeholder_counter = 0
         if call_template_name is None:
@@ -160,19 +163,22 @@ def convert(self) -> UtcpManual:
 
     def _extract_auth(self, operation: Dict[str, Any]) -> Optional[Auth]:
         """
-        Extracts authentication information from OpenAPI operation and global security schemes."""
+        Extracts authentication information from OpenAPI operation and global security schemes.
+        Uses auth_tools configuration when compatible with OpenAPI auth requirements.
+        Supports both OpenAPI 2.0 and 3.0 security schemes.
+        """
         # First check for operation-level security requirements
         security_requirements = operation.get("security", [])
 
         # If no operation-level security, check global security requirements
         if not security_requirements:
             security_requirements = self.spec.get("security", [])
 
-        # If no security requirements, return None
+        # If no security requirements, return None (endpoint is public)
         if not security_requirements:
             return None
 
-        # Get security schemes - support both OpenAPI 2.0 and 3.0
+        # Generate auth from OpenAPI security schemes - support both OpenAPI 2.0 and 3.0
         security_schemes = self._get_security_schemes()
 
         # Process the first security requirement (most common case)
@@ -181,9 +187,47 @@ def _extract_auth(self, operation: Dict[str, Any]) -> Optional[Auth]:
             for scheme_name, scopes in security_req.items():
                 if scheme_name in security_schemes:
                     scheme = security_schemes[scheme_name]
-                    return self._create_auth_from_scheme(scheme, scheme_name)
+                    openapi_auth = self._create_auth_from_scheme(scheme, scheme_name)
+                    
+                    # If compatible with auth_tools, use actual values from manual call template
+                    if self._is_auth_compatible(openapi_auth, self.auth_tools):
+                        return self.auth_tools
+                    else:
+                        return openapi_auth  # Use placeholder from OpenAPI scheme
 
         return None
+
+    def _is_auth_compatible(self, openapi_auth: Optional[Auth], auth_tools: Optional[Auth]) -> bool:
+        """
+        Checks if auth_tools configuration is compatible with OpenAPI auth requirements.
+        
+        Args:
+            openapi_auth: Auth generated from OpenAPI security scheme
+            auth_tools: Auth configuration from manual call template
+            
+        Returns:
+            True if compatible and auth_tools should be used, False otherwise
+        """
+        if not openapi_auth or not auth_tools:
+            return False
+            
+        # Must be same auth type
+        if type(openapi_auth) != type(auth_tools):
+            return False
+            
+        # For API Key auth, check header name and location compatibility
+        if hasattr(openapi_auth, 'var_name') and hasattr(auth_tools, 'var_name'):
+            openapi_var = openapi_auth.var_name.lower() if openapi_auth.var_name else ""
+            tools_var = auth_tools.var_name.lower() if auth_tools.var_name else ""
+            
+            if openapi_var != tools_var:
+                return False
+                
+            if hasattr(openapi_auth, 'location') and hasattr(auth_tools, 'location'):
+                if openapi_auth.location != auth_tools.location:
+                    return False
+        
+        return True
 
     def _get_security_schemes(self) -> Dict[str, Any]:
         """