Normalize Tools to Support OpenAI Function Calling Standards

[sayanshaw24]

Description

This PR introduces built-in normalization for tool definitions within the ApplyChatTemplate API in ORT Extensions, enabling support for multiple tool formats, including OpenAI’s function/tool schema.

Previously, ApplyChatTemplate assumed tools were always passed in the Minja/Phi-4 format, which caused incompatibilities when integrating models or clients that use OpenAI-style tool definitions.

With this change, callers can now pass tools in any supported format, and the API will normalize them internally — enabling tool structure–agnostic chat templating. This allows chat templates to render correctly regardless of whether the tools originate from Phi-4, OpenAI, or other ecosystems, without requiring any manual preprocessing on the user’s side.

Additionally, thorough unit tests have been added to validate this behavior across all supported formats.

---

Key Changes

• C++ Tool Normalization

  • Added a new NormalizeTools function that parses and transforms incoming tool definitions into the Minja/Phi-4 format expected by chat templates.
  • Supports three major formats:
    1. Phi-4 / Minja (already normalized)
    2. OpenAI type: "tool" format
    3. OpenAI type: "function" format with nested function object

• Parameter Conversion

  • Introduced ConvertParameters, a helper that detects OpenAI "parameters": { "type": "object", "properties": ... } schemas and flattens them into a simple parameter mapping.

• Integration with ApplyChatTemplate

  • Normalization occurs automatically before inserting tools into the Minja context, meaning callers can now pass tools in any of the supported formats without changing their code.

• Extensive Inline Documentation

  • Added detailed multi-line documentation blocks for both ConvertParameters and NormalizeTools to explain the transformation logic and expected input/output structures.

• Unit Tests

  • Added new tests based on the existing Phi4ChatTemplate test to cover all three tool formats, verifying that the rendered chat template output remains correct after normalization.

---

Rationale

Different ecosystems (e.g., OpenAI vs. Phi-4) define tools in subtly different JSON structures. Previously, users had to manually reformat OpenAI tool definitions to match Minja’s expected structure, increasing friction and reducing interoperability.

By introducing a centralized normalization step, this PR ensures:

• Interoperability with OpenAI tools without requiring external preprocessing
• Tool structure–agnostic chat templating, allowing templates to work regardless of the input format
• Future extensibility, allowing new tool schemas to be supported in one place
• Cleaner application code, since normalization happens automatically inside ApplyChatTemplate

---

Validation

• Native C++ unit testing for all three tool formats (Minja/Phi-4, OpenAI tool, OpenAI function)
• Verified that chat template rendering produces the expected Phi-4 system/user/assistant block from HuggingFace
• Confirmed that existing tests and APIs remain unaffected when tools are not provided/provided in previous standard