Skip to content

add csharp interface for vitisai #25827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: rel-1.23.0
Choose a base branch
from
+67 −0
Open

add csharp interface for vitisai #25827

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions csharp/src/Microsoft.ML.OnnxRuntime/NativeMethods.shared.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -597,6 +597,8 @@ static NativeMethods()
api_.SessionOptionsAppendExecutionProvider_CUDA, typeof(DSessionOptionsAppendExecutionProvider_CUDA));
SessionOptionsAppendExecutionProvider_CUDA_V2 = (DSessionOptionsAppendExecutionProvider_CUDA_V2)Marshal.GetDelegateForFunctionPointer(
api_.SessionOptionsAppendExecutionProvider_CUDA_V2, typeof(DSessionOptionsAppendExecutionProvider_CUDA_V2));
SessionOptionsAppendExecutionProvider_VitisAI = (DSessionOptionsAppendExecutionProvider_VitisAI)Marshal.GetDelegateForFunctionPointer(
api_.SessionOptionsAppendExecutionProvider_VitisAI, typeof(DSessionOptionsAppendExecutionProvider_VitisAI));
OrtCreateCUDAProviderOptions = (DOrtCreateCUDAProviderOptions)Marshal.GetDelegateForFunctionPointer(api_.CreateCUDAProviderOptions, typeof(DOrtCreateCUDAProviderOptions));
OrtUpdateCUDAProviderOptions = (DOrtUpdateCUDAProviderOptions)Marshal.GetDelegateForFunctionPointer(api_.UpdateCUDAProviderOptions, typeof(DOrtUpdateCUDAProviderOptions));
OrtGetCUDAProviderOptionsAsString = (DOrtGetCUDAProviderOptionsAsString)Marshal.GetDelegateForFunctionPointer(api_.GetCUDAProviderOptionsAsString, typeof(DOrtGetCUDAProviderOptionsAsString));
Expand Down Expand Up @@ -1312,6 +1314,20 @@ IntPtr[] outputValues /* An array of output value pointers. Array must be alloca

public static DSessionOptionsAppendExecutionProvider_TensorRT_V2 SessionOptionsAppendExecutionProvider_TensorRT_V2;

/// <summary>
/// Append a VitisAI EP instance (configured based on given provider options) to the native OrtSessionOptions instance
/// </summary>
/// <param name="options">Native OrtSessionOptions instance</param>
/// <param name="keys">Native keys instance</param>
/// <param name="values">Native values instance</param>
/// <param name="numEntries">Native numEntries instance</param>
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
public delegate IntPtr /*(OrtStatus*)*/ DSessionOptionsAppendExecutionProvider_VitisAI(
IntPtr /*(OrtSessionOptions*)*/ options,
IntPtr keys, IntPtr values, UIntPtr numEntries);

public static DSessionOptionsAppendExecutionProvider_VitisAI SessionOptionsAppendExecutionProvider_VitisAI;

/// <summary>
/// Append a CUDA EP instance (configured based on given provider options) to the native OrtSessionOptions instance
/// </summary>
Expand Down
51 changes: 51 additions & 0 deletions csharp/src/Microsoft.ML.OnnxRuntime/SessionOptions.shared.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Text;
using System.Linq;

namespace Microsoft.ML.OnnxRuntime
{
Expand Down Expand Up @@ -257,6 +258,56 @@ public void AppendExecutionProvider_CUDA(OrtCUDAProviderOptions cudaProviderOpti
#endif
}

/// <summary>
/// Append a VitisAI EP instance (configured based on given provider options) to the native OrtSessionOptions instance
/// </summary>
/// <param name="options">Native OrtSessionOptions instance</param>
/// <param name="keys">Native keys instance</param>
/// <param name="values">Native values instance</param>
/// <param name="numEntries">Native numEntries instance</param>
public void AppendExecutionProvider_VitisAI(Dictionary<string, string> config)
Copy link
Member

@yuslepukhin yuslepukhin Aug 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add a test to this functionality.

{
#if __MOBILE__
throw new NotSupportedException("The VitisAI Execution Provider is not supported in this build");
#else
int count = config.Count;
IntPtr[] keyPtrs = new IntPtr[count];
IntPtr[] valuePtrs = new IntPtr[count];

string[] keys = config.Keys.ToArray();
string[] values = config.Values.ToArray();

for (int i = 0; i < count; ++i) {
keyPtrs[i] = Marshal.StringToHGlobalAnsi(keys[i]);
valuePtrs[i] = Marshal.StringToHGlobalAnsi(values[i]);
}

IntPtr keysNative = Marshal.AllocHGlobal(IntPtr.Size * count);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global allocations are slow. The only advantage is that they do not need to be pinned.
Pinning is necessary so the allocations is not moved or GCed while being used by native calls.

However, in this case keysNatives aren valuesNative are automatically pinned when being passed to native method
because IntPtr are considered a blittable type so they donot need to be allocated and copied separatly, you can pass keysPtrs and valuesPtrs directly
to the native call even though those are managed arrays.

So you will not need to allocate and copy to *Native arrays. This will also take care of the memory leak in case Copy() throws.

The issue here, is that we need to convert strings to UTF-8 and not ANSI. Use utility functions to accomplish that.
Those will produce managed byte[] arrays with zero terminator containing UTF-8 bytes.

You will need to PIN those and obtain IntPtr which can be placed to keysPtrs and valuesPtrs.

byte[] buffer = new byte[1024];
GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
IntPtr ptr = handle.AddrOfPinnedObject();
// Use ptr...
handle.Free();

You will need to unpin those in finally block (Free()). This way Marshal functionality is not longer needed.

IntPtr valuesNative = Marshal.AllocHGlobal(IntPtr.Size * count);

Marshal.Copy(keyPtrs, 0, keysNative, count);
Marshal.Copy(valuePtrs, 0, valuesNative, count);

try
{
UIntPtr numKeys = new UIntPtr((uint)count);
NativeApiStatus.VerifySuccess(
NativeMethods.SessionOptionsAppendExecutionProvider_VitisAI(
handle, keysNative, valuesNative, numKeys));
}
finally
{
for (int i = 0; i < count; ++i)
{
Marshal.FreeHGlobal(keyPtrs[i]);
Marshal.FreeHGlobal(valuePtrs[i]);
}
Marshal.FreeHGlobal(keysNative);
Marshal.FreeHGlobal(valuesNative);
}
#endif
}

/// <summary>
/// Use only if you have the onnxruntime package specific to this Execution Provider.
/// </summary>
Expand Down
Loading