Class TensorRtRuntime

Namespace

JYPPX.TensorRtSharp

Assembly

JYPPX.TensorRtSharp.dll

Managed wrapper around a TensorRT runtime. TensorRT runtime 的托管封装。

public sealed class TensorRtRuntime : IDisposable

Inheritance

object

TensorRtRuntime

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

TensorRtRuntime(TensorRtLogger)

Creates a TensorRT runtime from a logger. 使用 logger 创建一个 TensorRT runtime。

public TensorRtRuntime(TensorRtLogger logger)

Parameters

logger TensorRtLogger

The TensorRT logger used by the runtime. runtime 使用的 TensorRT logger。

Remarks

TensorRT borrows the logger pointer. This runtime keeps the managed logger attached until the runtime is disposed. TensorRT 只借用 logger 指针；当前 runtime 会保持托管 logger 借用关系直到 runtime 释放。

Properties

DlaCore

Gets or sets the DLA core selected for TensorRT runtime deserialization. 获取或设置 TensorRT 11 runtime 反序列化时选择的 DLA core。

public int DlaCore { get; set; }

Property Value

int

DlaCoreCount

Gets the number of DLA cores visible to the TensorRT runtime. 获取 TensorRT 11 runtime 可见的 DLA core 数量。

public int DlaCoreCount { get; }

Property Value

int

EngineHostCodeAllowed

Gets or sets whether TensorRT may deserialize engines that contain host executable code. 获取或设置 TensorRT 是否允许反序列化包含主机可执行代码的 engine。

public bool EngineHostCodeAllowed { get; set; }

Property Value

bool

HasErrorRecorder

Gets whether this runtime currently has a TensorRT error recorder attached. 获取当前 runtime 是否附加了 TensorRT error recorder；不会暴露 recorder 指针或接管其生命周期。

public bool HasErrorRecorder { get; }

Property Value

bool

HasLogger

Gets whether this runtime still has a native TensorRT logger attached. 获取当前 runtime 是否仍附加 TensorRT 原生 logger；不会暴露 logger 指针或接管其生命周期。

public bool HasLogger { get; }

Property Value

bool

HasManagedGpuAllocator

Gets whether this wrapper retains a managed GPU allocator owner. 获取该 wrapper 是否保留托管 GPU allocator owner。

public bool HasManagedGpuAllocator { get; }

Property Value

bool

Line

Gets the TensorRT API line used by this runtime. 获取当前 runtime 使用的 TensorRT API line。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

MaxThreads

Gets or sets the maximum number of worker threads TensorRT may use in this runtime. 获取或设置当前 runtime 允许 TensorRT 使用的最大工作线程数。

public int MaxThreads { get; set; }

Property Value

int

TempfileControlFlags

Gets or sets TensorRT temporary-file control flags. 获取或设置 TensorRT 临时文件控制标志。

public TensorRtTempfileControlFlags TempfileControlFlags { get; set; }

Property Value

TensorRtTempfileControlFlags

Methods

ClearErrorRecorder()

Clears the native error recorder pointer if one was attached externally. 清除外部附加的原生 error recorder 指针；不会销毁 recorder 或接管其生命周期。

public void ClearErrorRecorder()

ClearGpuAllocator()

Clears the native GPU allocator pointer and returns runtime allocation to TensorRT defaults. 清除原生 GPU allocator 指针，让 runtime allocation 回到 TensorRT 默认行为；不会调用用户 allocator 的 free/deallocate 回调。

public void ClearGpuAllocator()

Remarks

A managed owner installed through SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner) remains alive while any engine deserialized by this runtime still holds its inherited borrower lease. 通过 SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner) 安装的托管 owner 会持续存活，直到该 runtime 反序列化出的所有 engine 都释放继承的借用租约。

ClearTemporaryDirectory()

Clears the explicit temporary directory and lets TensorRT use platform defaults. 清除显式临时目录，让 TensorRT 使用平台默认目录。

public void ClearTemporaryDirectory()

Deserialize(TensorRtHostMemory)

Deserializes an engine from TensorRT host memory. 从 TensorRT host memory 反序列化一个 engine。

public TensorRtEngine Deserialize(TensorRtHostMemory hostMemory)

Parameters

hostMemory TensorRtHostMemory

The serialized engine memory. 序列化 engine 内存。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Deserialize(TensorRtStreamReader)

Deserializes an engine through a real native TensorRT IStreamReaderV2 callback owner. 通过真实原生 TensorRT IStreamReaderV2 回调所有者反序列化引擎。

public TensorRtEngine Deserialize(TensorRtStreamReader streamReader)

Parameters

streamReader TensorRtStreamReader

The owner-safe reader retained by the returned engine. 由返回引擎保持存活的所有权安全 reader。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT 引擎包装对象。

Remarks

TensorRT 10 and 11 may request host or device destinations and may seek within the immutable source. The reader owner remains alive until the returned engine is disposed, even when the caller requests reader disposal earlier. TensorRT 10 与 11 可请求主机或设备目标并在不可变数据源中定位；即使提前请求释放，reader 所有者也会存活到返回的引擎释放。

Deserialize(ArraySegment<byte>)

Deserializes an engine from a managed byte-array segment. 从托管字节数组片段反序列化一个 engine。

public TensorRtEngine Deserialize(ArraySegment<byte> serializedEngine)

Parameters

serializedEngine ArraySegment<byte>

The serialized engine byte segment. 序列化 engine 字节数组片段。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Remarks

The segment is copied into an exact managed byte array before the native deserialize call, so TensorRT only observes a pinned managed buffer for the duration of the call. 调用 native 反序列化前会将片段复制为精确长度的托管字节数组，因此 TensorRT 只会在调用期间看到 pinned 托管缓冲区。

Deserialize(byte[])

Deserializes an engine from a managed byte buffer. 从托管字节缓冲区反序列化一个 engine。

public TensorRtEngine Deserialize(byte[] serializedEngine)

Parameters

serializedEngine byte[]

The serialized engine bytes. 序列化 engine 字节数组。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Deserialize(Stream)

Deserializes an engine from a managed stream. 从托管 stream 反序列化一个 engine。

public TensorRtEngine Deserialize(Stream serializedEngineStream)

Parameters

serializedEngineStream Stream

The readable stream containing serialized engine bytes. 包含序列化 engine 字节的可读 stream。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Remarks

The stream is copied into managed memory before native interop; this is not a TensorRT IStreamReader callback bridge. 调用 native interop 前会将 stream 内容复制到托管内存；该入口不是 TensorRT IStreamReader 回调桥。

Deserialize(ReadOnlySpan<byte>)

Deserializes an engine from a managed read-only byte span. 从托管只读字节 span 反序列化一个 engine。

public TensorRtEngine Deserialize(ReadOnlySpan<byte> serializedEngine)

Parameters

serializedEngine ReadOnlySpan<byte>

The serialized engine bytes. 序列化 engine 字节。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Remarks

The span is copied into a managed byte array before native interop so no caller-owned span is retained by TensorRT. 调用 native interop 前会将 span 复制到托管字节数组，因此 TensorRT 不会保留调用方拥有的 span。

DeserializeFromFile(string)

Deserializes an engine from a serialized engine file. 从序列化 engine 文件中反序列化一个 engine。

public TensorRtEngine DeserializeFromFile(string filePath)

Parameters

filePath string

The path to the serialized engine file. 序列化 engine 文件路径。

Returns

TensorRtEngine

A TensorRT engine wrapper. TensorRT engine 封装。

Dispose()

Releases the TensorRT runtime handle. 释放 TensorRT runtime 句柄。

public void Dispose()

GetDiagnosticSnapshot()

Gets a copied read-only diagnostic snapshot for this runtime. 获取当前 runtime 的复制型只读诊断快照。

public TensorRtRuntimeDiagnosticSnapshot GetDiagnosticSnapshot()

Returns

TensorRtRuntimeDiagnosticSnapshot

GetPluginRegistryInventory()

Gets a read-only snapshot of plugin creators registered in this runtime-local plugin registry. 获取当前 runtime-local plugin registry 中已注册 plugin creator 的只读快照。

public TensorRtPluginRegistryInventory GetPluginRegistryInventory()

Returns

TensorRtPluginRegistryInventory

A plugin registry inventory snapshot. Plugin registry inventory 快照。

Remarks

This method does not create plugins, deserialize plugins, load plugin libraries, or take ownership of creator objects. Returned creator and field metadata is copied into managed objects. TensorRT 8 does not expose a separate recursive creator count through this bridge; RecursiveCreatorCount is null for that line. 该方法不会创建 plugin、反序列化 plugin、加载 plugin library，也不会接管 creator 对象所有权；返回的 creator 和字段元数据会复制到托管对象中。TensorRT 8 不通过该桥接暴露单独的递归 creator count，因此 RecursiveCreatorCount 为 null。

GetTemporaryDirectory()

Gets the runtime temporary directory configured through TensorRT. 获取通过 TensorRT 配置的 runtime 临时目录。

public string GetTemporaryDirectory()

Returns

string

The configured temporary directory or an empty string when TensorRT uses defaults. 已配置目录；TensorRT 使用默认值时为空字符串。

IsPluginCreatorRegistered(string, string, string)

Checks whether this runtime-local plugin registry contains a creator matching the supplied metadata. 检查当前 runtime-local plugin registry 是否包含匹配给定元数据的 creator。

public bool IsPluginCreatorRegistered(string pluginName, string pluginVersion, string pluginNamespace)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

Returns

bool

true when TensorRT finds a matching creator. TensorRT 找到匹配 creator 时返回 true。

Remarks

This lookup does not create a plugin, does not return the native creator pointer, and does not take ownership of TensorRT objects. 该 lookup 不会创建 plugin、不会返回 native creator 指针，也不会接管 TensorRT 对象所有权。

IsPluginRegistryAvailable()

Checks whether this runtime exposes a runtime-local plugin registry. 检查当前 runtime 是否暴露 runtime-local plugin registry。

public bool IsPluginRegistryAvailable()

Returns

bool

true when TensorRT reports a runtime-local plugin registry. TensorRT 报告存在 runtime-local plugin registry 时返回 true。

Remarks

This method only checks registry availability. It does not return or own the native registry pointer. TensorRT 8, 10, and 11 are queried through copied metadata probes. 该方法只检查 registry 可用性；不会返回或持有 native registry 指针。TensorRT 8、10 和 11 均通过复制元数据探针查询。

SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner)

Attaches an owner-safe native IGpuAllocator to this runtime. 将 owner-safe native IGpuAllocator 挂载到当前 runtime。

public void SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner owner)

Parameters

owner TensorRtGpuAllocatorCallbackOwner

The owner retained by this runtime and every engine it creates. 由 runtime 及其创建的每个 engine 保留的 owner。

SetTemporaryDirectory(string)

Sets the runtime temporary directory used by TensorRT for executable temporary files. 设置 TensorRT 用于可执行临时文件的 runtime 临时目录。

public void SetTemporaryDirectory(string directoryPath)

Parameters

directoryPath string

The temporary directory path. 临时目录路径。

TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot)

Attempts to collect a copied read-only snapshot from the runtime error recorder. 尝试从 runtime error recorder 采集只读托管快照。

public bool TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot snapshot)

Parameters

snapshot TensorRtErrorRecorderSnapshot

The copied snapshot. 已复制到托管内存的快照。

Returns

bool

true when a recorder was attached; otherwise false. 附加了 recorder 时返回 true，否则返回 false。

Remarks

This method does not expose, retain, increment, decrement, or destroy the native recorder pointer. 此方法不会暴露、持有、增加引用、减少引用或销毁原生 recorder 指针。

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata)

Parameters

metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

metadata TensorRtVersionedInterfaceMetadata

diagnostic string

Returns

bool

TryGetPluginCreator(string, string, string, out TensorRtPluginCreatorInfo?, out string)

Tries to read metadata for a plugin creator from this runtime-local plugin registry. 尝试从当前 runtime-local plugin registry 读取某个 plugin creator 的元数据。

public bool TryGetPluginCreator(string pluginName, string pluginVersion, string pluginNamespace, out TensorRtPluginCreatorInfo? creator, out string diagnostic)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

creator TensorRtPluginCreatorInfo

The copied creator metadata when a matching creator is found. 找到匹配 creator 时复制出的 creator 元数据。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when a matching creator was found and copied. 找到并复制匹配 creator 时返回 true。

Remarks

The lookup does not create a plugin, does not return the native creator pointer, and does not take ownership of TensorRT objects. The returned metadata is copied into managed objects. 该 lookup 不会创建 plugin、不会返回 native creator 指针，也不会接管 TensorRT 对象所有权；返回的元数据会复制到托管对象中。

TryGetPluginRegistryInventory(out TensorRtPluginRegistryInventory, out string)

Tries to get a runtime-local plugin registry inventory without throwing probe exceptions. 尝试获取 runtime-local plugin registry inventory；探针异常会转为诊断字符串。

public bool TryGetPluginRegistryInventory(out TensorRtPluginRegistryInventory inventory, out string diagnostic)

Parameters

inventory TensorRtPluginRegistryInventory

The inventory snapshot when the query succeeds. 查询成功时的 inventory 快照。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the inventory was collected successfully. 成功收集 inventory 时返回 true。

TryIsPluginCreatorRegistered(string, string, string, out bool, out string)

Tries to check whether this runtime-local plugin registry contains a matching creator without throwing probe exceptions. 尝试检查当前 runtime-local plugin registry 是否存在匹配 creator；探针异常会转为诊断字符串。

public bool TryIsPluginCreatorRegistered(string pluginName, string pluginVersion, string pluginNamespace, out bool found, out string diagnostic)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

found bool

Set to true when TensorRT finds a matching creator. 找到匹配 creator 时设为 true。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the lookup completed successfully. lookup 成功完成时返回 true。

Remarks

This lookup does not create a plugin and does not return or own the native creator pointer. 该 lookup 不会创建 plugin，也不会返回或持有 native creator 指针。

TryIsPluginRegistryAvailable(out bool, out string)

Tries to check whether this runtime exposes a runtime-local plugin registry without throwing probe exceptions. 尝试检查当前 runtime 是否暴露 runtime-local plugin registry；探针异常会转为诊断字符串。

public bool TryIsPluginRegistryAvailable(out bool exists, out string diagnostic)

Parameters

exists bool

Set to true when the runtime-local plugin registry exists. 存在 runtime-local plugin registry 时设为 true。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the availability query completed successfully. 可用性查询成功完成时返回 true。