Class TensorRtBuilder

Namespace

JYPPX.TensorRtSharp

Assembly

JYPPX.TensorRtSharp.dll

Managed wrapper around a TensorRT builder. TensorRT builder 的托管封装。

public sealed class TensorRtBuilder : IDisposable

Inheritance

object

TensorRtBuilder

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

TensorRtBuilder(TensorRtLogger)

Creates a TensorRT builder from a logger. 使用 logger 创建一个 TensorRT builder。

public TensorRtBuilder(TensorRtLogger logger)

Parameters

logger TensorRtLogger

The TensorRT logger used by the builder. builder 使用的 TensorRT logger。

Remarks

TensorRT borrows the logger pointer. This builder keeps the managed logger attached until the builder is disposed. TensorRT 只借用 logger 指针；当前 builder 会保持托管 logger 借用关系直到 builder 释放。

Properties

DlaCoreCount

Gets the number of available DLA cores. 获取可用 DLA core 数量。

public int DlaCoreCount { get; }

Property Value

int

HasErrorRecorder

Gets whether a native TensorRT error recorder is currently attached to this builder. 获取当前 builder 是否绑定了 TensorRT 原生 error recorder；不会暴露 recorder 指针或接管其生命周期。

public bool HasErrorRecorder { get; }

Property Value

bool

HasLogger

Gets whether this builder still has a native TensorRT logger attached. 获取当前 builder 是否仍附加 TensorRT 原生 logger；不会暴露 logger 指针或接管其生命周期。

public bool HasLogger { get; }

Property Value

bool

HasManagedGpuAllocator

Gets whether this wrapper retains a managed GPU allocator owner. 获取该 wrapper 是否保留托管 GPU allocator owner。

public bool HasManagedGpuAllocator { get; }

Property Value

bool

Line

Gets the TensorRT API line used by this builder. 获取当前 builder 使用的 TensorRT API line。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

MaxBatchSizeCompatibility

Gets TensorRT 8's legacy explicit builder max batch size. 获取 TensorRT 8 legacy builder max batch size；TensorRT 10/11 已移除此 builder 级查询。

public int MaxBatchSizeCompatibility { get; }

Property Value

int

Remarks

Prefer explicit batch networks and optimization profiles for modern TensorRT builds. This property is a compatibility diagnostic only. 现代 TensorRT 构建请优先使用 explicit batch network 与 optimization profile；该属性仅作为兼容性诊断。

MaxDlaBatchSize

Gets TensorRT's maximum DLA batch size for legacy DLA checks. 获取 TensorRT 用于传统 DLA 检查的最大 DLA batch size。

public int MaxDlaBatchSize { get; }

Property Value

int

Remarks

TensorRT reports that this value does not apply to dynamic shapes. Use it as a compatibility diagnostic, not as a dynamic-shape capacity limit. TensorRT 官方说明该值不适用于 dynamic shape；请把它作为兼容性诊断信息，而不是动态 shape 的容量上限。

MaxThreads

Gets TensorRT's current builder worker-thread limit. 获取 TensorRT 当前 builder 工作线程上限。

public int MaxThreads { get; }

Property Value

int

PlatformHasFastFp16

Gets whether the platform reports fast FP16 support. 获取平台是否报告支持快速 FP16。

public bool PlatformHasFastFp16 { get; }

Property Value

bool

PlatformHasFastInt8

Gets whether the platform reports fast INT8 support. 获取平台是否报告支持快速 INT8。

public bool PlatformHasFastInt8 { get; }

Property Value

bool

PlatformHasTf32

Gets whether the platform reports TF32 support. 获取平台是否报告支持 TF32。

public bool PlatformHasTf32 { get; }

Property Value

bool

Methods

BuildEngineWithConfig(TensorRtNetworkDefinition, TensorRtBuilderConfig)

Builds a TensorRT 8, 10, or 11 CUDA engine directly from a network and builder configuration. 直接从 network 和 builder config 构建 TensorRT 8、10 或 11 CUDA engine。

public TensorRtEngine BuildEngineWithConfig(TensorRtNetworkDefinition network, TensorRtBuilderConfig config)

Parameters

network TensorRtNetworkDefinition

The TensorRT network definition to build. 要构建的 TensorRT network definition。

config TensorRtBuilderConfig

The TensorRT builder configuration. TensorRT 构建配置。

Returns

TensorRtEngine

A managed CUDA engine wrapper. 托管 CUDA engine 封装对象。

Remarks

This exposes TensorRT's IBuilder::buildEngineWithConfig path. For version-compatible deployment and persistence, prefer BuildSerializedNetwork(TensorRtNetworkDefinition, TensorRtBuilderConfig). 该方法对应 TensorRT 的 IBuilder::buildEngineWithConfig 路径；如果需要版本兼容部署或持久化，请优先使用 BuildSerializedNetwork(TensorRtNetworkDefinition, TensorRtBuilderConfig)。

BuildSerializedNetwork(TensorRtNetworkDefinition, TensorRtBuilderConfig)

Builds a serialized TensorRT engine from a network and configuration. 使用 network 和配置构建序列化的 TensorRT engine。

public TensorRtHostMemory BuildSerializedNetwork(TensorRtNetworkDefinition network, TensorRtBuilderConfig config)

Parameters

network TensorRtNetworkDefinition

The source network definition. 源 network definition。

config TensorRtBuilderConfig

The builder configuration. builder 配置。

Returns

TensorRtHostMemory

The serialized engine memory. 序列化 engine 内存。

BuildSerializedNetworkWithKernelText(TensorRtNetworkDefinition, TensorRtBuilderConfig)

Builds a serialized TensorRT 11 network and captures optional kernel text emitted by TensorRT. 构建 TensorRT 11 序列化网络，并获取 TensorRT 可选生成的 kernel text。

public TensorRtSerializedNetworkWithKernelText BuildSerializedNetworkWithKernelText(TensorRtNetworkDefinition network, TensorRtBuilderConfig config)

Parameters

network TensorRtNetworkDefinition

The TensorRT network definition to build. 要构建的 TensorRT network definition。

config TensorRtBuilderConfig

The TensorRT builder configuration. TensorRT 构建配置。

Returns

TensorRtSerializedNetworkWithKernelText

The serialized plan plus optional kernel text. 序列化 plan 以及可选 kernel text。

ClearErrorRecorder()

Clears the native TensorRT error recorder attached to this builder. 清除当前 builder 上绑定的 TensorRT 原生 error recorder；不会销毁 recorder 或接管其生命周期。

public void ClearErrorRecorder()

ClearGpuAllocator()

Clears the custom GPU allocator on this builder. 清除当前 builder 上绑定的自定义 GPU allocator；适用于 TensorRT 8/10/11，不会调用用户 allocator 的 free/deallocate 回调。

public void ClearGpuAllocator()

Remarks

Managed owners installed through SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner) are detached with their borrower leases intact. When no managed owner is installed, this method clears an externally installed allocator pointer. 通过 SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner) 安装的托管 owner 会在保留其子对象租约的前提下解除绑定； 未安装托管 owner 时，本方法清除外部设置的 allocator 指针。

CreateBuilderConfig()

Creates a builder configuration object. 创建一个 builder 配置对象。

public TensorRtBuilderConfig CreateBuilderConfig()

Returns

TensorRtBuilderConfig

A TensorRT builder configuration wrapper. TensorRT builder 配置封装。

CreateNetwork(TensorRtNetworkDefinitionCreationFlags)

Creates a network definition owned by this builder. 创建一个由当前 builder 拥有的 network definition。

public TensorRtNetworkDefinition CreateNetwork(TensorRtNetworkDefinitionCreationFlags flags = TensorRtNetworkDefinitionCreationFlags.None)

Parameters

flags TensorRtNetworkDefinitionCreationFlags

The network-definition creation flags. network definition 创建标志。

Returns

TensorRtNetworkDefinition

A TensorRT network-definition wrapper. TensorRT network definition 封装。

CreateNetwork(bool)

Creates an explicit-batch network and maps strongly typed creation to the selected TensorRT API line. 创建 explicit-batch network，并将 strongly typed 创建语义映射到所选 TensorRT API 版本。

public TensorRtNetworkDefinition CreateNetwork(bool stronglyTyped)

Parameters

stronglyTyped bool

Whether a strongly typed network is required. 是否要求 strongly typed network。

Returns

TensorRtNetworkDefinition

A caller-owned TensorRT network definition. 由调用方拥有的 TensorRT network definition。

Remarks

TensorRT 10 uses bit 1, TensorRT 11 is always strongly typed, and TensorRT 8 does not expose this policy. TensorRT 10 使用 bit 1，TensorRT 11 始终为 strongly typed，TensorRT 8 不公开该策略。

CreateOptimizationProfile()

Creates an optimization profile. 创建一个优化 profile。

public TensorRtOptimizationProfile CreateOptimizationProfile()

Returns

TensorRtOptimizationProfile

A TensorRT optimization-profile wrapper. TensorRT optimization profile 封装。

Dispose()

Releases the TensorRT builder handle. 释放 TensorRT builder 句柄。

public void Dispose()

GetPluginRegistryInventory()

Gets a read-only snapshot of TensorRT plugin creators registered for this builder. 获取当前 builder 可见的 TensorRT plugin creator 只读快照。

public TensorRtPluginRegistryInventory GetPluginRegistryInventory()

Returns

TensorRtPluginRegistryInventory

A plugin registry inventory snapshot. Plugin registry inventory 快照。

Remarks

This method does not create plugins, deserialize plugins, load plugin libraries, or take ownership of creator objects. 该方法不会创建 plugin、反序列化 plugin、加载 plugin library，也不会接管 creator 对象所有权。

IsNetworkSupported(TensorRtNetworkDefinition, TensorRtBuilderConfig)

Checks whether TensorRT can build the supplied network with the supplied builder config. 检查 TensorRT 是否支持使用指定 builder config 构建给定 network。

public bool IsNetworkSupported(TensorRtNetworkDefinition network, TensorRtBuilderConfig config)

Parameters

network TensorRtNetworkDefinition

The network definition to validate. / 要验证的 network definition。

config TensorRtBuilderConfig

The builder config used for validation. / 用于验证的 builder config。

Returns

bool

true when TensorRT reports the network as supported. / TensorRT 报告该 network 可支持时返回 true。

IsPluginCreatorRegistered(string, string, string)

Checks whether this builder-visible plugin registry contains a creator matching the supplied metadata. 检查当前 builder 可见的 plugin registry 是否包含匹配给定元数据的 creator。

public bool IsPluginCreatorRegistered(string pluginName, string pluginVersion, string pluginNamespace)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

Returns

bool

true when TensorRT finds a matching creator. TensorRT 找到匹配 creator 时返回 true。

Remarks

This lookup does not create a plugin, does not return the native creator pointer, and does not take ownership of TensorRT objects. 该 lookup 不会创建 plugin、不会返回 native creator 指针，也不会接管 TensorRT 对象所有权。

IsPluginRegistryAvailable()

Checks whether this builder exposes a local plugin registry through the safe inventory bridge. 检查当前 builder 是否能通过安全 inventory 桥接访问本地 plugin registry。

public bool IsPluginRegistryAvailable()

Returns

bool

true when the builder plugin registry can be queried. 可查询 builder plugin registry 时返回 true。

Reset()

Resets TensorRT 11 builder state back to its default configuration. 将 TensorRT 11 builder 状态重置为默认配置。

public void Reset()

SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner)

Attaches an owner-safe native IGpuAllocator to this builder. 将 owner-safe native IGpuAllocator 挂载到当前 builder。

public void SetGpuAllocator(TensorRtGpuAllocatorCallbackOwner owner)

Parameters

owner TensorRtGpuAllocatorCallbackOwner

The owner retained by this builder and every engine it creates. 由 builder 及其创建的每个 engine 保留的 owner。

SetMaxBatchSizeCompatibility(int)

Sets the deprecated TensorRT 8 implicit-batch maximum. 设置已弃用的 TensorRT 8 implicit-batch 最大值。

public void SetMaxBatchSizeCompatibility(int maxBatchSize)

Parameters

maxBatchSize int

A positive legacy maximum batch size. 大于零的 legacy 最大 batch size。

Remarks

Modern TensorRT code should use explicit-batch networks and optimization profiles. TensorRT 10 and 11 report this control as unsupported. 现代 TensorRT 代码应使用 explicit-batch network 与 optimization profile；TensorRT 10/11 会将此控制报告为不支持。

SetMaxThreads(int)

Sets TensorRT's builder worker-thread limit. 设置 TensorRT 的 builder 工作线程上限。

public bool SetMaxThreads(int maxThreads)

Parameters

maxThreads int

The desired worker-thread limit. / 期望的工作线程上限。

Returns

bool

true when TensorRT accepted the setting. / TensorRT 接受该设置时返回 true。

TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot)

Attempts to collect a copied read-only snapshot from this builder's error recorder. 尝试从当前 builder 的 error recorder 采集只读托管快照。

public bool TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot snapshot)

Parameters

snapshot TensorRtErrorRecorderSnapshot

The copied snapshot. 已复制到托管内存的快照。

Returns

bool

true when a recorder is attached. / 当前附加 recorder 时返回 true。

Remarks

The snapshot copies counts, overflow state, interface metadata when available, and error descriptions. It never exposes or retains an IErrorRecorder*. 该快照复制数量、溢出状态、可用时的 interface 元数据和错误描述；不会暴露或持有 IErrorRecorder*。

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata)

Parameters

metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

metadata TensorRtVersionedInterfaceMetadata

diagnostic string

Returns

bool

TryGetPluginCreator(string, string, string, out TensorRtPluginCreatorInfo?, out string)

Tries to read metadata for a plugin creator from this builder-visible plugin registry. 尝试从当前 builder 可见的 plugin registry 读取某个 plugin creator 的元数据。

public bool TryGetPluginCreator(string pluginName, string pluginVersion, string pluginNamespace, out TensorRtPluginCreatorInfo? creator, out string diagnostic)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

creator TensorRtPluginCreatorInfo

The copied creator metadata when a matching creator is found. 找到匹配 creator 时复制出的 creator 元数据。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when a matching creator was found and copied. 找到并复制匹配 creator 时返回 true。

Remarks

The lookup does not create a plugin, does not return the native creator pointer, and does not take ownership of TensorRT objects. The returned metadata is copied into managed objects. 该 lookup 不会创建 plugin、不会返回 native creator 指针，也不会接管 TensorRT 对象所有权；返回的元数据会复制到托管对象中。

TryGetPluginRegistryInventory(out TensorRtPluginRegistryInventory, out string)

Tries to get a plugin registry inventory without throwing for unsupported TensorRT lines. 尝试获取 plugin registry inventory；当 TensorRT line 不支持时不抛出异常。

public bool TryGetPluginRegistryInventory(out TensorRtPluginRegistryInventory inventory, out string diagnostic)

Parameters

inventory TensorRtPluginRegistryInventory

The inventory snapshot when the query succeeds. 查询成功时的 inventory 快照。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the inventory was collected successfully. 成功收集 inventory 时返回 true。

TryIsPluginCreatorRegistered(string, string, string, out bool, out string)

Tries to check whether this builder-visible plugin registry contains a matching creator without throwing probe exceptions. 尝试检查当前 builder 可见的 plugin registry 是否存在匹配 creator；探针异常会转为诊断字符串。

public bool TryIsPluginCreatorRegistered(string pluginName, string pluginVersion, string pluginNamespace, out bool found, out string diagnostic)

Parameters

pluginName string

The plugin creator name. plugin creator 名称。

pluginVersion string

The plugin creator version. plugin creator 版本。

pluginNamespace string

The plugin creator namespace. plugin creator 命名空间。

found bool

Set to true when TensorRT finds a matching creator. 找到匹配 creator 时设为 true。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the lookup completed successfully. lookup 成功完成时返回 true。

Remarks

This lookup does not create a plugin and does not return or own the native creator pointer. 该 lookup 不会创建 plugin，也不会返回或持有 native creator 指针。

TryIsPluginRegistryAvailable(out bool, out string)

Tries to check builder plugin registry availability without throwing probe exceptions. 尝试检查 builder plugin registry 可用性；探针异常会转为诊断字符串。

public bool TryIsPluginRegistryAvailable(out bool exists, out string diagnostic)

Parameters

exists bool

Set to true when the builder plugin registry can be queried. 可查询时设为 true。

diagnostic string

A diagnostic string describing success or failure. 描述成功或失败原因的诊断字符串。

Returns

bool

true when the availability check completed successfully. 可用性检查成功完成时返回 true。