Class TensorRtEngine

Namespace

JYPPX.TensorRtSharp

Assembly

JYPPX.TensorRtSharp.dll

Managed wrapper around a TensorRT engine. TensorRT engine 的托管封装。

public sealed class TensorRtEngine : IDisposable

Inheritance

object

TensorRtEngine

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

AuxiliaryStreamCount

Gets the number of auxiliary streams requested by the built engine. 获取已构建 engine 请求的辅助 CUDA stream 数量。

public int AuxiliaryStreamCount { get; }

Property Value

int

Capability

Gets the engine capability reported by TensorRT. 获取 TensorRT 报告的 engine capability。

public TensorRtEngineCapability Capability { get; }

Property Value

TensorRtEngineCapability

CompatibilityBindingCount

Gets the engine binding count using the TensorRT 8 legacy binding vocabulary. 使用 TensorRT 8 legacy binding 语义获取 engine binding 数量。

public int CompatibilityBindingCount { get; }

Property Value

int

Remarks

On TensorRT 10 and 11 this maps to the name-based I/O tensor count exposed by the bridge. 在 TensorRT 10 和 11 中，该属性映射到桥接层公开的基于名称的 I/O tensor 数量。

DeviceMemorySizeInBytes

Gets the engine device-memory requirement in bytes. 获取 engine 的设备内存需求，单位为字节。

public ulong DeviceMemorySizeInBytes { get; }

Property Value

ulong

DeviceMemorySizeV2InBytes

Gets the TensorRT 10 V2 device-memory requirement for this engine. 获取当前 engine 的 TensorRT 10 V2 设备内存需求；TensorRT 8 会返回不支持。

public ulong DeviceMemorySizeV2InBytes { get; }

Property Value

ulong

EngineHardwareCompatibilityLevel

Gets the hardware-compatibility level recorded on this TensorRT engine. 获取当前 TensorRT engine 记录的硬件兼容级别。

public TensorRtHardwareCompatibilityLevel EngineHardwareCompatibilityLevel { get; }

Property Value

TensorRtHardwareCompatibilityLevel

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。

HasErrorRecorder

Gets whether a native TensorRT error recorder is attached to this engine. 获取当前 engine 是否绑定了 TensorRT 原生 error recorder。

public bool HasErrorRecorder { get; }

Property Value

bool

IOTensorCount

Gets the number of engine I/O tensors. 获取 engine I/O tensor 数量。

public int IOTensorCount { get; }

Property Value

int

IsRefittable

Gets whether TensorRT reports this engine as refittable. 获取 TensorRT 是否报告当前 engine 支持 refit。

public bool IsRefittable { get; }

Property Value

bool

LayerCount

Gets the number of layers in the engine. 获取 engine 中的 layer 数量。

public int LayerCount { get; }

Property Value

int

Line

Gets the TensorRT API line used by this engine. 获取当前 engine 使用的 TensorRT API line。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

MaxBatchSizeCompatibility

Gets the TensorRT 8 compatibility max-batch-size value. 获取 TensorRT 8 兼容路径中的 max batch size 值。

public int MaxBatchSizeCompatibility { get; }

Property Value

int

Name

Gets the engine name reported by TensorRT. 获取 TensorRT 报告的 engine 名称。

public string Name { get; }

Property Value

string

OptimizationProfileCount

Gets the number of optimization profiles in the engine. 获取 engine 中的 optimization profile 数量。

public int OptimizationProfileCount { get; }

Property Value

int

ProfilingVerbosity

Gets the engine profiling verbosity. 获取 engine 的 profiling verbosity。

public TensorRtProfilingVerbosity ProfilingVerbosity { get; }

Property Value

TensorRtProfilingVerbosity

StreamableWeightsSizeInBytes

Gets the size of streamable weights reported by TensorRT. 获取 TensorRT 报告的可流式加载权重大小。

public long StreamableWeightsSizeInBytes { get; }

Property Value

long

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。

TacticSources

Gets the tactic-source mask used by the engine. 获取 engine 使用的 tactic source 掩码。

public TensorRtTacticSources TacticSources { get; }

Property Value

TensorRtTacticSources

WeightStreamingAutomaticBudgetInBytes

Gets TensorRT's automatically selected weight-streaming budget. 获取 TensorRT 自动选择的权重流式加载预算。

public long WeightStreamingAutomaticBudgetInBytes { get; }

Property Value

long

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。

WeightStreamingBudgetV2InBytes

Gets the currently configured TensorRT weight-streaming budget. 获取当前配置的 TensorRT 权重流式加载预算。

public long WeightStreamingBudgetV2InBytes { get; }

Property Value

long

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。

WeightStreamingScratchMemorySizeInBytes

Gets the scratch-memory requirement for TensorRT weight streaming. 获取 TensorRT 权重流式加载所需的临时显存大小。

public long WeightStreamingScratchMemorySizeInBytes { get; }

Property Value

long

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。

Methods

ClearErrorRecorder()

Clears the native TensorRT error recorder attached to this engine. 清除当前 engine 上绑定的 TensorRT 原生 error recorder。

public void ClearErrorRecorder()

CreateExecutionContext()

Creates a TensorRT execution context with engine-managed device memory. 创建一个由 engine 管理设备内存的 TensorRT execution context。

public TensorRtExecutionContext CreateExecutionContext()

Returns

TensorRtExecutionContext

A managed execution-context wrapper. 托管 execution context 封装。

CreateExecutionContext(TensorRtExecutionContextAllocationStrategy)

Creates an execution context with a TensorRT allocation strategy. 使用 TensorRT allocation strategy 创建 execution context。

public TensorRtExecutionContext CreateExecutionContext(TensorRtExecutionContextAllocationStrategy strategy)

Parameters

strategy TensorRtExecutionContextAllocationStrategy

The allocation strategy requested from TensorRT. / 请求 TensorRT 使用的分配策略。

Returns

TensorRtExecutionContext

A managed execution context wrapper. / 托管 execution context 封装。

Remarks

The allocation-strategy overload is currently a TensorRT 11 path. Use CreateExecutionContext(TensorRtRuntimeConfig) for TensorRT 10 runtime-config based creation. allocation-strategy 重载当前为 TensorRT 11 路径；TensorRT 10 可使用 CreateExecutionContext(TensorRtRuntimeConfig) 进行 runtime-config 创建。

CreateExecutionContext(TensorRtRuntimeConfig)

Creates an execution context with an explicit TensorRT runtime config. 使用显式 TensorRT runtime config 创建 execution context。

public TensorRtExecutionContext CreateExecutionContext(TensorRtRuntimeConfig runtimeConfig)

Parameters

runtimeConfig TensorRtRuntimeConfig

The runtime config to apply. / 要应用的 runtime config。

Returns

TensorRtExecutionContext

A managed execution context wrapper. / 托管 execution context 封装。

Remarks

This bridge currently supports this path for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11 的该路径。

CreateExecutionContextWithoutDeviceMemory()

Creates an execution context without internally allocated device memory. 创建一个不由 TensorRT 内部分配 device memory 的 execution context。

public TensorRtExecutionContext CreateExecutionContextWithoutDeviceMemory()

Returns

TensorRtExecutionContext

A managed execution context that requires device memory to be set before enqueue. 需要在 enqueue 前设置 device memory 的托管 execution context。

CreateInspector()

Creates a TensorRT engine inspector for this engine. 为当前 engine 创建一个 TensorRT engine inspector。

public TensorRtEngineInspector CreateInspector()

Returns

TensorRtEngineInspector

A managed engine-inspector wrapper. 托管 engine inspector 封装。

CreateRefitter(TensorRtLogger)

Creates a refitter for updating refittable weights in this engine. 为当前 engine 创建用于更新可 refit 权重的 refitter。

public TensorRtRefitter CreateRefitter(TensorRtLogger logger)

Parameters

logger TensorRtLogger

The TensorRT logger used by the refitter. Refitter 使用的 TensorRT logger。

Returns

TensorRtRefitter

A managed refitter wrapper. 托管 refitter 封装对象。

CreateRuntimeConfig()

Creates a TensorRT runtime config for execution-context creation when supported by the active line. 在当前 TensorRT 版本线支持时，创建用于 execution context 创建的 runtime config。

public TensorRtRuntimeConfig CreateRuntimeConfig()

Returns

TensorRtRuntimeConfig

A managed runtime config wrapper. / 托管 runtime config 封装。

Remarks

This bridge currently supports this path for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11 的该路径。

CreateSerializationConfig()

Creates a TensorRT serialization config for this engine when the active line supports it. 在当前 TensorRT 版本线支持时，为当前 engine 创建 serialization config。

public TensorRtSerializationConfig CreateSerializationConfig()

Returns

TensorRtSerializationConfig

A managed serialization config wrapper. / 托管 serialization config 封装。

Remarks

This bridge currently supports this path for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11 的该路径。

Dispose()

Releases the TensorRT engine handle. 释放 TensorRT engine 句柄。

public void Dispose()

GetAliasedInputTensorName(string)

Gets the input tensor aliased by a TensorRT 11 output tensor, when plugin I/O aliasing is used. 在使用插件 I/O aliasing 时，获取 TensorRT 11 输出 tensor 所别名引用的输入 tensor 名称。

public string GetAliasedInputTensorName(string tensorName)

Parameters

tensorName string

The output tensor name to query. / 要查询的输出 tensor 名称。

Returns

string

The aliased input tensor name, or an empty string when TensorRT reports no alias. 被别名引用的输入 tensor 名称；如果 TensorRT 未报告 alias，则返回空字符串。

GetBindingReport(TensorRtExecutionContext, int, bool)

Builds a deployment binding report and attaches execution-context readiness. 构建部署绑定报告，并附加 execution context 就绪状态。

public TensorRtEngineBindingReport GetBindingReport(TensorRtExecutionContext context, int profileIndex, bool runShapeInference = false)

Parameters

context TensorRtExecutionContext

The execution context to inspect. 要检查的 execution context。

profileIndex int

The optimization profile index used for profile-specific metadata. 用于 profile 相关元数据的 optimization profile 索引。

runShapeInference bool

Whether to run TensorRT shape inference while collecting readiness. 是否在收集就绪状态时执行 TensorRT shape inference。

Returns

TensorRtEngineBindingReport

An engine binding report with readiness. 带就绪状态的 engine 绑定报告。

GetBindingReport(int)

Builds a deployment binding report for all engine I/O tensors. 为所有 engine I/O tensor 构建部署绑定报告。

public TensorRtEngineBindingReport GetBindingReport(int profileIndex)

Parameters

profileIndex int

The optimization profile index used for profile-specific metadata. 用于 profile 相关元数据的 optimization profile 索引。

Returns

TensorRtEngineBindingReport

An engine binding report. Engine 绑定报告。

GetCompatibilityBindingIOMode(int)

Gets a legacy-binding-compatible I/O mode for an engine binding index. 根据 engine binding 索引获取兼容 legacy binding 语义的输入输出模式。

public TensorRtIOMode GetCompatibilityBindingIOMode(int bindingIndex)

Parameters

bindingIndex int

The zero-based binding index. 从零开始的 binding 索引。

Returns

TensorRtIOMode

The input/output mode for the binding. 该 binding 的输入输出模式。

GetCompatibilityBindingInfo(int)

Gets a legacy-binding-compatible tensor snapshot for an engine binding index. 根据 engine binding 索引获取兼容 legacy binding 语义的 tensor 快照。

public TensorRtTensorInfo GetCompatibilityBindingInfo(int bindingIndex)

Parameters

bindingIndex int

The zero-based binding index. 从零开始的 binding 索引。

Returns

TensorRtTensorInfo

A safe managed tensor metadata snapshot. 安全的托管 tensor 元数据快照。

GetCompatibilityBindingName(int)

Gets a legacy-binding-compatible tensor name for an engine binding index. 根据 engine binding 索引获取兼容 legacy binding 语义的 tensor 名称。

public string GetCompatibilityBindingName(int bindingIndex)

Parameters

bindingIndex int

The zero-based binding index. 从零开始的 binding 索引。

Returns

string

The tensor or binding name reported by TensorRT. TensorRT 报告的 tensor 或 binding 名称。

GetCompatibilityBindingShape(int)

Gets a legacy-binding-compatible shape for an engine binding index. 根据 engine binding 索引获取兼容 legacy binding 语义的形状。

public TensorRtDims GetCompatibilityBindingShape(int bindingIndex)

Parameters

bindingIndex int

The zero-based binding index. 从零开始的 binding 索引。

Returns

TensorRtDims

The static or profile-independent shape reported by TensorRT. TensorRT 报告的静态或 profile-independent 形状。

GetDeploymentSnapshot(int)

Builds a TensorRT 11 deployment snapshot for this engine. 为当前 engine 构建 TensorRT 11 部署快照。

public TensorRtEngineDeploymentSnapshot GetDeploymentSnapshot(int profileIndex = 0)

Parameters

profileIndex int

The optimization profile used for profile-specific fields. 用于 profile 相关字段的 optimization profile 索引。

Returns

TensorRtEngineDeploymentSnapshot

A deployment snapshot with engine metadata and tensor bindings. 包含 engine 元数据和 tensor binding 的部署快照。

Remarks

This method intentionally aggregates many deployment-critical getters into one managed object so applications can log or validate an engine before binding buffers. 该方法会有意把大量部署关键 getter 聚合到一个托管对象中，便于应用在绑定缓冲区前记录或验证 engine。

GetDeviceMemorySizeForProfile(int)

Gets the device-memory requirement for a specific optimization profile. 获取指定 optimization profile 的设备内存需求。

public ulong GetDeviceMemorySizeForProfile(int profileIndex)

Parameters

profileIndex int

The optimization profile index. Optimization profile 索引。

Returns

ulong

The required bytes reported by TensorRT. TensorRT 报告的字节数。

GetDeviceMemorySizeForProfileV2(int)

Gets the TensorRT 10 V2 device-memory requirement for a specific optimization profile. 获取指定 optimization profile 的 TensorRT 10 V2 设备内存需求。

public ulong GetDeviceMemorySizeForProfileV2(int profileIndex)

Parameters

profileIndex int

The optimization profile index. Optimization profile 索引。

Returns

ulong

The required bytes reported by TensorRT. TensorRT 报告的字节数。

GetEngineStat(TensorRtEngineStat)

Gets a TensorRT engine statistic. 获取 TensorRT engine 统计项。

public long GetEngineStat(TensorRtEngineStat stat)

Parameters

stat TensorRtEngineStat

The statistic to query. / 要查询的统计项。

Returns

long

The statistic value returned by TensorRT. / TensorRT 返回的统计值。

Remarks

This specific statistic API is currently exposed only for TensorRT 11 because TensorRT 10 headers do not expose ICudaEngine::getEngineStat. 该统计接口当前仅面向 TensorRT 11 暴露，因为 TensorRT 10 头文件没有公开 ICudaEngine::getEngineStat。

GetIOTensorInfo(int)

Gets high-level metadata for one engine I/O tensor. 获取一个 engine I/O tensor 的高层元数据。

public TensorRtTensorInfo GetIOTensorInfo(int index)

Parameters

index int

The zero-based tensor index. 从零开始的 tensor 索引。

Returns

TensorRtTensorInfo

The tensor metadata. tensor 元数据。

GetIOTensorName(int)

Gets the name of one engine I/O tensor. 获取一个 engine I/O tensor 的名称。

public string GetIOTensorName(int index)

Parameters

index int

The zero-based tensor index. 从零开始的 tensor 索引。

Returns

string

The TensorRT tensor name. TensorRT tensor 名称。

GetIOTensors()

Returns metadata for every engine I/O tensor. 返回所有 engine I/O tensor 的元数据。

public IReadOnlyList<TensorRtTensorInfo> GetIOTensors()

Returns

IReadOnlyList<TensorRtTensorInfo>

A read-only list of tensor metadata. 只读 tensor 元数据列表。

GetProfileShape(string, int, TensorRtOptimizationProfileSelector)

Gets one optimization-profile shape for one engine tensor. 获取一个 engine tensor 在某个 optimization profile 中的形状。

public TensorRtDims GetProfileShape(string tensorName, int profileIndex, TensorRtOptimizationProfileSelector selector)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

profileIndex int

The optimization profile index. optimization profile 索引。

selector TensorRtOptimizationProfileSelector

The min/opt/max selector. min/opt/max 选择器。

Returns

TensorRtDims

The profile shape. profile 形状。

GetProfileShape64(string, int, TensorRtOptimizationProfileSelector)

Gets one TensorRT 11 engine profile shape with 64-bit dimension extents. 获取 TensorRT 11 引擎中某个 profile selector 的形状，并保留 64 位维度 extent。

public TensorRtDims64 GetProfileShape64(string tensorName, int profileIndex, TensorRtOptimizationProfileSelector selector)

Parameters

tensorName string

The engine tensor name. 引擎张量名称。

profileIndex int

The optimization profile index. 优化 profile 索引。

selector TensorRtOptimizationProfileSelector

The min/opt/max profile selector. min/opt/max profile 选择器。

Returns

TensorRtDims64

The profile shape reported by TensorRT. TensorRT 报告的 profile 形状。

GetProfileShapeDimensionExtent64(string, int, TensorRtOptimizationProfileSelector, int)

Gets one dimension extent from a TensorRT 11 engine profile shape as a 64-bit value. 以 64 位整数获取 TensorRT 11 引擎 profile shape 的单个维度 extent。

public long GetProfileShapeDimensionExtent64(string tensorName, int profileIndex, TensorRtOptimizationProfileSelector selector, int dimensionIndex)

Parameters

tensorName string

The engine tensor name. 引擎张量名称。

profileIndex int

The optimization profile index. 优化 profile 索引。

selector TensorRtOptimizationProfileSelector

The min/opt/max profile selector. min/opt/max profile 选择器。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The profile dimension extent reported by TensorRT. TensorRT 报告的 profile 维度 extent。

GetProfileTensorValuesV2(string, int, TensorRtOptimizationProfileSelector, int)

Gets TensorRT 11 profile tensor values V2 for a named shape tensor. 获取命名 shape tensor 的 TensorRT 11 profile tensor values V2。

public long[] GetProfileTensorValuesV2(string tensorName, int profileIndex, TensorRtOptimizationProfileSelector selector, int valueCount)

Parameters

tensorName string

The engine tensor name. Engine tensor 名称。

profileIndex int

The optimization profile index. Optimization profile 索引。

selector TensorRtOptimizationProfileSelector

The min/opt/max selector. Min/Opt/Max 选择器。

valueCount int

Number of values to copy. TensorRT does not expose a count query on ICudaEngine, so callers should pass the expected shape-value count. 要复制的值数量。TensorRT 在 ICudaEngine 上不提供数量查询，因此调用者应传入预期的 shape-value 数量。

Returns

long[]

Copied profile tensor values, or an empty array when TensorRT reports no values for the tensor. 复制出的 profile tensor values；当 TensorRT 未报告该 tensor 的值时返回空数组。

GetTensorBinding(int, int)

Builds a deployment binding diagnostic snapshot for one engine I/O tensor. 为一个 engine I/O tensor 构建部署绑定诊断快照。

public TensorRtEngineTensorBinding GetTensorBinding(int index, int profileIndex)

Parameters

index int

The engine I/O tensor index. Engine I/O tensor 索引。

profileIndex int

The optimization profile index used for profile-specific metadata. 用于 profile 相关元数据的 optimization profile 索引。

Returns

TensorRtEngineTensorBinding

A high-level tensor binding snapshot. 高层 tensor 绑定快照。

GetTensorBinding(string, int)

Builds a deployment binding diagnostic snapshot for one engine tensor. 为一个 engine tensor 构建部署绑定诊断快照。

public TensorRtEngineTensorBinding GetTensorBinding(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. Engine tensor 名称。

profileIndex int

The optimization profile index used for profile-specific metadata. 用于 profile 相关元数据的 optimization profile 索引。

Returns

TensorRtEngineTensorBinding

A high-level tensor binding snapshot. 高层 tensor 绑定快照。

GetTensorBytesPerComponent(string)

Gets the bytes-per-component value for one engine tensor. 获取一个 engine tensor 的每分量字节数。

public int GetTensorBytesPerComponent(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

int

The bytes-per-component value. 每分量字节数。

GetTensorBytesPerComponent(string, int)

Gets the profile-specific bytes-per-component value for one engine tensor. 获取一个 engine tensor 在指定 profile 下的每分量字节数。

public int GetTensorBytesPerComponent(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

profileIndex int

The optimization profile index. optimization profile 索引。

Returns

int

The bytes-per-component value. 每分量字节数。

GetTensorComponentsPerElement(string)

Gets the components-per-element value for one engine tensor. 获取一个 engine tensor 的每元素分量数。

public int GetTensorComponentsPerElement(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

int

The components-per-element value. 每元素分量数。

GetTensorComponentsPerElement(string, int)

Gets the profile-specific components-per-element value for one engine tensor. 获取一个 engine tensor 在指定 profile 下的每元素分量数。

public int GetTensorComponentsPerElement(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

profileIndex int

The optimization profile index. optimization profile 索引。

Returns

int

The components-per-element value. 每元素分量数。

GetTensorDataType(string)

Gets the TensorRT data type for one engine tensor. 获取一个 engine tensor 的 TensorRT 数据类型。

public TensorRtDataType GetTensorDataType(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

TensorRtDataType

The TensorRT tensor data type. TensorRT tensor 数据类型。

GetTensorDimensionExtent64(string, int)

Gets one TensorRT 11 engine tensor dimension extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 引擎张量的单个维度 extent。

public long GetTensorDimensionExtent64(string tensorName, int dimensionIndex)

Parameters

tensorName string

The engine tensor name. 引擎张量名称。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The dimension extent reported by TensorRT. TensorRT 报告的维度 extent。

GetTensorFormat(string)

Gets the TensorRT tensor format for one engine tensor. 获取一个 engine tensor 的 TensorRT tensor format。

public TensorRtTensorFormat GetTensorFormat(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

TensorRtTensorFormat

The TensorRT tensor format. TensorRT tensor 格式。

GetTensorFormat(string, int)

Gets the profile-specific TensorRT tensor format for one engine tensor. 获取一个 engine tensor 在指定 profile 下的 TensorRT tensor format。

public TensorRtTensorFormat GetTensorFormat(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

profileIndex int

The optimization profile index. optimization profile 索引。

Returns

TensorRtTensorFormat

The TensorRT tensor format. TensorRT tensor 格式。

GetTensorFormatDescription(string)

Gets TensorRT's human-readable tensor format description. 获取 TensorRT 返回的可读 tensor format 描述。

public string GetTensorFormatDescription(string tensorName)

Parameters

tensorName string

The engine tensor name. Engine tensor 名称。

Returns

string

The TensorRT tensor format description. TensorRT tensor format 描述。

GetTensorFormatDescription(string, int)

Gets TensorRT's profile-specific human-readable tensor format description. 获取 TensorRT 针对指定 profile 返回的可读 tensor format 描述。

public string GetTensorFormatDescription(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. Engine tensor 名称。

profileIndex int

The optimization profile index. Optimization profile 索引。

Returns

string

The TensorRT tensor format description. TensorRT tensor format 描述。

GetTensorIOMode(string)

Gets whether one engine tensor is an input or output tensor. 获取一个 engine tensor 是输入还是输出。

public TensorRtIOMode GetTensorIOMode(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

TensorRtIOMode

The TensorRT I/O mode. TensorRT I/O 模式。

GetTensorIndex(string)

Gets the zero-based index for one named engine tensor. 获取一个已命名 engine tensor 的从零开始索引。

public int GetTensorIndex(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

int

The zero-based tensor index. 从零开始的 tensor 索引。

GetTensorLocation(string)

Gets the TensorRT tensor location for one engine tensor. 获取一个 engine tensor 的 TensorRT tensor location。

public TensorRtTensorLocation GetTensorLocation(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

TensorRtTensorLocation

The TensorRT tensor location. TensorRT tensor 的内存位置。

GetTensorShape(string)

Gets the shape of one engine tensor. 获取一个 engine tensor 的形状。

public TensorRtDims GetTensorShape(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

TensorRtDims

The tensor shape. tensor 形状。

GetTensorShape64(string)

Gets a TensorRT 11 engine tensor shape with 64-bit dimension extents. 获取 TensorRT 11 引擎张量形状，并保留 64 位维度 extent。

public TensorRtDims64 GetTensorShape64(string tensorName)

Parameters

tensorName string

The engine tensor name. 引擎张量名称。

Returns

TensorRtDims64

The tensor shape reported by TensorRT. TensorRT 报告的张量形状。

GetTensorVectorizedDimension(string)

Gets the vectorized dimension index for one engine tensor. 获取一个 engine tensor 的向量化维度索引。

public int GetTensorVectorizedDimension(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

int

The vectorized dimension index, or TensorRT's sentinel value. 向量化维度索引，或 TensorRT 的哨兵值。

GetTensorVectorizedDimension(string, int)

Gets the profile-specific vectorized dimension index for one engine tensor. 获取一个 engine tensor 在指定 profile 下的向量化维度索引。

public int GetTensorVectorizedDimension(string tensorName, int profileIndex)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

profileIndex int

The optimization profile index. optimization profile 索引。

Returns

int

The vectorized dimension index, or TensorRT's sentinel value. 向量化维度索引，或 TensorRT 的哨兵值。

IsDebugTensor(string)

Gets whether TensorRT marks the named tensor as a debug tensor. 获取 TensorRT 是否将指定 tensor 标记为 debug tensor。

public bool IsDebugTensor(string tensorName)

Parameters

tensorName string

The engine tensor name. Engine tensor 名称。

Returns

bool

true when the tensor is a debug tensor. 如果该 tensor 是 debug tensor，则返回 true。

IsShapeInferenceIO(string)

Returns whether one engine tensor participates in shape inference. 返回一个 engine tensor 是否参与 shape inference。

public bool IsShapeInferenceIO(string tensorName)

Parameters

tensorName string

The engine tensor name. engine tensor 名称。

Returns

bool

true when the tensor is shape-inference I/O. 当该 tensor 是 shape inference I/O 时返回 true。

Serialize()

Serializes this TensorRT engine into host memory using TensorRT's default serialization behavior. 使用 TensorRT 默认序列化行为将当前 engine 序列化到 host memory。

public TensorRtHostMemory Serialize()

Returns

TensorRtHostMemory

A host-memory buffer containing the serialized engine. / 包含序列化 engine 的 host-memory 缓冲区。

Serialize(TensorRtSerializationConfig)

Serializes this engine using an explicit TensorRT serialization config. 使用显式 TensorRT serialization config 序列化当前 engine。

public TensorRtHostMemory Serialize(TensorRtSerializationConfig config)

Parameters

config TensorRtSerializationConfig

The serialization config to apply. / 要应用的 serialization config。

Returns

TensorRtHostMemory

A host-memory buffer containing the serialized engine. / 包含序列化 engine 的 host-memory 缓冲区。

Remarks

This overload is available for TensorRT 10 and TensorRT 11. TensorRT 8 callers should use Serialize(). 该重载适用于 TensorRT 10 和 TensorRT 11；TensorRT 8 调用方应使用 Serialize()。

SetWeightStreamingBudgetV2(long)

Sets the TensorRT weight-streaming budget. 设置 TensorRT 权重流式加载预算。

public bool SetWeightStreamingBudgetV2(long budgetBytes)

Parameters

budgetBytes long

Budget in bytes. / 预算字节数。

Returns

bool

true when TensorRT accepted the budget. / TensorRT 接受该预算时返回 true。

Remarks

Supported by this bridge for TensorRT 10 and TensorRT 11. 当前桥接库支持 TensorRT 10 和 TensorRT 11。