Table of Contents

Class TensorRtExecutionContext

Namespace
JYPPX.TensorRtSharp
Assembly
JYPPX.TensorRtSharp.dll

Represents a managed TensorRT Tensor Rt Execution Context wrapper. 表示托管 TensorRT Tensor Rt Execution Context 包装器。

public sealed class TensorRtExecutionContext : IDisposable
Inheritance
TensorRtExecutionContext
Implements
Inherited Members

Properties

AllInputDimensionsSpecified

Gets the All Input Dimensions Specified value. 获取 All Input Dimensions Specified 值。

public bool AllInputDimensionsSpecified { get; }

Property Value

bool

AllInputShapesSpecified

Gets or sets the All Input Shapes Specified value. 获取或设置 All Input Shapes Specified 值。

public bool AllInputShapesSpecified { get; }

Property Value

bool

BoundDeviceMemorySizeInBytes

Gets the size of the currently bound device-memory lease. 获取当前绑定的 device-memory lease 大小。

public int BoundDeviceMemorySizeInBytes { get; }

Property Value

int

DebugSync

Gets or sets the Debug Sync value. 获取或设置 Debug Sync 值。

public bool DebugSync { get; set; }

Property Value

bool

DeviceMemorySizeInBytes

Gets or sets the Device Memory Size In Bytes value. 获取或设置 Device Memory Size In Bytes 值。

public ulong DeviceMemorySizeInBytes { get; }

Property Value

ulong

EngineIOTensorCount

Gets the engine I/O tensor count reached through this execution context. 获取通过当前 execution context 反查到的 engine I/O tensor 数量。

public int EngineIOTensorCount { get; }

Property Value

int

EngineLayerCount

Gets the engine layer count reached through this execution context. 获取通过当前 execution context 反查到的 engine layer 数量。

public int EngineLayerCount { get; }

Property Value

int

EngineName

Gets the engine name reached through this execution context. 获取通过当前 execution context 反查到的 engine 名称。

public string EngineName { get; }

Property Value

string

EngineOptimizationProfileCount

Gets the engine optimization profile count reached through this execution context. 获取通过当前 execution context 反查到的 engine optimization profile 数量。

public int EngineOptimizationProfileCount { get; }

Property Value

int

EnqueueEmitsProfile

Gets or sets the Enqueue Emits Profile value. 获取或设置 Enqueue Emits Profile 值。

public bool EnqueueEmitsProfile { get; set; }

Property Value

bool

HasBoundDeviceMemory

Gets whether caller-owned device memory is currently bound. 获取当前是否绑定了调用方拥有的 device memory。

public bool HasBoundDeviceMemory { get; }

Property Value

bool

HasDebugListener

Gets whether this TensorRT 10/11 execution context has a debug listener attached. 获取当前 TensorRT 10/11 execution context 是否绑定了 debug listener;不会暴露 listener 指针或接管其生命周期。

public bool HasDebugListener { get; }

Property Value

bool

HasErrorRecorder

Gets whether a native TensorRT error recorder is attached to this execution context. 获取当前 execution context 是否绑定了 TensorRT 原生 error recorder;支持 TensorRT 8/10/11,不会暴露 recorder 指针或接管其生命周期。

public bool HasErrorRecorder { get; }

Property Value

bool

HasManagedDebugListener

Gets whether this wrapper owns a managed debug-listener borrow. 获取当前 wrapper 是否持有 managed debug-listener 借用。

public bool HasManagedDebugListener { get; }

Property Value

bool

HasNativeProfiler

Gets whether TensorRT currently reports a non-null native profiler pointer. 获取 TensorRT 当前是否报告非空 native profiler 指针。

public bool HasNativeProfiler { get; }

Property Value

bool

Remarks

This diagnostic does not expose the pointer and must not be used as a managed ownership signal. 该诊断不会暴露指针,且不应作为托管 ownership 信号使用。

HasProfiler

Gets whether this execution context has a profiler attached. 获取当前 execution context 是否绑定了由托管 wrapper 借出的 profiler;不会暴露 profiler 指针或接管其生命周期。

public bool HasProfiler { get; }

Property Value

bool

Remarks

This is the managed ownership signal used by SetProfiler(TensorRtProfiler) and ClearProfiler(). TensorRT may still report an internal native profiler through HasNativeProfiler after the managed borrow has been detached. 该属性表示 SetProfiler(TensorRtProfiler) 与 ClearProfiler() 管理的托管借用状态。即使托管借用已解除, TensorRT 仍可能通过 HasNativeProfiler 报告内部 native profiler。

HasRuntimeConfig

Gets whether this TensorRT 11 execution context has an associated runtime config object. 获取当前 TensorRT 11 execution context 是否有关联的 runtime config 对象。

public bool HasRuntimeConfig { get; }

Property Value

bool

HasTemporaryStorageAllocator

Gets whether this execution context has a temporary-storage allocator attached. 获取此 execution context 是否绑定了 temporary-storage allocator;适用于 TensorRT 8/10/11,不会暴露 allocator 指针或接管其生命周期。

public bool HasTemporaryStorageAllocator { get; }

Property Value

bool

InputConsumedEventAddressValue

Gets the native address value of the CUDA event TensorRT uses to signal input-consumption completion. 获取 TensorRT 用于通知输入消费完成的 CUDA event 原生地址诊断值。

public ulong InputConsumedEventAddressValue { get; }

Property Value

ulong

Remarks

The value is diagnostic only. It is intentionally exposed as an integer instead of a user-owned pointer. 该值仅用于诊断;它有意以整数形式暴露,而不是用户可拥有或解引用的指针。

IsInputConsumedEventSet

Gets whether an input-consumed CUDA event is currently set on this TensorRT 11 execution context. 获取此 TensorRT 11 execution context 当前是否设置了 input-consumed CUDA event。

public bool IsInputConsumedEventSet { get; }

Property Value

bool

Line

Gets or sets the Line value. 获取或设置 Line 值。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

Name

Gets or sets the Name value. 获取或设置 Name 值。

public string Name { get; set; }

Property Value

string

OptimizationProfileIndex

Gets or sets the Optimization Profile Index value. 获取或设置 Optimization Profile Index 值。

public int OptimizationProfileIndex { get; }

Property Value

int

PersistentCacheLimitInBytes

Gets or sets the Persistent Cache Limit In Bytes value. 获取或设置 Persistent Cache Limit In Bytes 值。

public ulong PersistentCacheLimitInBytes { get; set; }

Property Value

ulong

RetainedDeviceMemoryLeaseCount

Gets the number of current and retired memory leases retained until context disposal. 获取在 context 释放前保留的当前及历史 memory lease 数量。

public int RetainedDeviceMemoryLeaseCount { get; }

Property Value

int

RuntimeConfigAllocationStrategy

Gets the allocation strategy from the runtime config attached to this TensorRT 11 execution context. 获取附加到当前 TensorRT 11 execution context 的 runtime config allocation strategy。

public TensorRtExecutionContextAllocationStrategy RuntimeConfigAllocationStrategy { get; }

Property Value

TensorRtExecutionContextAllocationStrategy

Exceptions

TensorRtException

Thrown when TensorRT does not expose a runtime config for this context. 当 TensorRT 未为该 context 暴露 runtime config 时抛出。

SupportsTensorDebugState

Gets whether the active TensorRT line supports per-tensor debug state on execution contexts. 获取当前 TensorRT 版本线是否支持 execution context 上的逐 tensor debug state。

public bool SupportsTensorDebugState { get; }

Property Value

bool

Methods

ClearAuxStreams()

Clears user-provided auxiliary streams so TensorRT may use its default auxiliary-stream behavior. 清除用户提供的 auxiliary stream,让 TensorRT 回到默认 auxiliary-stream 行为。

public void ClearAuxStreams()

ClearCallbackState(string)

Clears supported callback attachments and returns a copied post-clear callback boundary snapshot. 清除受支持的回调附加项,并返回清除后的回调边界复制快照。

public TensorRtExecutionContextCallbackStateSnapshot ClearCallbackState(string outputTensorName)

Parameters

outputTensorName string

The output tensor name used for output allocator clearing. / 用于 output allocator 清理的输出 tensor 名称。

Returns

TensorRtExecutionContextCallbackStateSnapshot

A copied post-clear callback state snapshot. 清理后的回调状态复制快照。

Remarks

This method may call TensorRT clear operations for output allocator, temporary-storage allocator, and TensorRT 10/11 debug listener. It never destroys the callback objects and never calls IOutputAllocator::reallocateOutput, IOutputAllocator::notifyShape, or IDebugListener::processDebugTensor. 该方法可能调用 TensorRT 的 output allocator、temporary-storage allocator 和 TensorRT 10/11 debug listener 清理操作; 它不会销毁回调对象,也不会调用 IOutputAllocator::reallocateOutput、IOutputAllocator::notifyShape 或 IDebugListener::processDebugTensor。

ClearDebugListener()

Clears the debug listener attached to this TensorRT 10/11 execution context. 清除绑定到当前 TensorRT 10/11 execution context 的 debug listener;不会销毁 listener 或调用用户回调。

public bool ClearDebugListener()

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearDeviceMemory()

Clears the externally supplied device-memory block for this TensorRT 10 or 11 execution context. 清除当前 TensorRT 10 或 11 execution context 外部传入的 device memory 块。

public void ClearDeviceMemory()

Remarks

Retired SafeHandle leases remain alive until context disposal, so a premature clear cannot free memory still used by queued inference. The native binding is nevertheless cleared immediately and must be rebound before inference. 历史 SafeHandle lease 会保留到 context 释放,因此提前清理不会释放已排队推理仍在使用的内存;但 native 绑定会立即 清空,继续推理前必须重新绑定。

ClearErrorRecorder()

Clears the native TensorRT error recorder attached to this execution context. 清除当前 execution context 上绑定的 TensorRT 原生 error recorder;支持 TensorRT 8/10/11,不会销毁 recorder 或接管其生命周期。

public void ClearErrorRecorder()

ClearInputConsumedEvent()

Clears the CUDA event used by TensorRT to signal input-consumption completion. 清除 TensorRT 用于通知输入消费完成的 CUDA event。

public bool ClearInputConsumedEvent()

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearInputTensorAddress(string)

Clears the CUDA memory address currently bound to a named input tensor. 清除当前绑定到指定输入 tensor 的 CUDA 内存地址。

public bool ClearInputTensorAddress(string tensorName)

Parameters

tensorName string

The input tensor name. 输入 tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearOutputAllocator(string)

Clears the output allocator attached to a named output tensor. 清除绑定到指定输出 tensor 的 output allocator;适用于 TensorRT 8/10/11,不会销毁 allocator 或调用用户 reallocate 回调。

public bool ClearOutputAllocator(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearOutputTensorAddress(string)

Clears the CUDA memory address currently bound to a named output tensor. 清除当前绑定到指定输出 tensor 的 CUDA 内存地址。

public bool ClearOutputTensorAddress(string tensorName)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearProfiler()

Clears the profiler attached to this execution context. 清除绑定到当前 execution context 的 profiler;不会销毁 profiler 或调用用户回调。

public void ClearProfiler()

ClearTemporaryStorageAllocator()

Clears the temporary-storage allocator attached to this execution context. 清除绑定到当前 execution context 的 temporary-storage allocator;适用于 TensorRT 8/10/11,不会销毁 allocator 或调用用户 deallocate 回调。

public bool ClearTemporaryStorageAllocator()

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearTensorAddress(string)

Clears the CUDA memory address currently bound to a named input or output tensor. 清除当前绑定到指定输入或输出 tensor 的 CUDA 内存地址。

public bool ClearTensorAddress(string tensorName)

Parameters

tensorName string

The tensor name. Tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

Remarks

This is a safe high-level wrapper over TensorRT 11 setTensorAddress(name, nullptr). 这是 TensorRT 11 setTensorAddress(name, nullptr) 的安全高层封装。

Dispose()

Releases the native TensorRT resources held by this object. 释放此对象持有的 native TensorRT 资源。

public void Dispose()

EnqueueAsync(CudaStream)

Enqueues the TensorRT execution work. 将 TensorRT execution work 加入队列。

public void EnqueueAsync(CudaStream stream)

Parameters

stream CudaStream

ExecuteLegacy(int)

Executes a TensorRT 8 implicit-batch network synchronously. 同步执行 TensorRT 8 implicit-batch 网络。

public void ExecuteLegacy(int batchSize)

Parameters

batchSize int

The positive legacy batch size. 正数 legacy batch size。

ExecuteV2()

Executes an explicit-batch network synchronously using the tensor addresses already bound to this context. 使用已绑定到当前 context 的 tensor 地址同步执行 explicit-batch 网络。

public void ExecuteV2()

Remarks

The native bridge copies bound addresses into a temporary TensorRT binding array and does not expose device pointers. native bridge 将已绑定地址复制到临时 TensorRT binding 数组,不向托管层暴露 device pointer。

GetAuxiliaryStreamAssignmentSnapshot()

Gets a pointer-free snapshot of auxiliary-stream assignment and managed handle-lease state. 获取不含指针的辅助 stream 分配与托管 handle lease 状态快照。

public TensorRtAuxiliaryStreamAssignmentSnapshot GetAuxiliaryStreamAssignmentSnapshot()

Returns

TensorRtAuxiliaryStreamAssignmentSnapshot

GetCallbackAllocatorSafeControlSummary(string)

Gets a high-level pointer-free summary of callback allocator safe controls for one output tensor. 获取单个 output tensor 的 callback allocator 安全控制高层无指针摘要。

public TensorRtExecutionContextCallbackAllocatorSafeControlSummary GetCallbackAllocatorSafeControlSummary(string outputTensorName)

Parameters

outputTensorName string

The output tensor name used for output allocator queries. / 用于 output allocator 查询的输出 tensor 名称。

Returns

TensorRtExecutionContextCallbackAllocatorSafeControlSummary

A copied safe-control summary. 复制式安全控制摘要。

Remarks

This helper aggregates copied metadata only from existing safe read-only queries. Borrowed pointer not exposed/owned, no callback invocation is attempted, and the result is not runtime proof of TensorRT callback execution. 该 helper 只聚合现有安全只读查询复制出的元数据;不会暴露或拥有 borrowed pointer,不会尝试 callback 调用, 其结果也不是 TensorRT callback 已真实执行的 runtime proof。

GetCallbackStateSnapshot(string)

Gets a copied snapshot of output allocator, temporary-storage allocator, and debug listener boundary state. 获取 output allocator、temporary-storage allocator 与 debug listener 边界状态的复制快照。

public TensorRtExecutionContextCallbackStateSnapshot GetCallbackStateSnapshot(string outputTensorName)

Parameters

outputTensorName string

The output tensor name used for output allocator queries. / 用于 output allocator 查询的输出 tensor 名称。

Returns

TensorRtExecutionContextCallbackStateSnapshot

A copied callback state snapshot. 复制出的回调状态快照。

Remarks

This query copies state and interface metadata immediately. It does not expose, retain, or take ownership of borrowed allocator/debug-listener pointers and it does not invoke output allocator or debug listener callbacks. 该查询会立即复制状态与 interface 元数据,不会暴露、保留或接管 borrowed allocator/debug-listener 指针,也不会调用 output allocator 或 debug listener 回调。

GetDeploymentSnapshot(TensorRtEngine)

Builds a TensorRT 11 deployment snapshot for this execution context. 为当前 execution context 构建 TensorRT 11 部署快照。

public TensorRtExecutionContextDeploymentSnapshot GetDeploymentSnapshot(TensorRtEngine engine)

Parameters

engine TensorRtEngine

The engine used to enumerate tensor metadata. 用于枚举 tensor 元数据的 engine。

Returns

TensorRtExecutionContextDeploymentSnapshot

A deployment snapshot with context readiness, address state, and runtime metadata. 包含 context 就绪状态、地址状态和运行时元数据的部署快照。

GetMaxOutputSize(string)

Queries TensorRT's maximum output buffer size estimate for a named output tensor in the current context. 查询当前 execution context 中指定输出 tensor 的最大输出缓冲区大小估计值。

public long GetMaxOutputSize(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

Returns

long

The maximum size in bytes, or a negative value when TensorRT cannot determine the value for the current shape state. 最大字节数;当 TensorRT 在当前 shape 状态下无法确定该值时,可能返回负值。

GetNvtxVerbosity()

Gets the NVTX verbosity currently used by this TensorRT execution context. 获取当前 TensorRT execution context 使用的 NVTX 详细程度。

public TensorRtProfilingVerbosity GetNvtxVerbosity()

Returns

TensorRtProfilingVerbosity

GetOutputTensorAddressValue(string)

Gets the native address value currently bound to a named TensorRT output tensor for diagnostics. 获取当前绑定到指定 TensorRT 输出 tensor 的原生地址数值,仅用于诊断。

public ulong GetOutputTensorAddressValue(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

ulong

The address value reported by TensorRT, or 0 when no address is bound. / TensorRT 报告的地址数值;未绑定时返回 0。

GetReadiness(TensorRtEngine, bool)

Creates a deployment readiness snapshot for this execution context and engine. 为当前 execution context 与 engine 创建部署就绪状态快照。

public TensorRtExecutionContextReadiness GetReadiness(TensorRtEngine engine, bool runShapeInference = false)

Parameters

engine TensorRtEngine

The engine that created this context. 创建该 context 的 engine。

runShapeInference bool

True to call TensorRT inferShapes before reporting missing-shape count. 为 true 时先调用 TensorRT inferShapes 并报告缺失 shape 数量。

Returns

TensorRtExecutionContextReadiness

A readiness snapshot suitable for diagnostics before enqueue. 可用于 enqueue 前诊断的就绪状态快照。

GetRuntimeDiagnosticSnapshot(string)

Gets a pointer-free runtime diagnostic snapshot for one output tensor. 获取单个 output tensor 对应的 pointer-free 运行时诊断快照。

public TensorRtExecutionContextRuntimeDiagnosticSnapshot GetRuntimeDiagnosticSnapshot(string outputTensorName)

Parameters

outputTensorName string

The output tensor name used for output-address and callback-state queries. / 用于输出地址与回调状态查询的 output tensor 名称。

Returns

TensorRtExecutionContextRuntimeDiagnosticSnapshot

A copied runtime diagnostic snapshot. 复制出的运行时诊断快照。

Remarks

This helper aggregates existing safe read-only queries. It does not expose, retain, or take ownership of borrowed TensorRT pointers, and address values are emitted only as integer diagnostics. 该 helper 聚合既有安全只读查询;不会暴露、保留或接管 TensorRT borrowed pointer,地址也仅以整数诊断值返回。

GetShapeBinding(int)

Gets TensorRT 8 legacy shape-binding values for a binding index. 获取 TensorRT 8 legacy shape binding 的运行时取值。

public int[] GetShapeBinding(int bindingIndex)

Parameters

bindingIndex int

The legacy binding index. legacy binding 索引。

Returns

int[]

Caller-owned copied shape-binding values. 调用方拥有的 shape-binding 值副本。

GetTensorAddressValue(string)

Gets the native address value currently bound to a named TensorRT tensor for diagnostics. 获取当前绑定到指定 TensorRT tensor 的原生地址数值,仅用于诊断。

public ulong GetTensorAddressValue(string tensorName)

Parameters

tensorName string

The input or output tensor name. / 输入或输出 tensor 名称。

Returns

ulong

The address value reported by TensorRT, or 0 when no address is bound. TensorRT 报告的地址数值;未绑定地址时返回 0。

Remarks

This value is intentionally exposed as an integer diagnostic value instead of a user-owned pointer. 该值特意以整数诊断值形式暴露,而不是用户可拥有或解引用的指针。

GetTensorDebugState(string)

Gets the TensorRT debug state for a named tensor on supported execution contexts. 获取受支持 execution context 上指定 tensor 的 debug state。

public bool GetTensorDebugState(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

Returns

bool

true when debug state is enabled. / 启用 debug state 时返回 true。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

GetTensorShape(string)

Gets the Tensor Shape value. 获取 Tensor Shape 值。

public TensorRtDims GetTensorShape(string tensorName)

Parameters

tensorName string

Returns

TensorRtDims

GetTensorShape64(string)

Gets a TensorRT 11 execution-context tensor shape with 64-bit dimension extents. 获取 TensorRT 11 execution context 中张量的形状,并保留 64 位维度 extent。

public TensorRtDims64 GetTensorShape64(string tensorName)

Parameters

tensorName string

The context tensor name. Context 张量名称。

Returns

TensorRtDims64

The runtime tensor shape reported by TensorRT. TensorRT 报告的运行时张量形状。

GetTensorShapeDimensionExtent64(string, int)

Gets one TensorRT 11 runtime tensor shape extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 运行时张量形状的单个 extent。

public long GetTensorShapeDimensionExtent64(string tensorName, int dimensionIndex)

Parameters

tensorName string

The context tensor name. Context 张量名称。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The runtime shape extent reported by TensorRT. TensorRT 报告的运行时 shape extent。

GetTensorStrideDimensionExtent64(string, int)

Gets one TensorRT 11 runtime stride extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 运行时张量 stride 的单个 extent。

public long GetTensorStrideDimensionExtent64(string tensorName, int dimensionIndex)

Parameters

tensorName string

The context tensor name. Context 张量名称。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The runtime stride extent reported by TensorRT. TensorRT 报告的运行时 stride extent。

GetTensorStrides(string)

Gets the Tensor Strides value. 获取 Tensor Strides 值。

public TensorRtDims GetTensorStrides(string tensorName)

Parameters

tensorName string

Returns

TensorRtDims

GetTensorStrides64(string)

Gets TensorRT 11 runtime tensor strides with 64-bit values. 获取 TensorRT 11 运行时张量 strides,并保留 64 位值。

public TensorRtDims64 GetTensorStrides64(string tensorName)

Parameters

tensorName string

The context tensor name. Context 张量名称。

Returns

TensorRtDims64

The runtime tensor strides reported by TensorRT. TensorRT 报告的运行时 tensor strides。

GetUnfusedTensorsDebugState()

Gets debug state for TensorRT 11 unfused debug tensors. 获取 TensorRT 11 未融合 debug tensor 的 debug state。

public bool GetUnfusedTensorsDebugState()

Returns

bool

HasManagedOutputAllocator(string)

Gets whether this wrapper owns a managed output allocator borrow for a tensor. 获取当前 wrapper 是否为指定 tensor 持有 managed output allocator 借用。

public bool HasManagedOutputAllocator(string tensorName)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

Returns

bool

Whether an owner is retained. 是否持有 owner。

HasOutputAllocator(string)

Gets whether a named output tensor uses a TensorRT output allocator. 获取指定输出 tensor 是否使用 TensorRT output allocator;适用于 TensorRT 8/10/11,不会暴露 allocator 指针或接管其生命周期。

public bool HasOutputAllocator(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true if TensorRT reports an output allocator. / TensorRT 报告存在 output allocator 时返回 true。

InferShapes()

Runs TensorRT shape inference for the current execution context. 对当前 execution context 执行 TensorRT shape inference。

public int InferShapes()

Returns

int

The number of tensors whose shapes are still insufficiently specified. 仍未完全指定 shape 的 tensor 数量。

IsOutputTensorAddressSet(string)

Gets whether a named output tensor has an explicit device address bound. 获取指定输出 tensor 是否已绑定显式设备地址。

public bool IsOutputTensorAddressSet(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true if TensorRT reports an output address. / TensorRT 报告已有输出地址时返回 true。

IsTensorAddressBound(string)

Checks whether Tensor Address Bound is true. 检查 Tensor Address Bound 是否为 true。

public bool IsTensorAddressBound(string tensorName)

Parameters

tensorName string

Returns

bool

ReportToProfiler()

Reports accumulated execution data to the profiler attached to the context, when one exists. 将当前执行上下文已累计的执行数据上报给关联的 profiler(如果存在)。

public bool ReportToProfiler()

Returns

bool

true when TensorRT reports that profiler data was emitted; otherwise false. 当 TensorRT 确认已上报 profiler 数据时返回 true,否则返回 false。

SetAllTensorsDebugState(bool)

Enables or disables TensorRT debug state for all tensors on supported execution contexts. 在受支持的 execution context 上启用或禁用所有 tensor 的 debug state。

public void SetAllTensorsDebugState(bool enabled)

Parameters

enabled bool

Whether debug state should be enabled. / 是否启用 debug state。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

SetAuxStreams(params CudaStream[])

Sets auxiliary CUDA streams used by TensorRT 8, 10, or 11 during inference. 设置 TensorRT 8、10 或 11 推理阶段使用的辅助 CUDA stream。

public void SetAuxStreams(params CudaStream[] streams)

Parameters

streams CudaStream[]

Auxiliary CUDA streams. 辅助 CUDA stream。

SetAuxStreams(IReadOnlyList<CudaStream>)

Sets auxiliary CUDA streams used by TensorRT 8, 10, or 11 during inference. 设置 TensorRT 8、10 或 11 推理阶段使用的辅助 CUDA stream。

public void SetAuxStreams(IReadOnlyList<CudaStream> streams)

Parameters

streams IReadOnlyList<CudaStream>

Caller-owned auxiliary streams. The context keeps their native handles alive while TensorRT may borrow them. 调用方拥有的辅助 stream;在 TensorRT 可能借用期间,context 会保持其 native handle 存活。

Remarks

Passing an empty collection clears user-provided auxiliary streams. TensorRT may still use its default internal behavior. 传入空集合会清除用户提供的辅助 stream;TensorRT 仍可能使用自身默认内部行为。

SetBindingDimensions(int, TensorRtDims)

Sets the Binding Dimensions value. 设置 Binding Dimensions 值。

public void SetBindingDimensions(int bindingIndex, TensorRtDims dims)

Parameters

bindingIndex int
dims TensorRtDims

SetDebugListener(TensorRtDebugListenerCallbackOwner)

Attaches an owner-safe managed debug listener to this TensorRT 10/11 execution context. 将 owner-safe 托管 debug listener 绑定到当前 TensorRT 10/11 execution context。

public void SetDebugListener(TensorRtDebugListenerCallbackOwner listener)

Parameters

listener TensorRtDebugListenerCallbackOwner

The callback owner borrowed by TensorRT until clear or context disposal. TensorRT 借用到清理或 context dispose 为止的 callback owner。

SetDeviceMemory(CudaMemory)

Sets caller-owned device memory and retains its native owner until this context is disposed. 设置调用方拥有的 device memory,并在当前 context 释放前保持其 native owner 有效。

public void SetDeviceMemory(CudaMemory memory)

Parameters

memory CudaMemory

The CUDA memory borrowed by TensorRT. TensorRT 借用的 CUDA 内存。

Remarks

TensorRT 8 uses setDeviceMemory; TensorRT 10 and 11 use the size-aware setDeviceMemoryV2 implementation behind the stable compatibility method. TensorRT 8 使用 setDeviceMemory;TensorRT 10/11 在稳定兼容方法后使用带 size 的 setDeviceMemoryV2 实现。

SetDeviceMemoryV2(CudaMemory)

Sets caller-owned device memory through the explicit TensorRT 10/11 setDeviceMemoryV2 bridge. 通过显式 TensorRT 10/11 setDeviceMemoryV2 bridge 设置调用方拥有的 device memory。

public void SetDeviceMemoryV2(CudaMemory memory)

Parameters

memory CudaMemory

The CUDA memory borrowed by TensorRT. TensorRT 借用的 CUDA 内存。

Remarks

The context retains a SafeHandle lease. Disposing memory after this call does not free the native allocation while the context or any queued inference may still use it. TensorRT 8 callers should use SetDeviceMemory(CudaMemory). Context 会保留 SafeHandle lease;调用后释放 memory 不会在 context 或已排队推理仍可能使用时 提前释放 native allocation。TensorRT 8 调用方应使用 SetDeviceMemory(CudaMemory)。

SetInputConsumedEvent(CudaEvent)

Sets the Input Consumed Event value. 设置 Input Consumed Event 值。

public void SetInputConsumedEvent(CudaEvent cudaEvent)

Parameters

cudaEvent CudaEvent

SetInputShape(string, TensorRtDims)

Sets the Input Shape value. 设置 Input Shape 值。

public void SetInputShape(string tensorName, TensorRtDims dims)

Parameters

tensorName string
dims TensorRtDims

SetInputShapeBinding(int, IReadOnlyList<int>)

Sets copied values for a TensorRT 8 legacy input shape binding. 为 TensorRT 8 legacy input shape binding 设置复制后的值。

public bool SetInputShapeBinding(int bindingIndex, IReadOnlyList<int> values)

Parameters

bindingIndex int

The legacy input shape-binding index. legacy input shape-binding 索引。

values IReadOnlyList<int>

Caller-owned shape values copied for the native call. 调用方拥有且会被复制用于原生调用的 shape 值。

Returns

bool

true when TensorRT accepts the values. TensorRT 接受这些值时返回 true。

SetInputShapeBinding(int, params int[])

Sets copied TensorRT 8 legacy input shape-binding values. 设置复制后的 TensorRT 8 legacy input shape-binding 值。

public bool SetInputShapeBinding(int bindingIndex, params int[] values)

Parameters

bindingIndex int

The legacy input shape-binding index. legacy input shape-binding 索引。

values int[]

Caller-owned shape values. 调用方拥有的 shape 值。

Returns

bool

true when TensorRT accepts the values. TensorRT 接受这些值时返回 true。

SetInputTensorAddress(string, CudaMemory)

Binds a CUDA allocation to a named input tensor. 将 CUDA 设备内存分配绑定到指定输入 tensor。

public void SetInputTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string

The input tensor name. 输入 tensor 名称。

memory CudaMemory

The CUDA allocation used as the input buffer. 作为输入缓冲区的 CUDA 设备内存。

SetNvtxVerbosity(TensorRtProfilingVerbosity)

Sets the NVTX verbosity used by this TensorRT execution context. 设置当前 TensorRT execution context 使用的 NVTX 详细程度。

public bool SetNvtxVerbosity(TensorRtProfilingVerbosity verbosity)

Parameters

verbosity TensorRtProfilingVerbosity

The desired NVTX verbosity. / 期望的 NVTX 详细程度。

Returns

bool

true when TensorRT accepts the value. / TensorRT 接受该值时返回 true。

SetOptimizationProfileAsync(int, CudaStream)

Sets the Optimization Profile Async value. 设置 Optimization Profile Async 值。

public void SetOptimizationProfileAsync(int profileIndex, CudaStream stream)

Parameters

profileIndex int
stream CudaStream

SetOutputAllocator(string, TensorRtOutputAllocatorCallbackOwner)

Attaches an owner-safe managed output allocator to a named output tensor. 将 owner-safe 托管 output allocator 绑定到指定输出 tensor。

public void SetOutputAllocator(string tensorName, TensorRtOutputAllocatorCallbackOwner owner)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

owner TensorRtOutputAllocatorCallbackOwner

The callback owner borrowed until clear or context disposal. 借用到 clear 或 context dispose 为止的 callback owner。

SetOutputTensorAddress(string, CudaMemory)

Binds a CUDA allocation to a named output tensor. 将 CUDA 设备内存分配绑定到指定输出 tensor。

public void SetOutputTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

memory CudaMemory

The CUDA allocation used as the output buffer. 作为输出缓冲区的 CUDA 设备内存。

SetProfiler(TensorRtProfiler)

Attaches a managed TensorRT profiler to this execution context. 将托管 TensorRT profiler 绑定到当前 execution context。

public void SetProfiler(TensorRtProfiler profiler)

Parameters

profiler TensorRtProfiler

The managed profiler to borrow. 要借用的托管 profiler。

Remarks

TensorRT borrows the native profiler pointer and does not take ownership. This execution context keeps the managed profiler alive until ClearProfiler() or Dispose() detaches it. Dispose the execution context or clear the profiler before disposing the profiler when possible; if the profiler is disposed first, native release is deferred until this context detaches it. TensorRT 只借用 native profiler 指针,不接管所有权。当前 execution context 会保持托管 profiler 存活,直到 ClearProfiler() 或 Dispose() 解除绑定。建议先释放 execution context 或清除 profiler 再释放 profiler; 如果先释放 profiler,native 释放会延迟到 context 解除绑定之后。

SetTensorAddress(string, CudaMemory)

Sets the Tensor Address value. 设置 Tensor Address 值。

public void SetTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string
memory CudaMemory

SetTensorDebugState(string, bool)

Enables or disables TensorRT debug state for a named tensor on supported execution contexts. 在受支持的 execution context 上启用或禁用指定 tensor 的 debug state。

public void SetTensorDebugState(string tensorName, bool enabled)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

enabled bool

Whether debug state should be enabled. / 是否启用 debug state。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

SetUnfusedTensorsDebugState(bool)

Enables or disables debug state for TensorRT 11 unfused debug tensors. 启用或禁用 TensorRT 11 未融合 debug tensor 的 debug state。

public bool SetUnfusedTensorsDebugState(bool enabled)

Parameters

enabled bool

Whether unfused tensor debug state should be enabled. / 是否启用未融合 tensor debug state。

Returns

bool

true when TensorRT accepts the setting. / TensorRT 接受该设置时返回 true。

TryGetCallbackStateSnapshot(string, out TensorRtExecutionContextCallbackStateSnapshot)

Tries to get a copied callback-state snapshot, including partial state when an individual native query fails. 尝试获取回调状态副本;单个原生查询失败时仍返回已经复制出的 partial state。

public bool TryGetCallbackStateSnapshot(string outputTensorName, out TensorRtExecutionContextCallbackStateSnapshot snapshot)

Parameters

outputTensorName string

The output tensor name used for output allocator queries. / 用于 output allocator 查询的输出 tensor 名称。

snapshot TensorRtExecutionContextCallbackStateSnapshot

The complete or partial pointer-free snapshot. / 完整或部分可用的 pointer-free 快照。

Returns

bool

true only when every snapshot phase completed successfully. / 仅当所有快照阶段均成功时返回 true。

TryGetCallbackStateSnapshot(string, out TensorRtExecutionContextCallbackStateSnapshot, out string)

Tries to get a copied callback-state snapshot, including partial state and a failure diagnostic. 尝试获取回调状态副本,并在失败时保留 partial state 与诊断信息。

public bool TryGetCallbackStateSnapshot(string outputTensorName, out TensorRtExecutionContextCallbackStateSnapshot snapshot, out string diagnostic)

Parameters

outputTensorName string

The output tensor name used for output allocator queries. / 用于 output allocator 查询的输出 tensor 名称。

snapshot TensorRtExecutionContextCallbackStateSnapshot

The complete or partial pointer-free snapshot. / 完整或部分可用的 pointer-free 快照。

diagnostic string

A copied native diagnostic for the first failed phase. / 首个失败阶段的原生诊断副本。

Returns

bool

true only when every snapshot phase completed successfully. / 仅当所有快照阶段均成功时返回 true。

Remarks

A false result does not invalidate fields collected before or after an optional interface-info failure. Check LastOperation and LastStatus before consuming partial metadata. 返回 false 不会使可选 interface-info 失败前后已采集的字段失效;使用 partial metadata 前应检查 LastOperation 与 LastStatus。

TryGetDebugListenerInterfaceInfo(out TensorRtInterfaceInfo)

Tries to get copied versioned-interface metadata for the debug listener attached to this execution context. 尝试获取当前 execution context 已绑定 debug listener 的 versioned-interface 元数据副本。

public bool TryGetDebugListenerInterfaceInfo(out TensorRtInterfaceInfo interfaceInfo)

Parameters

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

Returns

bool

true when TensorRT reports a debug listener and metadata was copied. / TensorRT 报告已绑定 debug listener 且成功复制元数据时返回 true。

TryGetDebugListenerInterfaceInfo(out TensorRtInterfaceInfo, out string)

Tries to get copied versioned-interface metadata for the debug listener attached to this execution context. 尝试获取当前 execution context 已绑定 debug listener 的 versioned-interface 元数据副本。

public bool TryGetDebugListenerInterfaceInfo(out TensorRtInterfaceInfo interfaceInfo, out string diagnostic)

Parameters

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

diagnostic string

A short diagnostic string describing success or the reason for failure. / 描述成功或失败原因的简短诊断。

Returns

bool

true when TensorRT reports a debug listener and metadata was copied. / TensorRT 报告已绑定 debug listener 且成功复制元数据时返回 true。

Remarks

This query copies TensorRT metadata immediately and does not expose, retain, or take ownership of the borrowed debug-listener pointer. 该查询会立即复制 TensorRT 元数据,不会暴露、保留或接管 borrowed debug-listener 指针。

TryGetDebugListenerVersionedMetadata(out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from the attached debug listener. 尝试复制已附加 debug listener 的元数据。

public bool TryGetDebugListenerVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata)

Parameters

metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetDebugListenerVersionedMetadata(out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from the attached debug listener. 尝试复制已附加 debug listener 的元数据。

public bool TryGetDebugListenerVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

metadata TensorRtVersionedInterfaceMetadata
diagnostic string

Returns

bool

TryGetErrorBuffer(out string)

Probes the retained execution-context error-buffer compatibility API. 探测保留的 execution context error-buffer 兼容 API。

public bool TryGetErrorBuffer(out string errorBuffer)

Parameters

errorBuffer string

The copied error text, or an empty string when unavailable. / 复制出的错误文本;不可用时为空字符串。

Returns

bool

false for the currently supported standard TensorRT execution-context types. / 对当前支持的标准 TensorRT execution context 类型返回 false。

Remarks

The standard nvinfer1::IExecutionContext type used by this bridge does not expose the safe-runtime getErrorBuffer query in the supported vendor headers. The ABI and managed method remain available only to return a controlled deferred diagnostic; no borrowed vendor pointer is accessed or exposed. 本 bridge 使用的标准 nvinfer1::IExecutionContext 类型在已支持 vendor headers 中不提供 safe-runtime getErrorBuffer 查询。ABI 与托管方法仅保留用于返回受控 deferred 诊断,不访问或暴露 vendor borrowed pointer。

TryGetErrorBuffer(out string, out string)

Probes the retained execution-context error-buffer compatibility API and returns its diagnostic. 探测保留的 execution context error-buffer 兼容 API,并返回诊断信息。

public bool TryGetErrorBuffer(out string errorBuffer, out string diagnostic)

Parameters

errorBuffer string

The copied error text, or an empty string when unavailable. / 复制出的错误文本;不可用时为空字符串。

diagnostic string

A short diagnostic string describing success or failure. / 描述成功或失败原因的简短诊断。

Returns

bool

false while the vendor query remains deferred. / vendor 查询保持 deferred 时返回 false。

TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot)

Attempts to collect a copied read-only snapshot from this execution context's TensorRT error recorder. 尝试从当前 execution context 的 TensorRT error recorder 采集只读托管快照。

public bool TryGetErrorRecorderSnapshot(out TensorRtErrorRecorderSnapshot snapshot)

Parameters

snapshot TensorRtErrorRecorderSnapshot

The copied snapshot. 已复制到托管内存的快照。

Returns

bool

true when a recorder was attached; otherwise false. 附加了 recorder 时返回 true,否则返回 false。

Remarks

This method does not expose, retain, increment, decrement, or destroy the native recorder pointer. 此方法不会暴露、持有、增加引用、减少引用或销毁原生 recorder 指针;支持 TensorRT 8/10/11。

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata)

Parameters

metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from the attached error recorder. 尝试复制已附加 error recorder 的元数据。

public bool TryGetErrorRecorderVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

metadata TensorRtVersionedInterfaceMetadata
diagnostic string

Returns

bool

TryGetOutputAllocatorInterfaceInfo(string, out TensorRtInterfaceInfo)

Tries to get copied versioned-interface metadata for the output allocator attached to a named output tensor. 尝试获取指定输出 tensor 已绑定 output allocator 的 versioned-interface 元数据副本。

public bool TryGetOutputAllocatorInterfaceInfo(string tensorName, out TensorRtInterfaceInfo interfaceInfo)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

Returns

bool

true when TensorRT reports an output allocator for the tensor and metadata was copied. / TensorRT 报告该 tensor 已绑定 output allocator 且成功复制元数据时返回 true。

TryGetOutputAllocatorInterfaceInfo(string, out TensorRtInterfaceInfo, out string)

Tries to get copied versioned-interface metadata for the output allocator attached to a named output tensor. 尝试获取指定输出 tensor 已绑定 output allocator 的 versioned-interface 元数据副本。

public bool TryGetOutputAllocatorInterfaceInfo(string tensorName, out TensorRtInterfaceInfo interfaceInfo, out string diagnostic)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

diagnostic string

A short diagnostic string describing success or the reason for failure. / 描述成功或失败原因的简短诊断。

Returns

bool

true when TensorRT reports an output allocator for the tensor and metadata was copied. / TensorRT 报告该 tensor 已绑定 output allocator 且成功复制元数据时返回 true。

Remarks

This query copies TensorRT metadata immediately and does not expose, retain, or take ownership of the borrowed allocator pointer. 该查询会立即复制 TensorRT 元数据,不会暴露、保留或接管 borrowed allocator 指针。

TryGetOutputAllocatorVersionedMetadata(string, out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from a named output allocator. 尝试复制指定 output allocator 的元数据。

public bool TryGetOutputAllocatorVersionedMetadata(string tensorName, out TensorRtVersionedInterfaceMetadata metadata)

Parameters

tensorName string
metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetOutputAllocatorVersionedMetadata(string, out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from a named output allocator. 尝试复制指定 output allocator 的元数据。

public bool TryGetOutputAllocatorVersionedMetadata(string tensorName, out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

tensorName string
metadata TensorRtVersionedInterfaceMetadata
diagnostic string

Returns

bool

TryGetTemporaryStorageAllocatorInterfaceInfo(out TensorRtInterfaceInfo)

Tries to get copied versioned-interface metadata for the temporary-storage allocator attached to this execution context. 尝试获取当前 execution context 已绑定 temporary-storage allocator 的 versioned-interface 元数据副本。

public bool TryGetTemporaryStorageAllocatorInterfaceInfo(out TensorRtInterfaceInfo interfaceInfo)

Parameters

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

Returns

bool

true when TensorRT reports a temporary-storage allocator and metadata was copied. / TensorRT 报告已绑定 temporary-storage allocator 且成功复制元数据时返回 true。

TryGetTemporaryStorageAllocatorInterfaceInfo(out TensorRtInterfaceInfo, out string)

Tries to get copied versioned-interface metadata for the temporary-storage allocator attached to this execution context. 尝试获取当前 execution context 已绑定 temporary-storage allocator 的 versioned-interface 元数据副本。

public bool TryGetTemporaryStorageAllocatorInterfaceInfo(out TensorRtInterfaceInfo interfaceInfo, out string diagnostic)

Parameters

interfaceInfo TensorRtInterfaceInfo

The copied interface metadata when the query succeeds. / 查询成功时复制出的 interface 元数据。

diagnostic string

A short diagnostic string describing success or the reason for failure. / 描述成功或失败原因的简短诊断。

Returns

bool

true when TensorRT reports a temporary-storage allocator and metadata was copied. / TensorRT 报告已绑定 temporary-storage allocator 且成功复制元数据时返回 true。

Remarks

This query maps the TensorRT IGpuAllocator::getInterfaceInfo surface through the execution context's borrowed temporary-storage allocator. It copies metadata only and does not expose or own the allocator pointer. 该查询通过 execution context 借出的 temporary-storage allocator 覆盖 TensorRT IGpuAllocator::getInterfaceInfo 表面; 只复制元数据,不暴露或拥有 allocator 指针。

TryGetTemporaryStorageAllocatorVersionedMetadata(out TensorRtVersionedInterfaceMetadata)

Tries to copy metadata from the temporary-storage allocator. 尝试复制 temporary-storage allocator 元数据。

public bool TryGetTemporaryStorageAllocatorVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata)

Parameters

metadata TensorRtVersionedInterfaceMetadata

Returns

bool

TryGetTemporaryStorageAllocatorVersionedMetadata(out TensorRtVersionedInterfaceMetadata, out string)

Tries to copy metadata from the temporary-storage allocator. 尝试复制 temporary-storage allocator 元数据。

public bool TryGetTemporaryStorageAllocatorVersionedMetadata(out TensorRtVersionedInterfaceMetadata metadata, out string diagnostic)

Parameters

metadata TensorRtVersionedInterfaceMetadata
diagnostic string

Returns

bool

UpdateDeviceMemorySizeForShapes()

Updates the Device Memory Size For Shapes value. 更新 Device Memory Size For Shapes 值。

public ulong UpdateDeviceMemorySizeForShapes()

Returns

ulong