Class TensorRtExecutionContext

Namespace

JYPPX.TensorRtSharp

Assembly

JYPPX.TensorRtSharp.dll

Represents a managed TensorRT Tensor Rt Execution Context wrapper. 表示托管 TensorRT Tensor Rt Execution Context 包装器。

public sealed class TensorRtExecutionContext : IDisposable

Inheritance

object

TensorRtExecutionContext

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

AllInputDimensionsSpecified

Gets the All Input Dimensions Specified value. 获取 All Input Dimensions Specified 值。

public bool AllInputDimensionsSpecified { get; }

Property Value

bool

AllInputShapesSpecified

Gets or sets the All Input Shapes Specified value. 获取或设置 All Input Shapes Specified 值。

public bool AllInputShapesSpecified { get; }

Property Value

bool

DebugSync

Gets or sets the Debug Sync value. 获取或设置 Debug Sync 值。

public bool DebugSync { get; set; }

Property Value

bool

DeviceMemorySizeInBytes

Gets or sets the Device Memory Size In Bytes value. 获取或设置 Device Memory Size In Bytes 值。

public ulong DeviceMemorySizeInBytes { get; }

Property Value

ulong

EngineIOTensorCount

Gets the engine I/O tensor count reached through this execution context. 获取通过当前 execution context 反查到的 engine I/O tensor 数量。

public int EngineIOTensorCount { get; }

Property Value

int

EngineLayerCount

Gets the engine layer count reached through this execution context. 获取通过当前 execution context 反查到的 engine layer 数量。

public int EngineLayerCount { get; }

Property Value

int

EngineName

Gets the engine name reached through this execution context. 获取通过当前 execution context 反查到的 engine 名称。

public string EngineName { get; }

Property Value

string

EngineOptimizationProfileCount

Gets the engine optimization profile count reached through this execution context. 获取通过当前 execution context 反查到的 engine optimization profile 数量。

public int EngineOptimizationProfileCount { get; }

Property Value

int

EnqueueEmitsProfile

Gets or sets the Enqueue Emits Profile value. 获取或设置 Enqueue Emits Profile 值。

public bool EnqueueEmitsProfile { get; set; }

Property Value

bool

HasDebugListener

Gets whether this TensorRT 11 execution context has a debug listener attached. 获取当前 TensorRT 11 execution context 是否绑定了 debug listener。

public bool HasDebugListener { get; }

Property Value

bool

HasErrorRecorder

Gets whether a native TensorRT error recorder is attached to this execution context. 获取当前 execution context 是否绑定了 TensorRT 原生 error recorder。

public bool HasErrorRecorder { get; }

Property Value

bool

HasProfiler

Gets whether this TensorRT 11 execution context has a profiler attached. 获取当前 TensorRT 11 execution context 是否绑定了 profiler。

public bool HasProfiler { get; }

Property Value

bool

HasRuntimeConfig

Gets whether this TensorRT 11 execution context has an associated runtime config object. 获取当前 TensorRT 11 execution context 是否有关联的 runtime config 对象。

public bool HasRuntimeConfig { get; }

Property Value

bool

HasTemporaryStorageAllocator

Gets whether this execution context has a temporary-storage allocator attached. 获取此 execution context 是否绑定了 temporary-storage allocator。

public bool HasTemporaryStorageAllocator { get; }

Property Value

bool

InputConsumedEventAddressValue

Gets the native address value of the CUDA event TensorRT uses to signal input-consumption completion. 获取 TensorRT 用于通知输入消费完成的 CUDA event 原生地址诊断值。

public ulong InputConsumedEventAddressValue { get; }

Property Value

ulong

Remarks

The value is diagnostic only. It is intentionally exposed as an integer instead of a user-owned pointer. 该值仅用于诊断；它有意以整数形式暴露，而不是用户可拥有或解引用的指针。

IsInputConsumedEventSet

Gets whether an input-consumed CUDA event is currently set on this TensorRT 11 execution context. 获取此 TensorRT 11 execution context 当前是否设置了 input-consumed CUDA event。

public bool IsInputConsumedEventSet { get; }

Property Value

bool

Line

Gets or sets the Line value. 获取或设置 Line 值。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

Name

Gets or sets the Name value. 获取或设置 Name 值。

public string Name { get; set; }

Property Value

string

OptimizationProfileIndex

Gets or sets the Optimization Profile Index value. 获取或设置 Optimization Profile Index 值。

public int OptimizationProfileIndex { get; }

Property Value

int

PersistentCacheLimitInBytes

Gets or sets the Persistent Cache Limit In Bytes value. 获取或设置 Persistent Cache Limit In Bytes 值。

public ulong PersistentCacheLimitInBytes { get; set; }

Property Value

ulong

RuntimeConfigAllocationStrategy

Gets the allocation strategy from the runtime config attached to this TensorRT 11 execution context. 获取附加到当前 TensorRT 11 execution context 的 runtime config allocation strategy。

public TensorRtExecutionContextAllocationStrategy RuntimeConfigAllocationStrategy { get; }

Property Value

TensorRtExecutionContextAllocationStrategy

Exceptions

TensorRtException

Thrown when TensorRT does not expose a runtime config for this context. 当 TensorRT 未为该 context 暴露 runtime config 时抛出。

SupportsTensorDebugState

Gets whether the active TensorRT line supports per-tensor debug state on execution contexts. 获取当前 TensorRT 版本线是否支持 execution context 上的逐 tensor debug state。

public bool SupportsTensorDebugState { get; }

Property Value

bool

Methods

ClearAuxStreams()

Clears user-provided auxiliary streams so TensorRT may use its default auxiliary-stream behavior. 清除用户提供的 auxiliary stream，让 TensorRT 回到默认 auxiliary-stream 行为。

public void ClearAuxStreams()

ClearDebugListener()

Clears the debug listener attached to this TensorRT 11 execution context. 清除绑定到当前 TensorRT 11 execution context 的 debug listener。

public bool ClearDebugListener()

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearDeviceMemory()

Clears the externally supplied device-memory block for this execution context. 清除当前 execution context 外部传入的 device memory 块。

public void ClearDeviceMemory()

Remarks

Use this only after queued inference work has completed. The managed wrapper does not expose the raw native pointer. 请仅在已提交的推理任务完成后使用。托管封装不会向普通用户暴露原生裸指针。

ClearErrorRecorder()

Clears the native TensorRT error recorder attached to this execution context. 清除当前 execution context 上绑定的 TensorRT 原生 error recorder。

public void ClearErrorRecorder()

ClearInputConsumedEvent()

Clears the CUDA event used by TensorRT to signal input-consumption completion. 清除 TensorRT 用于通知输入消费完成的 CUDA event。

public bool ClearInputConsumedEvent()

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearInputTensorAddress(string)

Clears the CUDA memory address currently bound to a named input tensor. 清除当前绑定到指定输入 tensor 的 CUDA 内存地址。

public bool ClearInputTensorAddress(string tensorName)

Parameters

tensorName string

The input tensor name. 输入 tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearOutputAllocator(string)

Clears the output allocator attached to a named output tensor. 清除绑定到指定输出 tensor 的 output allocator。

public bool ClearOutputAllocator(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearOutputTensorAddress(string)

Clears the CUDA memory address currently bound to a named output tensor. 清除当前绑定到指定输出 tensor 的 CUDA 内存地址。

public bool ClearOutputTensorAddress(string tensorName)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

ClearProfiler()

Clears the profiler attached to this TensorRT 11 execution context. 清除绑定到当前 TensorRT 11 execution context 的 profiler。

public void ClearProfiler()

ClearTemporaryStorageAllocator()

Clears the temporary-storage allocator attached to this execution context. 清除绑定到当前 execution context 的 temporary-storage allocator。

public bool ClearTemporaryStorageAllocator()

Returns

bool

true when TensorRT accepts the clear operation. / TensorRT 接受清理操作时返回 true。

ClearTensorAddress(string)

Clears the CUDA memory address currently bound to a named input or output tensor. 清除当前绑定到指定输入或输出 tensor 的 CUDA 内存地址。

public bool ClearTensorAddress(string tensorName)

Parameters

tensorName string

The tensor name. Tensor 名称。

Returns

bool

true when TensorRT accepted the clear operation. 当 TensorRT 接受清理操作时返回 true。

Remarks

This is a safe high-level wrapper over TensorRT 11 setTensorAddress(name, nullptr). 这是 TensorRT 11 setTensorAddress(name, nullptr) 的安全高层封装。

Dispose()

Releases the native TensorRT resources held by this object. 释放此对象持有的 native TensorRT 资源。

public void Dispose()

EnqueueAsync(CudaStream)

Enqueues the TensorRT execution work. 将 TensorRT execution work 加入队列。

public void EnqueueAsync(CudaStream stream)

Parameters

stream CudaStream

GetDeploymentSnapshot(TensorRtEngine)

Builds a TensorRT 11 deployment snapshot for this execution context. 为当前 execution context 构建 TensorRT 11 部署快照。

public TensorRtExecutionContextDeploymentSnapshot GetDeploymentSnapshot(TensorRtEngine engine)

Parameters

engine TensorRtEngine

The engine used to enumerate tensor metadata. 用于枚举 tensor 元数据的 engine。

Returns

TensorRtExecutionContextDeploymentSnapshot

A deployment snapshot with context readiness, address state, and runtime metadata. 包含 context 就绪状态、地址状态和运行时元数据的部署快照。

GetMaxOutputSize(string)

Queries TensorRT's maximum output buffer size estimate for a named output tensor in the current context. 查询当前 execution context 中指定输出 tensor 的最大输出缓冲区大小估计值。

public long GetMaxOutputSize(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

Returns

long

The maximum size in bytes, or a negative value when TensorRT cannot determine the value for the current shape state. 最大字节数；当 TensorRT 在当前 shape 状态下无法确定该值时，可能返回负值。

GetNvtxVerbosity()

Gets the NVTX verbosity currently used by this TensorRT 11 execution context. 获取当前 TensorRT 11 execution context 使用的 NVTX 详细程度。

public TensorRtProfilingVerbosity GetNvtxVerbosity()

Returns

TensorRtProfilingVerbosity

GetOutputTensorAddressValue(string)

Gets the native address value currently bound to a named TensorRT output tensor for diagnostics. 获取当前绑定到指定 TensorRT 输出 tensor 的原生地址数值，仅用于诊断。

public ulong GetOutputTensorAddressValue(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

ulong

The address value reported by TensorRT, or 0 when no address is bound. / TensorRT 报告的地址数值；未绑定时返回 0。

GetReadiness(TensorRtEngine, bool)

Creates a deployment readiness snapshot for this execution context and engine. 为当前 execution context 与 engine 创建部署就绪状态快照。

public TensorRtExecutionContextReadiness GetReadiness(TensorRtEngine engine, bool runShapeInference = false)

Parameters

engine TensorRtEngine

The engine that created this context. 创建该 context 的 engine。

runShapeInference bool

True to call TensorRT inferShapes before reporting missing-shape count. 为 true 时先调用 TensorRT inferShapes 并报告缺失 shape 数量。

Returns

TensorRtExecutionContextReadiness

A readiness snapshot suitable for diagnostics before enqueue. 可用于 enqueue 前诊断的就绪状态快照。

GetTensorAddressValue(string)

Gets the native address value currently bound to a named TensorRT tensor for diagnostics. 获取当前绑定到指定 TensorRT tensor 的原生地址数值，仅用于诊断。

public ulong GetTensorAddressValue(string tensorName)

Parameters

tensorName string

The input or output tensor name. / 输入或输出 tensor 名称。

Returns

ulong

The address value reported by TensorRT, or 0 when no address is bound. TensorRT 报告的地址数值；未绑定地址时返回 0。

Remarks

This value is intentionally exposed as an integer diagnostic value instead of a user-owned pointer. 该值特意以整数诊断值形式暴露，而不是用户可拥有或解引用的指针。

GetTensorDebugState(string)

Gets the TensorRT debug state for a named tensor on supported execution contexts. 获取受支持 execution context 上指定 tensor 的 debug state。

public bool GetTensorDebugState(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

Returns

bool

true when debug state is enabled. / 启用 debug state 时返回 true。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

GetTensorShape(string)

Gets the Tensor Shape value. 获取 Tensor Shape 值。

public TensorRtDims GetTensorShape(string tensorName)

Parameters

tensorName string

Returns

TensorRtDims

GetTensorShape64(string)

Gets a TensorRT 11 execution-context tensor shape with 64-bit dimension extents. 获取 TensorRT 11 execution context 中张量的形状，并保留 64 位维度 extent。

public TensorRtDims64 GetTensorShape64(string tensorName)

Parameters

tensorName string

The context tensor name. Context 张量名称。

Returns

TensorRtDims64

The runtime tensor shape reported by TensorRT. TensorRT 报告的运行时张量形状。

GetTensorShapeDimensionExtent64(string, int)

Gets one TensorRT 11 runtime tensor shape extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 运行时张量形状的单个 extent。

public long GetTensorShapeDimensionExtent64(string tensorName, int dimensionIndex)

Parameters

tensorName string

The context tensor name. Context 张量名称。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The runtime shape extent reported by TensorRT. TensorRT 报告的运行时 shape extent。

GetTensorStrideDimensionExtent64(string, int)

Gets one TensorRT 11 runtime stride extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 运行时张量 stride 的单个 extent。

public long GetTensorStrideDimensionExtent64(string tensorName, int dimensionIndex)

Parameters

tensorName string

The context tensor name. Context 张量名称。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The runtime stride extent reported by TensorRT. TensorRT 报告的运行时 stride extent。

GetTensorStrides(string)

Gets the Tensor Strides value. 获取 Tensor Strides 值。

public TensorRtDims GetTensorStrides(string tensorName)

Parameters

tensorName string

Returns

TensorRtDims

GetTensorStrides64(string)

Gets TensorRT 11 runtime tensor strides with 64-bit values. 获取 TensorRT 11 运行时张量 strides，并保留 64 位值。

public TensorRtDims64 GetTensorStrides64(string tensorName)

Parameters

tensorName string

The context tensor name. Context 张量名称。

Returns

TensorRtDims64

The runtime tensor strides reported by TensorRT. TensorRT 报告的运行时 tensor strides。

GetUnfusedTensorsDebugState()

Gets debug state for TensorRT 11 unfused debug tensors. 获取 TensorRT 11 未融合 debug tensor 的 debug state。

public bool GetUnfusedTensorsDebugState()

Returns

bool

HasOutputAllocator(string)

Gets whether a named output tensor uses a TensorRT output allocator. 获取指定输出 tensor 是否使用 TensorRT output allocator。

public bool HasOutputAllocator(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true if TensorRT reports an output allocator. / TensorRT 报告存在 output allocator 时返回 true。

InferShapes()

Runs TensorRT shape inference for the current execution context. 对当前 execution context 执行 TensorRT shape inference。

public int InferShapes()

Returns

int

The number of tensors whose shapes are still insufficiently specified. 仍未完全指定 shape 的 tensor 数量。

IsOutputTensorAddressSet(string)

Gets whether a named output tensor has an explicit device address bound. 获取指定输出 tensor 是否已绑定显式设备地址。

public bool IsOutputTensorAddressSet(string tensorName)

Parameters

tensorName string

The output tensor name. / 输出 tensor 名称。

Returns

bool

true if TensorRT reports an output address. / TensorRT 报告已有输出地址时返回 true。

IsTensorAddressBound(string)

Checks whether Tensor Address Bound is true. 检查 Tensor Address Bound 是否为 true。

public bool IsTensorAddressBound(string tensorName)

Parameters

tensorName string

Returns

bool

ReportToProfiler()

Reports accumulated execution data to the profiler attached to the context, when one exists. 将当前执行上下文已累计的执行数据上报给关联的 profiler（如果存在）。

public bool ReportToProfiler()

Returns

bool

true when TensorRT reports that profiler data was emitted; otherwise false. 当 TensorRT 确认已上报 profiler 数据时返回 true，否则返回 false。

SetAllTensorsDebugState(bool)

Enables or disables TensorRT debug state for all tensors on supported execution contexts. 在受支持的 execution context 上启用或禁用所有 tensor 的 debug state。

public void SetAllTensorsDebugState(bool enabled)

Parameters

enabled bool

Whether debug state should be enabled. / 是否启用 debug state。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

SetAuxStreams(params CudaStream[])

Sets auxiliary CUDA streams used by TensorRT 11 during inference. 设置 TensorRT 11 推理阶段使用的辅助 CUDA stream。

public void SetAuxStreams(params CudaStream[] streams)

Parameters

streams CudaStream[]

Auxiliary CUDA streams. 辅助 CUDA stream。

SetAuxStreams(IReadOnlyList<CudaStream>)

Sets auxiliary CUDA streams used by TensorRT 11 during inference. 设置 TensorRT 11 推理阶段使用的辅助 CUDA stream。

public void SetAuxStreams(IReadOnlyList<CudaStream> streams)

Parameters

streams IReadOnlyList<CudaStream>

Auxiliary streams that must remain alive while the context may use them. 在 context 可能使用期间必须保持存活的辅助 stream。

Remarks

Passing an empty collection clears user-provided auxiliary streams. TensorRT may still use its default internal behavior. 传入空集合会清除用户提供的辅助 stream；TensorRT 仍可能使用自身默认内部行为。

SetBindingDimensions(int, TensorRtDims)

Sets the Binding Dimensions value. 设置 Binding Dimensions 值。

public void SetBindingDimensions(int bindingIndex, TensorRtDims dims)

Parameters

bindingIndex int

dims TensorRtDims

SetDeviceMemory(CudaMemory)

Sets the Device Memory value. 设置 Device Memory 值。

public void SetDeviceMemory(CudaMemory memory)

Parameters

memory CudaMemory

SetInputConsumedEvent(CudaEvent)

Sets the Input Consumed Event value. 设置 Input Consumed Event 值。

public void SetInputConsumedEvent(CudaEvent cudaEvent)

Parameters

cudaEvent CudaEvent

SetInputShape(string, TensorRtDims)

Sets the Input Shape value. 设置 Input Shape 值。

public void SetInputShape(string tensorName, TensorRtDims dims)

Parameters

tensorName string

dims TensorRtDims

SetInputTensorAddress(string, CudaMemory)

Binds a CUDA allocation to a named input tensor. 将 CUDA 设备内存分配绑定到指定输入 tensor。

public void SetInputTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string

The input tensor name. 输入 tensor 名称。

memory CudaMemory

The CUDA allocation used as the input buffer. 作为输入缓冲区的 CUDA 设备内存。

SetNvtxVerbosity(TensorRtProfilingVerbosity)

Sets the NVTX verbosity used by this TensorRT 11 execution context. 设置当前 TensorRT 11 execution context 使用的 NVTX 详细程度。

public bool SetNvtxVerbosity(TensorRtProfilingVerbosity verbosity)

Parameters

verbosity TensorRtProfilingVerbosity

The desired NVTX verbosity. / 期望的 NVTX 详细程度。

Returns

bool

true when TensorRT accepts the value. / TensorRT 接受该值时返回 true。

SetOptimizationProfileAsync(int, CudaStream)

Sets the Optimization Profile Async value. 设置 Optimization Profile Async 值。

public void SetOptimizationProfileAsync(int profileIndex, CudaStream stream)

Parameters

profileIndex int

stream CudaStream

SetOutputTensorAddress(string, CudaMemory)

Binds a CUDA allocation to a named output tensor. 将 CUDA 设备内存分配绑定到指定输出 tensor。

public void SetOutputTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string

The output tensor name. 输出 tensor 名称。

memory CudaMemory

The CUDA allocation used as the output buffer. 作为输出缓冲区的 CUDA 设备内存。

SetTensorAddress(string, CudaMemory)

Sets the Tensor Address value. 设置 Tensor Address 值。

public void SetTensorAddress(string tensorName, CudaMemory memory)

Parameters

tensorName string

memory CudaMemory

SetTensorDebugState(string, bool)

Enables or disables TensorRT debug state for a named tensor on supported execution contexts. 在受支持的 execution context 上启用或禁用指定 tensor 的 debug state。

public void SetTensorDebugState(string tensorName, bool enabled)

Parameters

tensorName string

The TensorRT tensor name. / TensorRT tensor 名称。

enabled bool

Whether debug state should be enabled. / 是否启用 debug state。

Exceptions

NotSupportedException

Thrown when the active TensorRT line does not expose tensor debug-state APIs. 当前 TensorRT 版本线未暴露 tensor debug-state API 时抛出。

SetUnfusedTensorsDebugState(bool)

Enables or disables debug state for TensorRT 11 unfused debug tensors. 启用或禁用 TensorRT 11 未融合 debug tensor 的 debug state。

public bool SetUnfusedTensorsDebugState(bool enabled)

Parameters

enabled bool

Whether unfused tensor debug state should be enabled. / 是否启用未融合 tensor debug state。

Returns

bool

true when TensorRT accepts the setting. / TensorRT 接受该设置时返回 true。

UpdateDeviceMemorySizeForShapes()

Updates the Device Memory Size For Shapes value. 更新 Device Memory Size For Shapes 值。

public ulong UpdateDeviceMemorySizeForShapes()

Returns

ulong