Class TensorRtRefitter

Namespace

JYPPX.TensorRtSharp

Assembly

JYPPX.TensorRtSharp.dll

Provides a safe managed wrapper over TensorRT refitter operations. 提供 TensorRT refitter 操作的安全托管封装。

public sealed class TensorRtRefitter : IDisposable

Inheritance

object

TensorRtRefitter

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

AllNamedWeightCount

Gets the number of all named weights TensorRT reports as refittable. 获取 TensorRT 报告的全部可 refit named weights 数量。适用于 TensorRT 8/10，返回的是拷贝后的名称集合计数。

public int AllNamedWeightCount { get; }

Property Value

int

AllRefittableWeightCount

Gets the number of weights TensorRT reports as refittable for the engine. 获取 TensorRT 报告的当前 engine 可 refit 权重数量。

public int AllRefittableWeightCount { get; }

Property Value

int

DynamicRangeTensorCount

Gets the number of refittable dynamic-range tensor names reported by TensorRT. 获取 TensorRT 报告的可 refit dynamic range tensor 名称数量。适用于 TensorRT 8/10；TensorRT 10.1 起该能力由显式量化取代。

public int DynamicRangeTensorCount { get; }

Property Value

int

HasErrorRecorder

Gets whether this refitter currently has a TensorRT error recorder attached. 获取当前 refitter 是否附加了 TensorRT error recorder；不会暴露 recorder 指针。适用于 TensorRT 8/10/11。

public bool HasErrorRecorder { get; }

Property Value

bool

HasLogger

Gets whether this refitter has a TensorRT logger associated with it. 获取当前 refitter 是否关联了 TensorRT logger；只返回布尔值，不跨 ABI 暴露借用的 logger 指针。适用于 TensorRT 8/10。

public bool HasLogger { get; }

Property Value

bool

Line

Gets the TensorRT API line that owns this refitter. 获取拥有当前 refitter 的 TensorRT API 版本线。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

MaxThreads

Gets or sets the maximum number of worker threads TensorRT may use while refitting. 获取或设置 TensorRT 在 refit 时可使用的最大工作线程数。适用于 TensorRT 8/10/11。

public int MaxThreads { get; set; }

Property Value

int

MissingNamedWeightCount

Gets the number of missing named weights before the refit can complete. 获取 refit 完成前仍缺失的 named weights 数量。适用于 TensorRT 8/10，返回的是拷贝后的名称集合计数。

public int MissingNamedWeightCount { get; }

Property Value

int

MissingWeightCount

Gets the number of required weights still missing before refit can complete. 获取执行 refit 前仍缺失的必要权重数量。

public int MissingWeightCount { get; }

Property Value

int

WeightsValidation

Gets or sets whether TensorRT validates refit weights before applying them. 获取或设置 TensorRT 在应用 refit weights 前是否执行权重校验。适用于 TensorRT 10/11。

public bool WeightsValidation { get; set; }

Property Value

bool

Methods

ClearErrorRecorder()

Clears the native error recorder pointer if one was attached externally. 清除外部附加的原生 error recorder 指针；不会接管 recorder 生命周期。适用于 TensorRT 8/10/11。

public void ClearErrorRecorder()

Dispose()

Releases the native refitter handle. 释放原生 refitter 句柄。

public void Dispose()

GetAllEntries()

Gets all layer/role pairs that TensorRT reports as refittable. 获取 TensorRT 报告的全部可 refit layer/role 组合。

public IReadOnlyList<TensorRtRefitEntry> GetAllEntries()

Returns

IReadOnlyList<TensorRtRefitEntry>

All refittable entries. 全部可 refit 条目。

GetAllNamedWeights()

Gets all named weights TensorRT reports as refittable. 获取 TensorRT 报告的全部可 refit named weights；名称已复制到托管内存。适用于 TensorRT 8/10。

public IReadOnlyList<string> GetAllNamedWeights()

Returns

IReadOnlyList<string>

Copied refittable named weights. 已复制的可 refit named weights 名称集合。

GetDynamicRangeMaximum(string)

Gets the maximum dynamic range value for a refittable tensor. 获取可 refit tensor 的 dynamic range 最大值。适用于 TensorRT 8/10。

public float GetDynamicRangeMaximum(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. TensorRT tensor 名称。

Returns

float

The dynamic range maximum. dynamic range 最大值。

GetDynamicRangeMinimum(string)

Gets the minimum dynamic range value for a refittable tensor. 获取可 refit tensor 的 dynamic range 最小值。适用于 TensorRT 8/10。

public float GetDynamicRangeMinimum(string tensorName)

Parameters

tensorName string

The TensorRT tensor name. TensorRT tensor 名称。

Returns

float

The dynamic range minimum. dynamic range 最小值。

GetDynamicRangeTensorNames()

Gets all tensor names that TensorRT reports as having refittable dynamic ranges. 获取 TensorRT 报告的全部可 refit dynamic range tensor 名称；名称已复制到托管内存。适用于 TensorRT 8/10。

public IReadOnlyList<string> GetDynamicRangeTensorNames()

Returns

IReadOnlyList<string>

Copied tensor names. 已复制的 tensor 名称集合。

GetMissingEntries()

Gets the layer/role pairs still missing before TensorRT can complete refit. 获取 TensorRT 完成 refit 前仍缺失的 layer/role 组合。

public IReadOnlyList<TensorRtRefitEntry> GetMissingEntries()

Returns

IReadOnlyList<TensorRtRefitEntry>

The missing refit entries. 缺失的 refit 条目。

GetMissingNamedWeights()

Gets missing named weights before TensorRT can complete refit. 获取 TensorRT 完成 refit 前仍缺失的 named weights；名称已复制到托管内存。适用于 TensorRT 8/10。

public IReadOnlyList<string> GetMissingNamedWeights()

Returns

IReadOnlyList<string>

Copied missing named weights. 已复制的缺失 named weights 名称集合。

GetNamedWeightsInfo(string)

Gets metadata for the current named weights supplied to the refitter. 获取当前已提供给 refitter 的 named weights 元数据；不会暴露原生 weights 指针。适用于 TensorRT 10/11。

public TensorRtWeightsInfo GetNamedWeightsInfo(string weightsName)

Parameters

weightsName string

The refittable weights name. 可 refit 权重名称。

Returns

TensorRtWeightsInfo

Weights metadata without native pointer exposure. 不暴露原生指针的 weights 元数据。

GetWeightsLocation(string)

Gets the TensorRT memory location for a named refit weights entry. 获取指定 refit weights 的 TensorRT 内存位置。适用于 TensorRT 10/11。

public TensorRtTensorLocation GetWeightsLocation(string weightsName)

Parameters

weightsName string

The refittable weights name. 可 refit 权重名称。

Returns

TensorRtTensorLocation

The TensorRT tensor location. TensorRT tensor 的内存位置。

GetWeightsPrototypeInfo(string)

Gets TensorRT's expected prototype metadata for a named refit weights entry. 获取 TensorRT 对指定 refit weights 的期望 prototype 元数据；不会暴露原生 weights 指针。适用于 TensorRT 10/11。

public TensorRtWeightsInfo GetWeightsPrototypeInfo(string weightsName)

Parameters

weightsName string

The refittable weights name. 可 refit 权重名称。

Returns

TensorRtWeightsInfo

Expected weights metadata without native pointer exposure. 不暴露原生指针的期望 weights 元数据。

RefitCudaEngine()

Applies the currently configured weights to the CUDA engine. 将当前已配置的权重应用到 CUDA engine。

public bool RefitCudaEngine()

Returns

bool

true when TensorRT reports the refit operation succeeded; otherwise false. 当 TensorRT 报告 refit 操作成功时返回 true，否则返回 false。

RefitCudaEngineAsync(CudaStream)

Applies configured refit weights asynchronously on the supplied CUDA stream. 在指定 CUDA stream 上异步应用已配置的 refit weights。适用于 TensorRT 10/11。

public bool RefitCudaEngineAsync(CudaStream stream)

Parameters

stream CudaStream

The CUDA stream used for the async refit. 用于异步 refit 的 CUDA stream。

Returns

bool

true when TensorRT reports the refit was accepted. TensorRT 接受 refit 时返回 true。

SetDynamicRange(string, float, float)

Updates the dynamic range for a refittable tensor. 更新可 refit tensor 的 dynamic range。适用于 TensorRT 8/10；不会持有或返回原生字符串指针。

public bool SetDynamicRange(string tensorName, float minimum, float maximum)

Parameters

tensorName string

The TensorRT tensor name. TensorRT tensor 名称。

minimum float

The minimum dynamic range value. dynamic range 最小值。

maximum float

The maximum dynamic range value. dynamic range 最大值。

Returns

bool

true if TensorRT accepted the range. TensorRT 接受该范围时返回 true。

SetNamedWeights(string, TensorRtRefitWeightsBuffer)

Sets named weights using a pinned refit weights buffer. 使用 pinned refit weights buffer 设置 named weights；调用方必须保证 buffer 在 refit 完成前保持有效。适用于 TensorRT 8/10。

public bool SetNamedWeights(string weightsName, TensorRtRefitWeightsBuffer weights)

Parameters

weightsName string

The refittable weights name. 可 refit 权重名称。

weights TensorRtRefitWeightsBuffer

The pinned refit weights buffer. pinned refit 权重缓冲区。

Returns

bool

true if TensorRT accepted the named weights. TensorRT 接受该 named weights 时返回 true。

SetWeights(TensorRtRefitEntry, TensorRtRefitWeightsBuffer)

Sets weights for a refit entry using a pinned refit weights buffer. 使用 pinned refit 权重缓冲区为指定 refit 条目设置权重。

public bool SetWeights(TensorRtRefitEntry entry, TensorRtRefitWeightsBuffer weights)

Parameters

entry TensorRtRefitEntry

The layer/role entry to update. 要更新的 layer/role 条目。

weights TensorRtRefitWeightsBuffer

The pinned refit weights buffer. pinned refit 权重缓冲区。

Returns

bool

true if TensorRT accepted the weights; otherwise false. 如果 TensorRT 接受该权重则返回 true，否则返回 false。

SetWeights(string, TensorRtWeightsRole, TensorRtRefitWeightsBuffer)

Sets weights for a layer/role pair using a pinned refit weights buffer. 使用 pinned refit 权重缓冲区为指定 layer/role 设置权重。

public bool SetWeights(string layerName, TensorRtWeightsRole role, TensorRtRefitWeightsBuffer weights)

Parameters

layerName string

The TensorRT layer name. TensorRT 层名称。

role TensorRtWeightsRole

The TensorRT weight role. TensorRT 权重角色。

weights TensorRtRefitWeightsBuffer

The pinned refit weights buffer. pinned refit 权重缓冲区。

Returns

bool

true if TensorRT accepted the weights; otherwise false. 如果 TensorRT 接受该权重则返回 true，否则返回 false。

UnsetNamedWeights(string)

Unsets a named refit weights entry. 取消设置指定名称的 refit weights。适用于 TensorRT 10/11。

public bool UnsetNamedWeights(string weightsName)

Parameters

weightsName string

The refittable weights name. 可 refit 权重名称。

Returns

bool

true when TensorRT accepted the unset operation. TensorRT 接受取消设置操作时返回 true。