Table of Contents

Class TensorRtNetworkDefinition

Namespace
JYPPX.TensorRtSharp
Assembly
JYPPX.TensorRtSharp.dll

Represents a managed TensorRT Tensor Rt Network Definition wrapper. 表示托管 TensorRT Tensor Rt Network Definition 包装器。

public sealed class TensorRtNetworkDefinition : IDisposable
Inheritance
TensorRtNetworkDefinition
Implements
Inherited Members

Properties

Flags

Gets the Flags value. 获取 Flags 值。

public TensorRtNetworkDefinitionCreationFlags Flags { get; }

Property Value

TensorRtNetworkDefinitionCreationFlags

HasErrorRecorder

Gets whether a native TensorRT error recorder is attached to this network definition. 获取当前 network definition 是否绑定了 TensorRT 原生 error recorder。

public bool HasErrorRecorder { get; }

Property Value

bool

HasImplicitBatchDimension

Gets whether this network uses TensorRT implicit batch dimensions. 获取当前网络是否使用 TensorRT 隐式 batch 维度。

public bool HasImplicitBatchDimension { get; }

Property Value

bool

InputCount

Gets the Input Count value. 获取 Input Count 值。

public int InputCount { get; }

Property Value

int

LayerCount

Gets the Layer Count value. 获取 Layer Count 值。

public int LayerCount { get; }

Property Value

int

Line

Gets or sets the Line value. 获取或设置 Line 值。

public TensorRtApiLine Line { get; }

Property Value

TensorRtApiLine

Name

Gets or sets the Name value. 获取或设置 Name 值。

public string Name { get; set; }

Property Value

string

OutputCount

Gets the Output Count value. 获取 Output Count 值。

public int OutputCount { get; }

Property Value

int

Methods

AddActivation(TensorRtTensor, TensorRtActivationType)

Adds a Activation layer or object. 添加 Activation 层或对象。

public TensorRtLayer AddActivation(TensorRtTensor input, TensorRtActivationType activationType)

Parameters

input TensorRtTensor
activationType TensorRtActivationType

Returns

TensorRtLayer

AddAssertion(TensorRtTensor, string)

Adds a TensorRT assertion layer. 添加 TensorRT assertion 断言层。

public TensorRtLayer AddAssertion(TensorRtTensor condition, string message)

Parameters

condition TensorRtTensor
message string

Returns

TensorRtLayer

Remarks

The condition tensor must evaluate to a TensorRT boolean condition. condition 张量必须表示 TensorRT 布尔断言条件。

AddAttentionV2(TensorRtTensor, TensorRtTensor, TensorRtTensor, TensorRtAttentionNormalizationOperation, TensorRtCausalMaskKind)

Adds a TensorRT 11 attention object to the network. 向 network 添加一个 TensorRT 11 attention 对象。

public TensorRtAttention AddAttentionV2(TensorRtTensor query, TensorRtTensor key, TensorRtTensor value, TensorRtAttentionNormalizationOperation normalizationOperation = TensorRtAttentionNormalizationOperation.Softmax, TensorRtCausalMaskKind causalKind = TensorRtCausalMaskKind.None)

Parameters

query TensorRtTensor

Query tensor. / Query 张量。

key TensorRtTensor

Key tensor. / Key 张量。

value TensorRtTensor

Value tensor. / Value 张量。

normalizationOperation TensorRtAttentionNormalizationOperation

Normalization operation inside attention. / attention 内部归一化操作。

causalKind TensorRtCausalMaskKind

Causal mask alignment kind. / 因果 mask 对齐方向。

Returns

TensorRtAttention

The network-owned TensorRT attention wrapper. / network 持有生命周期的 TensorRT attention 包装对象。

AddCast(TensorRtTensor, TensorRtDataType)

Adds a TensorRT 11 cast layer that converts a tensor to another data type. 添加 TensorRT 11 Cast 层，用于把张量转换为另一个数据类型。

public TensorRtLayer AddCast(TensorRtTensor input, TensorRtDataType toType)

Parameters

input TensorRtTensor

Input tensor to cast. / 需要转换的输入张量。

toType TensorRtDataType

Destination TensorRT data type. / 目标 TensorRT 数据类型。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddConcatenation(params TensorRtTensor[])

Adds a Concatenation layer or object. 添加 Concatenation 层或对象。

public TensorRtLayer AddConcatenation(params TensorRtTensor[] inputs)

Parameters

inputs TensorRtTensor[]

Returns

TensorRtLayer

AddConstant(TensorRtDims, TensorRtWeights)

Adds a Constant layer or object. 添加 Constant 层或对象。

public TensorRtLayer AddConstant(TensorRtDims shape, TensorRtWeights weights)

Parameters

shape TensorRtDims
weights TensorRtWeights

Returns

TensorRtLayer

AddConvolution(TensorRtTensor, int, TensorRtDims, TensorRtWeights, TensorRtWeights?)

Adds a Convolution layer or object. 添加 Convolution 层或对象。

public TensorRtLayer AddConvolution(TensorRtTensor input, int outputMaps, TensorRtDims kernelSize, TensorRtWeights kernelWeights, TensorRtWeights? biasWeights = null)

Parameters

input TensorRtTensor
outputMaps int
kernelSize TensorRtDims
kernelWeights TensorRtWeights
biasWeights TensorRtWeights

Returns

TensorRtLayer

AddCumulative(TensorRtTensor, TensorRtTensor, TensorRtCumulativeOperation, bool, bool)

Adds a TensorRT cumulative operation layer. 添加 TensorRT cumulative 累计运算层。

public TensorRtLayer AddCumulative(TensorRtTensor input, TensorRtTensor axis, TensorRtCumulativeOperation operation, bool exclusive = false, bool reverse = false)

Parameters

input TensorRtTensor
axis TensorRtTensor
operation TensorRtCumulativeOperation
exclusive bool
reverse bool

Returns

TensorRtLayer

Remarks

The axis tensor must be a TensorRT build-time constant 0D shape tensor; a 1D constant with one element is rejected by TensorRT 11. axis 张量必须是 TensorRT 构建期常量 0D shape tensor；只有一个元素的一维常量会被 TensorRT 11 拒绝。

AddDeconvolution(TensorRtTensor, int, TensorRtDims, TensorRtWeights, TensorRtWeights?)

Adds a Deconvolution layer or object. 添加 Deconvolution 层或对象。

public TensorRtLayer AddDeconvolution(TensorRtTensor input, int outputMaps, TensorRtDims kernelSize, TensorRtWeights kernelWeights, TensorRtWeights? biasWeights = null)

Parameters

input TensorRtTensor
outputMaps int
kernelSize TensorRtDims
kernelWeights TensorRtWeights
biasWeights TensorRtWeights

Returns

TensorRtLayer

AddDequantize(TensorRtTensor, TensorRtTensor)

Adds a Dequantize layer or object. 添加 Dequantize 层或对象。

public TensorRtLayer AddDequantize(TensorRtTensor input, TensorRtTensor scale)

Parameters

input TensorRtTensor
scale TensorRtTensor

Returns

TensorRtLayer

AddDequantizeV2(TensorRtTensor, TensorRtTensor, TensorRtDataType)

Adds a TensorRT 10 or TensorRT 11 dequantize layer and selects the requested output type at creation time. 添加 TensorRT 10 或 TensorRT 11 Dequantize 层，并在创建时指定输出数据类型。

public TensorRtLayer AddDequantizeV2(TensorRtTensor input, TensorRtTensor scale, TensorRtDataType outputType)

Parameters

input TensorRtTensor

Quantized input tensor. / 量化输入张量。

scale TensorRtTensor

Scale tensor used by dequantization. / 反量化使用的 scale 张量。

outputType TensorRtDataType

Requested dequantized output type. / 反量化输出数据类型。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddDistCollective(TensorRtTensor, TensorRtCollectiveOperation, TensorRtDistributedReduceOperation, long, IReadOnlyList<long>?)

Adds a TensorRT 11 DistCollective layer. This layer may require a TensorRT multi-device capability set at build or runtime. 添加 TensorRT 11 DistCollective 层；该层在构建或运行时可能需要 TensorRT 多设备能力集。

public TensorRtLayer AddDistCollective(TensorRtTensor input, TensorRtCollectiveOperation collectiveOperation, TensorRtDistributedReduceOperation reduceOperation = TensorRtDistributedReduceOperation.None, long root = -1, IReadOnlyList<long>? groups = null)

Parameters

input TensorRtTensor

Input tensor. / 输入张量。

collectiveOperation TensorRtCollectiveOperation

Collective operation to perform. / 要执行的集合通信操作。

reduceOperation TensorRtDistributedReduceOperation

Reduction operation, or None when unused. / 归约操作；不使用归约时传 None。

root long

Root rank for root-based collectives, or -1 when unused. / root 型集合通信的根 rank，不使用时传 -1。

groups IReadOnlyList<long>

Optional ordered participating rank IDs. / 可选的参与 rank ID 有序列表。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddDynamicQuantize(TensorRtTensor, int, int, TensorRtDataType, TensorRtDataType)

Adds a TensorRT 10 dynamic-quantize layer using axis and block-size metadata. 使用 axis 和 block-size 元数据添加 TensorRT 10 dynamic-quantize 层。

public TensorRtLayer AddDynamicQuantize(TensorRtTensor input, int axis, int blockSize, TensorRtDataType outputType, TensorRtDataType scaleType)

Parameters

input TensorRtTensor
axis int
blockSize int
outputType TensorRtDataType
scaleType TensorRtDataType

Returns

TensorRtLayer

Remarks

TensorRT 10 uses axis and blockSize. TensorRT 11 uses AddDynamicQuantizeV2(TensorRtTensor, TensorRtDims, TensorRtDataType, TensorRtDataType) with a block shape. TensorRT 10 使用 axis 和 blockSize；TensorRT 11 请使用带 block shape 的 AddDynamicQuantizeV2(TensorRtTensor, TensorRtDims, TensorRtDataType, TensorRtDataType)。

AddDynamicQuantizeV2(TensorRtTensor, TensorRtDims, TensorRtDataType, TensorRtDataType)

Adds a TensorRT dynamic-quantize V2 layer. 添加 TensorRT dynamic-quantize V2 动态量化层。

public TensorRtLayer AddDynamicQuantizeV2(TensorRtTensor input, TensorRtDims blockShape, TensorRtDataType outputType, TensorRtDataType scaleType)

Parameters

input TensorRtTensor
blockShape TensorRtDims
outputType TensorRtDataType
scaleType TensorRtDataType

Returns

TensorRtLayer

Remarks

The block shape and data types are forwarded to TensorRT 11. TensorRT validates whether the combination is legal. block shape 和数据类型会传递给 TensorRT 11；组合是否合法由 TensorRT 原生侧校验。

AddEinsum(string, params TensorRtTensor[])

Adds a TensorRT Einsum layer from an equation and one or more input tensors. 根据公式和一个或多个输入张量添加 TensorRT Einsum 层。

public TensorRtLayer AddEinsum(string equation, params TensorRtTensor[] inputs)

Parameters

equation string

Einsum equation accepted by TensorRT. / TensorRT 可接受的 Einsum 公式。

inputs TensorRtTensor[]

Input tensors. / 输入张量列表。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddElementWise(TensorRtTensor, TensorRtTensor, TensorRtElementWiseOperation)

Adds a Element Wise layer or object. 添加 Element Wise 层或对象。

public TensorRtLayer AddElementWise(TensorRtTensor left, TensorRtTensor right, TensorRtElementWiseOperation operation)

Parameters

left TensorRtTensor
right TensorRtTensor
operation TensorRtElementWiseOperation

Returns

TensorRtLayer

AddFill(TensorRtDims, TensorRtFillOperation)

Adds a Fill layer or object. 添加 Fill 层或对象。

public TensorRtLayer AddFill(TensorRtDims dimensions, TensorRtFillOperation operation)

Parameters

dimensions TensorRtDims
operation TensorRtFillOperation

Returns

TensorRtLayer

AddFillV2(TensorRtDims, TensorRtFillOperation, TensorRtDataType)

Adds a TensorRT 10 or TensorRT 11 FillV2 layer and selects the output tensor type at creation time. 添加 TensorRT 10 或 TensorRT 11 FillV2 层，并在创建时指定输出张量类型。

public TensorRtLayer AddFillV2(TensorRtDims dimensions, TensorRtFillOperation operation, TensorRtDataType outputType)

Parameters

dimensions TensorRtDims

Static output dimensions when input 0 is absent. / 当第 0 输入不存在时使用的静态输出维度。

operation TensorRtFillOperation

Fill operation. / Fill 操作。

outputType TensorRtDataType

Requested output tensor data type. / 请求的输出张量数据类型。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddGather(TensorRtTensor, TensorRtTensor, int)

Adds a Gather layer or object. 添加 Gather 层或对象。

public TensorRtLayer AddGather(TensorRtTensor data, TensorRtTensor indices, int axis)

Parameters

data TensorRtTensor
indices TensorRtTensor
axis int

Returns

TensorRtLayer

AddGatherV2(TensorRtTensor, TensorRtTensor, TensorRtGatherMode)

Adds a TensorRT 10 or TensorRT 11 GatherV2 layer with the requested gather mode. 添加 TensorRT 10 或 TensorRT 11 GatherV2 层，并指定 gather 模式。

public TensorRtLayer AddGatherV2(TensorRtTensor data, TensorRtTensor indices, TensorRtGatherMode mode)

Parameters

data TensorRtTensor

Tensor to gather from. / 被 gather 的数据张量。

indices TensorRtTensor

Tensor containing gather indices. / 包含 gather 索引的张量。

mode TensorRtGatherMode

Gather semantic mode. / Gather 语义模式。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddGridSample(TensorRtTensor, TensorRtTensor)

Adds a TensorRT grid-sample layer. 添加 TensorRT grid-sample 层。

public TensorRtLayer AddGridSample(TensorRtTensor input, TensorRtTensor grid)

Parameters

input TensorRtTensor
grid TensorRtTensor

Returns

TensorRtLayer

Remarks

Configure interpolation and sample modes on the returned layer. 可在返回的层对象上继续配置插值模式和越界采样模式。

AddIdentity(TensorRtTensor)

Adds a Identity layer or object. 添加 Identity 层或对象。

public TensorRtLayer AddIdentity(TensorRtTensor input)

Parameters

input TensorRtTensor

Returns

TensorRtLayer

AddIfConditional()

Adds a TensorRT 11 if-conditional object to the network. 向 network 添加 TensorRT 11 if-conditional 对象。

public TensorRtIfConditional AddIfConditional()

Returns

TensorRtIfConditional

A network-owned conditional wrapper. / 由 network 持有生命周期的 conditional 包装对象。

Remarks

This API creates the conditional container. Configure condition, branch inputs, and branch outputs through the returned object. 此 API 创建 conditional 容器；请通过返回对象继续配置 condition、分支输入和分支输出。

AddInput(string, TensorRtDataType, TensorRtDims)

Adds a Input layer or object. 添加 Input 层或对象。

public TensorRtTensor AddInput(string name, TensorRtDataType dataType, TensorRtDims shape)

Parameters

name string
dataType TensorRtDataType
shape TensorRtDims

Returns

TensorRtTensor

AddKvCacheUpdate(TensorRtTensor, TensorRtTensor, TensorRtTensor, TensorRtKvCacheMode)

Adds a TensorRT 11 KV-cache update layer. 添加 TensorRT 11 KV-cache update 层。

public TensorRtLayer AddKvCacheUpdate(TensorRtTensor cache, TensorRtTensor update, TensorRtTensor writeIndices, TensorRtKvCacheMode cacheMode = TensorRtKvCacheMode.Linear)

Parameters

cache TensorRtTensor
update TensorRtTensor
writeIndices TensorRtTensor
cacheMode TensorRtKvCacheMode

Returns

TensorRtLayer

Remarks

The returned layer can be configured with update-form and optional update-length metadata. 返回层可继续配置 update form 以及可选 update lengths 元数据。

AddLoop()

Adds a TensorRT 11 loop object to the network. 向 network 添加 TensorRT 11 loop 对象。

public TensorRtLoop AddLoop()

Returns

TensorRtLoop

A network-owned loop wrapper. / 由 network 持有生命周期的 loop 包装对象。

Remarks

This API creates the loop container. Add trip-limit, iterator, recurrence, and output boundary layers through the returned object. 此 API 创建 loop 容器；请通过返回对象继续添加 trip-limit、iterator、recurrence 和 output 边界层。

AddLrn(TensorRtTensor, int, float, float, float)

Adds a Lrn layer or object. 添加 Lrn 层或对象。

public TensorRtLayer AddLrn(TensorRtTensor input, int windowSize, float alpha, float beta, float k)

Parameters

input TensorRtTensor
windowSize int
alpha float
beta float
k float

Returns

TensorRtLayer

AddMatrixMultiply(TensorRtTensor, TensorRtMatrixOperation, TensorRtTensor, TensorRtMatrixOperation)

Adds a Matrix Multiply layer or object. 添加 Matrix Multiply 层或对象。

public TensorRtLayer AddMatrixMultiply(TensorRtTensor left, TensorRtMatrixOperation leftOperation, TensorRtTensor right, TensorRtMatrixOperation rightOperation)

Parameters

left TensorRtTensor
leftOperation TensorRtMatrixOperation
right TensorRtTensor
rightOperation TensorRtMatrixOperation

Returns

TensorRtLayer

AddMoE(TensorRtTensor, TensorRtTensor, TensorRtTensor)

Adds a TensorRT 11 MoE layer. 添加 TensorRT 11 MoE 层。

public TensorRtLayer AddMoE(TensorRtTensor hiddenStates, TensorRtTensor selectedExpertsForTokens, TensorRtTensor scoresForSelectedExperts)

Parameters

hiddenStates TensorRtTensor
selectedExpertsForTokens TensorRtTensor
scoresForSelectedExperts TensorRtTensor

Returns

TensorRtLayer

Remarks

This wraps TensorRT's lightweight MoE layer creation only; expert weights and quantization metadata are configured on the returned layer. 此方法仅封装 TensorRT 轻量级 MoE 层创建；专家权重和量化元数据在返回层上继续配置。

AddNms(TensorRtTensor, TensorRtTensor, TensorRtTensor, TensorRtDataType)

Adds a TensorRT NMS layer for deployment-side post-processing. 添加 TensorRT NMS 层，用于部署端目标检测后处理。

public TensorRtLayer AddNms(TensorRtTensor boxes, TensorRtTensor scores, TensorRtTensor maxOutputBoxesPerClass, TensorRtDataType indicesType = TensorRtDataType.Int32)

Parameters

boxes TensorRtTensor

Boxes tensor. / 边界框张量。

scores TensorRtTensor

Scores tensor. / 置信度分数张量。

maxOutputBoxesPerClass TensorRtTensor

Scalar tensor for max boxes per class. / 每个类别最大输出框数量的标量张量。

indicesType TensorRtDataType

Indices output type, usually Int32 or Int64. / 索引输出类型，通常为 Int32 或 Int64。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddNonZero(TensorRtTensor, TensorRtDataType)

Adds a TensorRT 10 or TensorRT 11 NonZero layer. 添加 TensorRT 10 或 TensorRT 11 NonZero 层。

public TensorRtLayer AddNonZero(TensorRtTensor input, TensorRtDataType indicesType = TensorRtDataType.Int32)

Parameters

input TensorRtTensor

Input tensor to scan for non-zero elements. / 用于查找非零元素的输入张量。

indicesType TensorRtDataType

Indices output type. TensorRT 10 only supports Int32 here. / 索引输出类型；TensorRT 10 仅支持 Int32。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddNormalizationV2(TensorRtTensor, TensorRtTensor, TensorRtTensor, uint)

Adds a TensorRT normalization V2 layer. 添加 TensorRT normalization V2 层。

public TensorRtLayer AddNormalizationV2(TensorRtTensor input, TensorRtTensor scale, TensorRtTensor bias, uint axes)

Parameters

input TensorRtTensor
scale TensorRtTensor
bias TensorRtTensor
axes uint

Returns

TensorRtLayer

Remarks

The scale and bias tensors are passed directly to TensorRT; their shape rules are validated by TensorRT. scale 和 bias 张量直接传入 TensorRT；其 shape 规则由 TensorRT 原生侧校验。

AddOneHot(TensorRtTensor, TensorRtTensor, TensorRtTensor, int)

Adds a TensorRT one-hot layer. 添加 TensorRT one-hot 层。

public TensorRtLayer AddOneHot(TensorRtTensor indices, TensorRtTensor values, TensorRtTensor depth, int axis)

Parameters

indices TensorRtTensor
values TensorRtTensor
depth TensorRtTensor
axis int

Returns

TensorRtLayer

Remarks

This API is currently exposed through the TensorRT 11 adapter. 当前此 API 通过 TensorRT 11 适配器暴露。

AddPadding(TensorRtTensor, TensorRtDims, TensorRtDims)

Adds a Padding layer or object. 添加 Padding 层或对象。

public TensorRtLayer AddPadding(TensorRtTensor input, TensorRtDims prePadding, TensorRtDims postPadding)

Parameters

input TensorRtTensor
prePadding TensorRtDims
postPadding TensorRtDims

Returns

TensorRtLayer

AddParametricReLU(TensorRtTensor, TensorRtTensor)

Adds a TensorRT parametric ReLU layer. 添加 TensorRT ParametricReLU 层。

public TensorRtLayer AddParametricReLU(TensorRtTensor input, TensorRtTensor slope)

Parameters

input TensorRtTensor

Input tensor. / 输入张量。

slope TensorRtTensor

Slope tensor broadcastable to the input tensor. / 可广播到输入张量的 slope 张量。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddPooling(TensorRtTensor, TensorRtPoolingType, TensorRtDims)

Adds a Pooling layer or object. 添加 Pooling 层或对象。

public TensorRtLayer AddPooling(TensorRtTensor input, TensorRtPoolingType poolingType, TensorRtDims windowSize)

Parameters

input TensorRtTensor
poolingType TensorRtPoolingType
windowSize TensorRtDims

Returns

TensorRtLayer

AddQuantize(TensorRtTensor, TensorRtTensor)

Adds a Quantize layer or object. 添加 Quantize 层或对象。

public TensorRtLayer AddQuantize(TensorRtTensor input, TensorRtTensor scale)

Parameters

input TensorRtTensor
scale TensorRtTensor

Returns

TensorRtLayer

AddQuantizeV2(TensorRtTensor, TensorRtTensor, TensorRtDataType)

Adds a TensorRT 10 quantize layer and selects the requested output type at creation time. 添加 TensorRT 10 Quantize 层，并在创建时指定输出数据类型。

public TensorRtLayer AddQuantizeV2(TensorRtTensor input, TensorRtTensor scale, TensorRtDataType outputType)

Parameters

input TensorRtTensor

Floating-point input tensor. / 浮点输入张量。

scale TensorRtTensor

Scale tensor used by quantization. / 量化使用的 scale 张量。

outputType TensorRtDataType

Requested quantized output type. / 量化输出数据类型。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddRaggedSoftMax(TensorRtTensor, TensorRtTensor)

Adds a TensorRT ragged softmax layer for variable-length sequences. 添加 TensorRT RaggedSoftMax 层，用于可变长度序列的 softmax。

public TensorRtLayer AddRaggedSoftMax(TensorRtTensor input, TensorRtTensor bounds)

Parameters

input TensorRtTensor

Input tensor. / 输入张量。

bounds TensorRtTensor

Bounds tensor that describes valid sequence lengths. / 描述有效序列长度的边界张量。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddReduce(TensorRtTensor, TensorRtReduceOperation, uint, bool)

Adds a Reduce layer or object. 添加 Reduce 层或对象。

public TensorRtLayer AddReduce(TensorRtTensor input, TensorRtReduceOperation operation, uint axes, bool keepDimensions)

Parameters

input TensorRtTensor
operation TensorRtReduceOperation
axes uint
keepDimensions bool

Returns

TensorRtLayer

AddResize(TensorRtTensor, TensorRtDims, TensorRtResizeMode)

Adds a Resize layer or object. 添加 Resize 层或对象。

public TensorRtLayer AddResize(TensorRtTensor input, TensorRtDims outputDimensions, TensorRtResizeMode resizeMode = TensorRtResizeMode.Nearest)

Parameters

input TensorRtTensor
outputDimensions TensorRtDims
resizeMode TensorRtResizeMode

Returns

TensorRtLayer

AddReverseSequence(TensorRtTensor, TensorRtTensor)

Adds a TensorRT ReverseSequence layer on TensorRT 8, 10, or 11 adapters. 在 TensorRT 8、10 或 11 适配线上添加 ReverseSequence 层。

public TensorRtLayer AddReverseSequence(TensorRtTensor input, TensorRtTensor sequenceLengths)

Parameters

input TensorRtTensor

Input tensor with rank greater than or equal to 2. / rank 大于等于 2 的输入张量。

sequenceLengths TensorRtTensor

1D tensor containing lengths to reverse. / 包含反转长度的一维张量。

Returns

TensorRtLayer

The created network-owned layer. / 返回由网络持有生命周期的层对象。

AddRotaryEmbedding(TensorRtTensor, TensorRtTensor, TensorRtTensor, bool, int)

Adds a TensorRT 11 rotary-embedding layer. 添加 TensorRT 11 rotary-embedding 层。

public TensorRtLayer AddRotaryEmbedding(TensorRtTensor input, TensorRtTensor cosCache, TensorRtTensor sinCache, bool interleaved, int rotaryEmbeddingDimension)

Parameters

input TensorRtTensor
cosCache TensorRtTensor
sinCache TensorRtTensor
interleaved bool
rotaryEmbeddingDimension int

Returns

TensorRtLayer

Remarks

TensorRT validates the RoPE cache tensor shapes and whether rotaryEmbeddingDimension is legal for the input. RoPE cache 张量形状以及 rotaryEmbeddingDimension 是否合法由 TensorRT 原生侧校验。

AddScale(TensorRtTensor, TensorRtScaleMode, TensorRtWeights?, TensorRtWeights?, TensorRtWeights?, int)

Adds a Scale layer or object. 添加 Scale 层或对象。

public TensorRtLayer AddScale(TensorRtTensor input, TensorRtScaleMode mode, TensorRtWeights? shift = null, TensorRtWeights? scale = null, TensorRtWeights? power = null, int channelAxis = 0)

Parameters

input TensorRtTensor
mode TensorRtScaleMode
shift TensorRtWeights
scale TensorRtWeights
power TensorRtWeights
channelAxis int

Returns

TensorRtLayer

AddScatter(TensorRtTensor, TensorRtTensor, TensorRtTensor, TensorRtScatterMode)

Adds a TensorRT scatter layer. 添加 TensorRT scatter 层。

public TensorRtLayer AddScatter(TensorRtTensor data, TensorRtTensor indices, TensorRtTensor updates, TensorRtScatterMode mode)

Parameters

data TensorRtTensor
indices TensorRtTensor
updates TensorRtTensor
mode TensorRtScatterMode

Returns

TensorRtLayer

Remarks

This API is currently exposed through the TensorRT 11 adapter. The returned layer is owned by the network. 当前此 API 通过 TensorRT 11 适配器暴露。返回的层由 network 持有生命周期。

AddSelect(TensorRtTensor, TensorRtTensor, TensorRtTensor)

Adds a Select layer or object. 添加 Select 层或对象。

public TensorRtLayer AddSelect(TensorRtTensor condition, TensorRtTensor thenInput, TensorRtTensor elseInput)

Parameters

condition TensorRtTensor
thenInput TensorRtTensor
elseInput TensorRtTensor

Returns

TensorRtLayer

AddShape(TensorRtTensor)

Adds a Shape layer or object. 添加 Shape 层或对象。

public TensorRtLayer AddShape(TensorRtTensor input)

Parameters

input TensorRtTensor

Returns

TensorRtLayer

AddShuffle(TensorRtTensor, TensorRtDims)

Adds a Shuffle layer or object. 添加 Shuffle 层或对象。

public TensorRtLayer AddShuffle(TensorRtTensor input, TensorRtDims reshapeDimensions)

Parameters

input TensorRtTensor
reshapeDimensions TensorRtDims

Returns

TensorRtLayer

AddSlice(TensorRtTensor, TensorRtDims, TensorRtDims, TensorRtDims)

Adds a Slice layer or object. 添加 Slice 层或对象。

public TensorRtLayer AddSlice(TensorRtTensor input, TensorRtDims start, TensorRtDims size, TensorRtDims stride)

Parameters

input TensorRtTensor
start TensorRtDims
size TensorRtDims
stride TensorRtDims

Returns

TensorRtLayer

AddSoftMax(TensorRtTensor, uint)

Adds a Soft Max layer or object. 添加 Soft Max 层或对象。

public TensorRtLayer AddSoftMax(TensorRtTensor input, uint axes)

Parameters

input TensorRtTensor
axes uint

Returns

TensorRtLayer

AddSqueeze(TensorRtTensor, TensorRtTensor)

Adds a TensorRT squeeze layer. 添加 TensorRT squeeze 层。

public TensorRtLayer AddSqueeze(TensorRtTensor input, TensorRtTensor axes)

Parameters

input TensorRtTensor
axes TensorRtTensor

Returns

TensorRtLayer

Remarks

The axes tensor controls which dimensions are removed. axes 张量控制需要移除的维度。

AddTopK(TensorRtTensor, TensorRtTopKOperation, int, uint)

Adds a Top K layer or object. 添加 Top K 层或对象。

public TensorRtLayer AddTopK(TensorRtTensor input, TensorRtTopKOperation operation, int k, uint axes)

Parameters

input TensorRtTensor
operation TensorRtTopKOperation
k int
axes uint

Returns

TensorRtLayer

AddTopKV2(TensorRtTensor, TensorRtTopKOperation, int, uint, TensorRtDataType)

Adds the TensorRT 11 five-argument TopK layer variant with explicit output-indices type. 添加 TensorRT 11 五参数 TopK layer 变体，并显式指定输出 indices 类型。

public TensorRtLayer AddTopKV2(TensorRtTensor input, TensorRtTopKOperation operation, int k, uint axes, TensorRtDataType indicesType)

Parameters

input TensorRtTensor

The input tensor. / 输入 tensor。

operation TensorRtTopKOperation

The TopK operation. / TopK 操作类型。

k int

The static K value. / 静态 K 值。

axes uint

The reduction axes bitmask. / reduce axes 位掩码。

indicesType TensorRtDataType

The output indices tensor type. Only Int32 and Int64 are valid. / 输出 indices tensor 类型，仅支持 Int32 和 Int64。

Returns

TensorRtLayer

The created TopK layer. / 创建出的 TopK layer。

AddUnary(TensorRtTensor, TensorRtUnaryOperation)

Adds a Unary layer or object. 添加 Unary 层或对象。

public TensorRtLayer AddUnary(TensorRtTensor input, TensorRtUnaryOperation operation)

Parameters

input TensorRtTensor
operation TensorRtUnaryOperation

Returns

TensorRtLayer

AddUnsqueeze(TensorRtTensor, TensorRtTensor)

Adds a TensorRT unsqueeze layer. 添加 TensorRT unsqueeze 层。

public TensorRtLayer AddUnsqueeze(TensorRtTensor input, TensorRtTensor axes)

Parameters

input TensorRtTensor
axes TensorRtTensor

Returns

TensorRtLayer

Remarks

The axes tensor controls where new dimensions are inserted. axes 张量控制新增维度插入的位置。

AreWeightsMarkedRefittable(string)

Queries whether a named weights object is marked as refittable. 查询指定名称的权重是否已经标记为可 refit。

public bool AreWeightsMarkedRefittable(string weightsName)

Parameters

weightsName string

TensorRT weights name. / TensorRT 权重名称。

Returns

bool

True when the named weights are currently marked as refittable. / 当指定权重当前已标记为可 refit 时返回 true。

ClearErrorRecorder()

Clears the native TensorRT error recorder attached to this network definition. 清除当前 network definition 上绑定的 TensorRT 原生 error recorder。

public void ClearErrorRecorder()

Dispose()

Releases the native TensorRT resources held by this object. 释放此对象持有的 native TensorRT 资源。

public void Dispose()

GetFlag(TensorRtNetworkDefinitionCreationFlags)

Gets the Flag value. 获取 Flag 值。

public bool GetFlag(TensorRtNetworkDefinitionCreationFlags flag)

Parameters

flag TensorRtNetworkDefinitionCreationFlags

Returns

bool

GetInput(int)

Gets the Input value. 获取 Input 值。

public TensorRtTensor GetInput(int index)

Parameters

index int

Returns

TensorRtTensor

GetInputDimensionExtent64(int, int)

Gets one TensorRT 11 network input tensor dimension extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 网络输入张量的单个维度 extent。

public long GetInputDimensionExtent64(int index, int dimensionIndex)

Parameters

index int

The zero-based input index. 从零开始的输入索引。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The input dimension extent reported by TensorRT. TensorRT 报告的输入维度 extent。

GetInputShape64(int)

Gets a TensorRT 11 network input tensor shape with 64-bit dimension extents. 获取 TensorRT 11 网络输入张量形状，并保留 64 位维度 extent。

public TensorRtDims64 GetInputShape64(int index)

Parameters

index int

The zero-based input index. 从零开始的输入索引。

Returns

TensorRtDims64

The input tensor shape reported by TensorRT. TensorRT 报告的输入张量形状。

GetLayer(int)

Gets the Layer value. 获取 Layer 值。

public TensorRtLayer GetLayer(int index)

Parameters

index int

Returns

TensorRtLayer

GetOutput(int)

Gets the Output value. 获取 Output 值。

public TensorRtTensor GetOutput(int index)

Parameters

index int

Returns

TensorRtTensor

GetOutputDimensionExtent64(int, int)

Gets one TensorRT 11 network output tensor dimension extent as a 64-bit value. 以 64 位整数获取 TensorRT 11 网络输出张量的单个维度 extent。

public long GetOutputDimensionExtent64(int index, int dimensionIndex)

Parameters

index int

The zero-based output index. 从零开始的输出索引。

dimensionIndex int

The zero-based dimension index. 从零开始的维度索引。

Returns

long

The output dimension extent reported by TensorRT. TensorRT 报告的输出维度 extent。

GetOutputShape64(int)

Gets a TensorRT 11 network output tensor shape with 64-bit dimension extents. 获取 TensorRT 11 网络输出张量形状，并保留 64 位维度 extent。

public TensorRtDims64 GetOutputShape64(int index)

Parameters

index int

The zero-based output index. 从零开始的输出索引。

Returns

TensorRtDims64

The output tensor shape reported by TensorRT. TensorRT 报告的输出张量形状。

IsDebugTensor(TensorRtTensor)

Gets whether a tensor is currently marked as a TensorRT debug tensor. 获取指定 tensor 当前是否已被标记为 TensorRT debug tensor。

public bool IsDebugTensor(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The network tensor to query. / 要查询的 network tensor。

Returns

bool

MarkDebugTensor(TensorRtTensor)

Marks a tensor as a TensorRT debug tensor. 将指定 tensor 标记为 TensorRT debug tensor。

public bool MarkDebugTensor(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The network tensor to mark. / 要标记的 network tensor。

Returns

bool

true when TensorRT accepts or already has the mark. / TensorRT 接受标记或该 tensor 已被标记时返回 true。

Remarks

Debug tensors can be emitted by runtime debug listeners. Tensor names should be stable before enabling this. debug tensor 可由运行时 debug listener 输出；启用前应保证 tensor 名称稳定。

MarkOutput(TensorRtTensor)

Marks the Output value. 标记 Output 值。

public void MarkOutput(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

MarkOutputForShapes(TensorRtTensor)

Marks an INT32 tensor as a TensorRT shape output. 将 INT32 tensor 标记为 TensorRT shape output。

public bool MarkOutputForShapes(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The tensor whose value should be exposed through shape output APIs. / 需要通过 shape output API 暴露值的 tensor。

Returns

bool

true when TensorRT accepts the shape-output mark. / TensorRT 接受 shape-output 标记时返回 true。

Remarks

TensorRT requires shape outputs to have build-time constant dimensions and no more than one dimension. TensorRT 要求 shape output 的维度可在构建期确定，且维度数量不超过一维。

MarkUnfusedTensorsAsDebugTensors()

Asks TensorRT 11 to mark unfused tensors as debug tensors. 请求 TensorRT 11 将未融合 tensor 标记为 debug tensor。

public bool MarkUnfusedTensorsAsDebugTensors()

Returns

bool

true when TensorRT accepts the request. / TensorRT 接受请求时返回 true。

Remarks

This preserves optimizer fusion behavior better than marking individual tensors, but TensorRT may use internal tensor names. 该方式比逐个标记 tensor 更能保留优化器融合行为，但 TensorRT 可能使用内部 tensor 名称。

MarkWeightsRefittable(string)

Marks a named weights object as refittable when TensorRT individual refit is enabled. 在启用 TensorRT 单独 refit 能力时，把指定名称的权重标记为可 refit。

public bool MarkWeightsRefittable(string weightsName)

Parameters

weightsName string

TensorRT weights name. / TensorRT 权重名称。

Returns

bool

True if TensorRT accepted the mark operation; otherwise false. / 如果 TensorRT 接受标记操作则返回 true，否则返回 false。

RemoveTensor(TensorRtTensor)

Removes a tensor from the TensorRT 11 network definition. 从 TensorRT 11 network definition 中移除一个 tensor。

public void RemoveTensor(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The tensor to remove. / 要移除的 tensor。

Remarks

TensorRT only permits removing tensors that are not used as layer inputs or outputs. The bridge forwards TensorRT's validation result. TensorRT 只允许移除没有被 layer 输入或输出使用的 tensor；桥接层会直接转发 TensorRT 的校验结果。

SetWeightsName(TensorRtLayer, string)

Assigns a TensorRT name to a weights object before engine build. 在 engine 构建前为 weights 对象分配 TensorRT 名称。

public bool SetWeightsName(TensorRtLayer constantLayer, string weightsName)

Parameters

constantLayer TensorRtLayer

A constant layer created by this network. 由当前 network 创建的 constant layer。

weightsName string

TensorRT weights name. TensorRT 权重名称。

Returns

bool

True if TensorRT accepted the name assignment; otherwise false. 如果 TensorRT 接受名称分配则返回 true，否则返回 false。

UnmarkDebugTensor(TensorRtTensor)

Removes the TensorRT debug mark from a tensor. 移除指定 tensor 的 TensorRT debug 标记。

public bool UnmarkDebugTensor(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The network tensor to unmark. / 要取消标记的 network tensor。

Returns

bool

true when TensorRT accepts or already has no mark. / TensorRT 接受取消或该 tensor 本就未标记时返回 true。

UnmarkOutput(TensorRtTensor)

Unmarks the Output value. 取消标记 Output 值。

public void UnmarkOutput(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

UnmarkOutputForShapes(TensorRtTensor)

Removes a TensorRT shape-output mark from a tensor. 从指定 tensor 移除 TensorRT shape-output 标记。

public bool UnmarkOutputForShapes(TensorRtTensor tensor)

Parameters

tensor TensorRtTensor

The tensor to unmark. / 要取消标记的 tensor。

Returns

bool

true when TensorRT accepts the request. / TensorRT 接受请求时返回 true。

UnmarkUnfusedTensorsAsDebugTensors()

Removes the TensorRT 11 unfused-debug-tensor network mark. 移除 TensorRT 11 未融合 tensor debug 标记。

public bool UnmarkUnfusedTensorsAsDebugTensors()

Returns

bool

true when TensorRT accepts the request. / TensorRT 接受请求时返回 true。

UnmarkWeightsRefittable(string)

Removes the refittable mark from a named weights object. 从指定名称的权重上移除可 refit 标记。

public bool UnmarkWeightsRefittable(string weightsName)

Parameters

weightsName string

TensorRT weights name. / TensorRT 权重名称。

Returns

bool

True if TensorRT accepted the unmark operation; otherwise false. / 如果 TensorRT 接受取消标记操作则返回 true，否则返回 false。

Edit this page