Class TensorRtAttention
- Namespace
- JYPPX.TensorRtSharp
- Assembly
- JYPPX.TensorRtSharp.dll
Represents a TensorRT 11 attention object owned by a network. 表示由 network 持有生命周期的 TensorRT 11 attention 对象。
public sealed class TensorRtAttention : IDisposable- Inheritance
-
TensorRtAttention
- Implements
- Inherited Members
Properties
InputCount
Gets the number of attention inputs. 获取 attention 输入数量。
public int InputCount { get; }Property Value
Line
Gets the TensorRT API line used by this attention. 获取此 attention 使用的 TensorRT API 系列。
public TensorRtApiLine Line { get; }Property Value
Metadata
Gets or sets per-attention metadata emitted by detailed engine inspection. 获取或设置由详细 engine inspector 输出的 attention 元数据。
public string Metadata { get; set; }Property Value
Name
Gets or sets the TensorRT diagnostic name for the attention. 获取或设置此 attention 的 TensorRT 诊断名称。
public string Name { get; set; }Property Value
OutputCount
Gets the number of attention outputs. 获取 attention 输出数量。
public int OutputCount { get; }Property Value
Methods
Dispose()
Releases the managed bridge reference for this network-owned attention. 释放此 network-owned attention 的托管桥接引用。
public void Dispose()GetCausalKind()
Gets the causal mask alignment kind. 获取因果 mask 的对齐方向。
public TensorRtCausalMaskKind GetCausalKind()Returns
GetDecomposable()
Gets whether TensorRT may decompose attention into multiple kernels. 获取 TensorRT 是否可以将 attention 分解为多个 kernel。
public bool GetDecomposable()Returns
GetInput(int)
Gets an attention input tensor by index. 按索引获取 attention 输入张量。
public TensorRtTensor GetInput(int index)Parameters
indexint
Returns
GetKeyValueForm()
Gets the key/value tensor layout form. 获取 key/value 张量布局形式。
public TensorRtAttentionIoForm GetKeyValueForm()Returns
GetNormalizationOperation()
Gets the normalization operation used inside attention. 获取 attention 内部使用的归一化操作。
public TensorRtAttentionNormalizationOperation GetNormalizationOperation()Returns
GetNormalizationQuantizeToType()
Gets the data type used for quantized normalization output. 获取 normalization 量化输出使用的数据类型。
public TensorRtDataType GetNormalizationQuantizeToType()Returns
GetOutput(int)
Gets an attention output tensor by index. 按索引获取 attention 输出张量。
public TensorRtTensor GetOutput(int index)Parameters
indexint
Returns
GetQueryForm()
Gets the query tensor layout form. 获取 query 张量布局形式。
public TensorRtAttentionIoForm GetQueryForm()Returns
GetRankCount()
Gets the number of ranks for multi-device attention. 获取多设备 attention 的 rank 数量。
public int GetRankCount()Returns
SetCausalKind(TensorRtCausalMaskKind)
Sets the causal mask alignment kind. 设置因果 mask 的对齐方向。
public bool SetCausalKind(TensorRtCausalMaskKind causalKind)Parameters
causalKindTensorRtCausalMaskKind
Returns
SetDecomposable(bool)
Sets whether TensorRT may decompose attention into multiple kernels. 设置 TensorRT 是否可以将 attention 分解为多个 kernel。
public bool SetDecomposable(bool decomposable)Parameters
decomposablebool
Returns
SetInput(int, TensorRtTensor)
Replaces one attention input tensor. 替换一个 attention 输入张量。
public bool SetInput(int index, TensorRtTensor input)Parameters
indexintinputTensorRtTensor
Returns
SetKeyValueForm(TensorRtAttentionIoForm)
Sets the key/value tensor layout form. 设置 key/value 张量布局形式。
public bool SetKeyValueForm(TensorRtAttentionIoForm form)Parameters
Returns
SetKeyValueLengths(TensorRtTensor)
Sets the optional key/value lengths tensor. 设置可选的 key/value lengths 张量。
public bool SetKeyValueLengths(TensorRtTensor lengths)Parameters
lengthsTensorRtTensor
Returns
SetMask(TensorRtTensor)
Sets the optional attention mask tensor. 设置可选的 attention mask 张量。
public bool SetMask(TensorRtTensor mask)Parameters
maskTensorRtTensor
Returns
SetNormalizationOperation(TensorRtAttentionNormalizationOperation)
Sets the normalization operation used inside attention. 设置 attention 内部使用的归一化操作。
public bool SetNormalizationOperation(TensorRtAttentionNormalizationOperation operation)Parameters
operationTensorRtAttentionNormalizationOperation
Returns
SetNormalizationQuantizeScale(TensorRtTensor)
Sets the normalization quantization scale tensor. 设置 normalization 输出量化 scale 张量。
public bool SetNormalizationQuantizeScale(TensorRtTensor scale)Parameters
scaleTensorRtTensor
Returns
SetNormalizationQuantizeToType(TensorRtDataType)
Sets the data type used for quantized normalization output. 设置 normalization 量化输出使用的数据类型。
public bool SetNormalizationQuantizeToType(TensorRtDataType dataType)Parameters
dataTypeTensorRtDataType
Returns
SetQueryForm(TensorRtAttentionIoForm)
Sets the query tensor layout form. 设置 query 张量布局形式。
public bool SetQueryForm(TensorRtAttentionIoForm form)Parameters
Returns
SetQueryLengths(TensorRtTensor)
Sets the optional query lengths tensor. 设置可选的 query lengths 张量。
public bool SetQueryLengths(TensorRtTensor lengths)Parameters
lengthsTensorRtTensor
Returns
SetRankCount(int)
Sets the number of ranks for multi-device attention. 设置多设备 attention 的 rank 数量。
public bool SetRankCount(int rankCount)Parameters
rankCountint
Returns
TryGetKeyValueLengths(out TensorRtTensor?)
Tries to get the optional key/value lengths tensor. 尝试获取可选的 key/value lengths 张量。
public bool TryGetKeyValueLengths(out TensorRtTensor? lengths)Parameters
lengthsTensorRtTensor
Returns
TryGetMask(out TensorRtTensor?)
Tries to get the optional attention mask tensor. 尝试获取可选的 attention mask 张量。
public bool TryGetMask(out TensorRtTensor? mask)Parameters
maskTensorRtTensor
Returns
TryGetNormalizationQuantizeScale(out TensorRtTensor?)
Tries to get the normalization quantization scale tensor. 尝试获取 normalization 输出量化 scale 张量。
public bool TryGetNormalizationQuantizeScale(out TensorRtTensor? scale)Parameters
scaleTensorRtTensor
Returns
TryGetQueryLengths(out TensorRtTensor?)
Tries to get the optional query lengths tensor. 尝试获取可选的 query lengths 张量。
public bool TryGetQueryLengths(out TensorRtTensor? lengths)Parameters
lengthsTensorRtTensor