CANN/ge C++融合Pass开发指南

C++ Fusion Pass Development Guide

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge

This guide is for developers who want to write GE fusion passes in C++. It is recommended to first read the language-independent mechanism description: Fusion Pattern Pass Mechanism.

C++ passes are delivered as dynamic libraries. Developers implement a pass class, register it with GE, and compile it into a.so. When GE compiles a model, it loads the.soand executes the pass at a specified stage.

If you are still exploring patterns, it is recommended to use the Python Fusion Pass Development Guide for quick validation; migrate to C++ once the pattern is stable.

1. Which Pass to Choose

This guide coversPatternFusionPassfirst, thenDecomposePass.

2. Minimal Example: Delete Add(x, 0)

Goal:

x ----\ Add ---- out ==> x ---- out 0 ----/

The core C++ pass code consists of four parts:

1. InheritPatternFusionPass.
2. Patterns()defines the structure to match.
3. MeetRequirements()checks if the constant is 0.
4. Replacement()returns the replacement structure.

#include <cmath> #include <cstdint> #include <iostream> #include "es_all_ops.h" #include "ge/fusion/pass/pattern_fusion_pass.h" using namespace ge; using namespace ge::fusion; class AddZeroPass : public PatternFusionPass { protected: std::vector<PatternUniqPtr> Patterns() override { std::vector<PatternUniqPtr> patterns; auto builder = es::EsGraphBuilder("add_zero_pattern"); auto x = builder.CreateInput(0); auto zero = es::Const(builder); auto add = es::Add(x, zero); auto graph = builder.BuildAndReset({add}); patterns.emplace_back(std::make_unique<Pattern>(std::move(*graph))); return patterns; } bool MeetRequirements(const std::unique_ptr<MatchResult> &match_result) override { for (const auto &node : match_result->GetMatchedNodes()) { AscendString type; node.GetType(type); if (type != "Const") { continue; } Tensor value; if (node.GetAttr("value", value) != GRAPH_SUCCESS) { return false; } return IsZero(value); } return false; } GraphUniqPtr Replacement(const std::unique_ptr<MatchResult> &match_result) override { auto builder = es::EsGraphBuilder("add_zero_replacement"); auto x = builder.CreateInput(0); return builder.BuildAndReset({x}); } private: bool IsZero(const Tensor &tensor) const { switch (tensor.GetTensorDesc().GetDataType()) { case DT_FLOAT: return std::fabs(*reinterpret_cast<const float *>(tensor.GetData())) < 1e-6; case DT_DOUBLE: return std::fabs(*reinterpret_cast<const double *>(tensor.GetData())) < 1e-15; case DT_INT32: return *reinterpret_cast<const int32_t *>(tensor.GetData()) == 0; default: return false; } } }; REG_FUSION_PASS(AddZeroPass).Stage(CustomPassStage::kBeforeInferShape);

A complete runnable example is available at AddZeroPass C++ Example.

3. Patterns: Define What to Find

Patterns()returns one or more patterns. Each pattern is a small graph.

std::vector<PatternUniqPtr> Patterns() override { std::vector<PatternUniqPtr> patterns; auto builder = es::EsGraphBuilder("pattern"); auto a = builder.CreateInput(0); auto b = builder.CreateInput(1); auto c = builder.CreateInput(2); auto matmul = es::MatMul(a, b); auto add = es::Add(matmul, c); auto graph = builder.BuildAndReset({add}); patterns.emplace_back(std::make_unique<Pattern>(std::move(*graph))); return patterns; }

This pattern represents:

a ----\ MatMul ----\ b ----/ Add ---- pattern output c ----------------/

To support bothMatMul + AddandBatchMatMulV2 + Add, create two patterns and add both topatterns.

When writing patterns, note:

• External inputs are declared withCreateInput.
• Tensors that will still be used externally after replacement must be outputs of the pattern.
• Input count for normal operators must match the real graph.
• Do not use control edges, subgraphs, or nodes with dynamic input/output counts in patterns.

4. MeetRequirements: Determine Whether to Replace

Patterns()only handles topology matching. If additional checks are needed after topology matching, writeMeetRequirements().

For example, after matchingAdd(x, Const), verify that Const equals 0:

bool MeetRequirements(const std::unique_ptr<MatchResult> &match_result) override { for (const auto &node : match_result->GetMatchedNodes()) { AscendString type; node.GetType(type); if (type != "Const") { continue; } Tensor value; if (node.GetAttr("value", value) != GRAPH_SUCCESS) { return false; } return IsZero(value); } return false; }

If no filtering is needed, this method can be omitted; it returnstrueby default.

5. Replacement: Define What to Replace With

Replacement()returns the replacement graph.

When deletingAdd(x, 0), the replacement graph has only one external input:

GraphUniqPtr Replacement(const std::unique_ptr<MatchResult> &match_result) override { auto builder = es::EsGraphBuilder("replacement"); auto x = builder.CreateInput(0); return builder.BuildAndReset({x}); }

When fusingMatMul + AddintoGEMM:

GraphUniqPtr Replacement(const std::unique_ptr<MatchResult> &match_result) override { auto builder = es::EsGraphBuilder("replacement"); auto a = builder.CreateInput(0); auto b = builder.CreateInput(1); auto c = builder.CreateInput(2); auto alpha = builder.CreateScalar(1); auto beta = builder.CreateScalar(1); auto gemm = es::GEMM(a, b, c, alpha, beta); return builder.BuildAndReset({gemm}); }

If the pass is registered after InferShape, shape information for new nodes in the replacement needs to be handled manually. Refer to existing examples for usingGeUtils::InferShapewhen shape inference is needed for replacement.

6. CaptureTensor: Read Key Tensors in Pattern

WhenMeetRequirements()orReplacement()needs to know which real node corresponds to a intermediate tensor, capture it in the pattern.

auto matmul = es::MatMul(a, b); auto add = es::Add(matmul, c); auto graph = builder.BuildAndReset({add}); auto pattern = std::make_unique<Pattern>(std::move(*graph)); pattern->CaptureTensor({*matmul.GetProducer(), 0}); patterns.emplace_back(std::move(pattern));

After successful matching, retrieve frommatch_result:

NodeIo matmul_output; if (match_result->GetCapturedTensor(0, matmul_output) != GRAPH_SUCCESS) { return false; }

Refer to capture tensor C++ example.

7. PatternMatcherConfig: Put Simple Conditions in Matcher

If you want the matcher to directly check Const values or IR attributes, pass configuration to thePatternFusionPassconstructor.

class MatmulAddFusionPass : public PatternFusionPass { public: MatmulAddFusionPass() : PatternFusionPass(PatternMatcherConfigBuilder() .EnableConstValueMatch() .EnableIrAttrMatch() .Build()) {} };

Common configurations:

If judgment requires floating-point tolerance, dtype normalization, or more complex logic, it is still recommended to put it inMeetRequirements().

Refer to PatternMatcherConfig C++ example.

8. Register Execution Stage

UseREG_FUSION_PASSto registerPatternFusionPass:

REG_FUSION_PASS(AddZeroPass).Stage(CustomPassStage::kBeforeInferShape);

Common stages:

For initial development, usekBeforeInferShape.

9. Writing DecomposePass

If you want to decompose one node into multiple nodes, useDecomposePass.

Skeleton is as follows:

#include "ge/fusion/pass/decompose_pass.h" #include "es_all_ops.h" using namespace ge; using namespace ge::fusion; class MyDecomposePass : public DecomposePass { public: explicit MyDecomposePass(const std::vector<AscendString> &op_types) : DecomposePass(op_types) {} protected: bool MeetRequirements(const GNode &matched_node) override { // Read matched_node attributes to determine if decomposition is needed return true; } GraphUniqPtr Replacement(const GNode &matched_node) override { auto builder = es::EsGraphBuilder("replacement"); // Construct subgraph for replacing matched_node ... return builder.BuildAndReset({output}); } }; REG_DECOMPOSE_PASS(MyDecomposePass, {"Conv2D"}).Stage(CustomPassStage::kAfterInferShape);

The second parameter ofREG_DECOMPOSE_PASSis the list of operator types to match. GE will pass real nodes of these types to the pass, thenMeetRequirements()makes further judgment.

Complete example see DecomposePass C++ example.

10. Compilation and Running

Each example directory comes withCMakeLists.txt. General process is as follows.

Set CANN environment variables:

source ${ASCEND_PATH}/set_env.sh

Compile and install pass dynamic library:

mkdir build cd build cmake .. make -j$(nproc) <target_name> make install

CMake configuration not expanded in this document, use examples as template during development:

• AddZeroPass CMakeLists.txt
• MatMul+Add CMakeLists.txt

If need to add new header file paths or link libraries, append in corresponding positions of exampleCMakeLists.txt, do not delete original configuration.

Offline compilation can useatcto trigger:

atc --model=./model.onnx --framework=5 --soc_version=xxx --output=./model

Online scenario usually triggers GE compilation throughtorch_forward.pyin examples.

11. Verification and Troubleshooting

Recommend enabling graph dump:

export DUMP_GE_GRAPH=1

Compare graphs before and after pass:

• PreRunBegin: Before pass execution.
• RunCustomPass...: After custom pass execution.

Common problems:

When more logs needed, can set:

export ASCEND_SLOG_PRINT_TO_STDOUT=1 export ASCEND_GLOBAL_LOG_LEVEL=0

When usingatc, can add--log=debug.

12. Recommended Reading Order

1. Fusion Pattern Pass Mechanism
2. AddZeroPass C++ example
3. MatMul+Add C++ example
4. capture tensor C++ example
5. PatternMatcherConfig C++ example
6. DecomposePass C++ example

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge