Skip to content

Update DIV for OpenVINO backend#167

Open
zhaixuejun1993 wants to merge 1 commit into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/arch-qwen2moe
Open

Update DIV for OpenVINO backend#167
zhaixuejun1993 wants to merge 1 commit into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/arch-qwen2moe

Conversation

@zhaixuejun1993
Copy link
Copy Markdown
Collaborator

This pull request enhances the OpenVINO backend's handling of division operations in ggml-openvino by introducing a special case for the SiLU (Sigmoid Linear Unit) division pattern and improving numerical stability for certain compute paths. The changes aim to optimize performance and prevent precision-related issues, especially on GPU backends.

Pattern recognition and optimization:

  • Added detection of the SiLU division pattern (silu(x) / x) in translate_div and replaced it with a direct sigmoid operation for efficiency and accuracy. [1] [2]

Numerical precision improvements:

  • Marked reciprocal/divide subgraphs and their output conversions as precision-sensitive when using FP32 computation to prevent unwanted downcasting to FP16, which can cause overflows on small values (e.g., in silu(x)/x for certain models).

Dependency and header updates:

  • Included additional OpenVINO headers (precision_sensitive_attribute.hpp, multiply.hpp, sigmoid.hpp) and the main ggml.h header to support the new functionality and pattern recognition logic.## Overview

Additional information

Requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant