Skip to content

[TASK] Add GELU Ball Unit #8

@shirohasuki

Description

@shirohasuki

Deliverables

  • Add a GELU ball RTL implementation in the prototype lib (under the arch path).
  • A Pull Request (PR) containing a test written in C for the GELU operation and a README to introduce your design.
  • Report the performance results in this issue.

Task Description

  • GELU (Gaussian Error Linear Unit) is a smooth, non-monotonic activation function that has become increasingly popular in modern deep learning architectures, particularly in transformer models like BERT. Unlike ReLU, which applies a hard threshold, GELU provides a probabilistic approach to neuron activation.
  • Hardware implementation of GELU activation function primarily falls into two categories: lookup table and approximation methods. Lookup table approaches store pre-computed GELU values in memory and use interpolation between table entries to calculate intermediate values, offering predictable latency and good accuracy but requiring significant memory resources. Approximation methods use mathematical functions to approximate the GELU behaviour, utilizing arithmetic operations for computation while requiring minimal memory storage.
  • For this implementation, we will adopt the approximation method to approximate the GELU function
  • Please refer to the previous Pull Request (Completed the development of ReluBall and further improved the operation manual #6) for the detailed implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions