The deprecated attention_pattern() and attention_patterns() functions provided rich interactive token-level visualization that has been lost in the transition to the new attention_heads() function:
This is a significant regression in user experience for exploring attention patterns. It would great to add this visualization to the new attention_heads() function as well.
The deprecated
attention_pattern()andattention_patterns()functions provided rich interactive token-level visualization that has been lost in the transition to the newattention_heads()function:This is a significant regression in user experience for exploring attention patterns. It would great to add this visualization to the new
attention_heads()function as well.