We have seen this requested a number of times by now. https://forum.posit.co/t/using-custom-metric-sets-in-tidymodel-workflows/208824