Using SMART for judging search queries?

[epugh]

Hi, I steward Quepid (github.com/o19s/quepid), and we're evaluating a better solution for rating query/document pairs. We have a basic rating interface, but it is definitely not a good UX for coders..

I'm debating building a more robust solution, however it appears that SMART does much of what we want.. I poked at it and the part that I struggle with is that I need to label query/doc pairs as a 0,1,2,3 or a 0,1... SO it's not labeling per se, it's judgement...

Is that too far away from what SMART does?

Eric

[AstridKery]

Hi Steward! So to clarify, you are trying to label things with multiple labels? SMART right now only supports single-label coding. One workaround is that you could make a label for each combination, but of course that scales pretty fast.