Update String Evaluator #6615
Update String Evaluator #6615
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
vowelparrot
force-pushed
the
vwp/string_eval_protocol
branch
from
78e2b33 to
b1b9b16
Compare
| _SUPPORTED_CRITERIA.update(d) | ||
|
|
||
|
|
||
| class CriteriaEvalChain(LLMChain): |
There was a problem hiding this comment.
Why don't we make this subclass StringEvaluator?
There was a problem hiding this comment.
Because it's a protocol and the metaclass differs from the chain. We could also make it just StringEvaluatorChain, which would probably be more conventional in langchain. I'll go ahead and do that
| _SUPPORTED_CRITERIA.update(d) | ||
|
|
||
|
|
||
| def get_criteria_evaluator( |
There was a problem hiding this comment.
Q: instead of having this method here, can we just have an to_run_evaluator method on StringEvaluator that could convert any arbitrary StringEvaluator into a RunEvaluator (which expects a Run)?
There was a problem hiding this comment.
Hmmmm yeah we could
There was a problem hiding this comment.
Think I'd do in a subsequent PR
vowelparrot
force-pushed
the
vwp/string_eval_protocol
branch
5 times, most recently
from
79f8d13 to
bdd6116
Compare
vowelparrot
force-pushed
the
vwp/string_eval_protocol
branch
from
bdd6116 to
fd82f46
Compare
evaluate_strings