Regex Match
To evaluate chain or runnable string predictions against a custom regex, you can use the regex_match
evaluator.
from langchain.evaluation import RegexMatchStringEvaluator
evaluator = RegexMatchStringEvaluator()
Alternatively via the loader:
from langchain.evaluation import load_evaluator
evaluator = load_evaluator("regex_match")
# Check for the presence of a YYYY-MM-DD string.
evaluator.evaluate_strings(
prediction="The delivery will be made on 2024-01-05",
reference=".*\\b\\d{4}-\\d{2}-\\d{2}\\b.*",
)
{'score': 1}
# Check for the presence of a MM-DD-YYYY string.
evaluator.evaluate_strings(
prediction="The delivery will be made on 2024-01-05",
reference=".*\\b\\d{2}-\\d{2}-\\d{4}\\b.*",
)
{'score': 0}
# Check for the presence of a MM-DD-YYYY string.
evaluator.evaluate_strings(
prediction="The delivery will be made on 01-05-2024",
reference=".*\\b\\d{2}-\\d{2}-\\d{4}\\b.*",
)
{'score': 1}
Match against multiple patterns
To match against multiple patterns, use a regex union "|".
# Check for the presence of a MM-DD-YYYY string or YYYY-MM-DD
evaluator.evaluate_strings(
prediction="The delivery will be made on 01-05-2024",
reference="|".join(
[".*\\b\\d{4}-\\d{2}-\\d{2}\\b.*", ".*\\b\\d{2}-\\d{2}-\\d{4}\\b.*"]
),
)
{'score': 1}
Configure the RegexMatchStringEvaluator
You can specify any regex flags to use when matching.
import re
evaluator = RegexMatchStringEvaluator(flags=re.IGNORECASE)
# Alternatively
# evaluator = load_evaluator("exact_match", flags=re.IGNORECASE)
evaluator.evaluate_strings(
prediction="I LOVE testing",
reference="I love testing",
)
{'score': 1}