Support QA — gpt-4o regression check

Customer Support QA · gpt-4o · Jul 15, 09:31 PM

Status
passed
Pass rate
100%
Duration
21.81s

What is your refund policy?

passed
Expected output

Refunds are available within 30 days of purchase with a valid receipt.

Actual output

Refunds are available within 30 days of purchase with a valid receipt.

Metric scores
Answer Relevancy0.85

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.63

Output meets the faithfulness threshold of 0.5.

Hallucination0.42

Output meets the hallucination threshold of 0.5.

Contextual Precision0.66

Output meets the contextual precision threshold of 0.5.

How do I reset my password?

passed
Expected output

Go to Settings > Security and click 'Reset Password', then follow the emailed link.

Actual output

Go to Settings > Security and click 'Reset Password', then follow the emailed link.

Metric scores
Answer Relevancy0.55

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.85

Output meets the faithfulness threshold of 0.5.

Hallucination0.25

Output meets the hallucination threshold of 0.5.

Contextual Precision0.50

Output meets the contextual precision threshold of 0.5.

Can I upgrade my plan mid-cycle?

passed
Expected output

Yes, upgrades take effect immediately and are prorated for the remainder of the billing cycle.

Actual output

Yes, upgrades take effect immediately and are prorated for the remainder of the billing cycle.

Metric scores
Answer Relevancy0.79

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.75

Output meets the faithfulness threshold of 0.5.

Hallucination0.45

Output meets the hallucination threshold of 0.5.

Contextual Precision0.86

Output meets the contextual precision threshold of 0.5.

Do you support single sign-on (SSO)?

passed
Expected output

Yes, SSO via SAML and OIDC is available on the Enterprise plan.

Actual output

Yes, SSO via SAML and OIDC is available on the Enterprise plan.

Metric scores
Answer Relevancy0.58

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.74

Output meets the faithfulness threshold of 0.5.

Hallucination0.30

Output meets the hallucination threshold of 0.5.

Contextual Precision0.57

Output meets the contextual precision threshold of 0.5.

What happens to my data if I cancel my subscription?

passed
Expected output

Your data is retained for 90 days after cancellation, after which it is permanently deleted.

Actual output

Your data is retained for 90 days after cancellation, after which it is permanently deleted.

Metric scores
Answer Relevancy0.52

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.78

Output meets the faithfulness threshold of 0.5.

Hallucination0.38

Output meets the hallucination threshold of 0.5.

Contextual Precision0.62

Output meets the contextual precision threshold of 0.5.

Is there a free trial available?

passed
Expected output

Yes, a 14-day free trial is available on all paid plans, no credit card required.

Actual output

Yes, a 14-day free trial is available on all paid plans, no credit card required.

Metric scores
Answer Relevancy0.61

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.63

Output meets the faithfulness threshold of 0.5.

Hallucination0.11

Output meets the hallucination threshold of 0.5.

Contextual Precision0.59

Output meets the contextual precision threshold of 0.5.

How do I invite team members?

passed
Expected output

Navigate to Team Settings and click 'Invite Member' to send an email invitation.

Actual output

Navigate to Team Settings and click 'Invite Member' to send an email invitation.

Metric scores
Answer Relevancy0.90

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.73

Output meets the faithfulness threshold of 0.5.

Hallucination0.50

Output meets the hallucination threshold of 0.5.

Contextual Precision0.57

Output meets the contextual precision threshold of 0.5.

What regions do you have data centers in?

passed
Expected output

We operate data centers in US-East, US-West, EU-Central, and AP-Southeast.

Actual output

We operate data centers in US-East, US-West, EU-Central, and AP-Southeast.

Metric scores
Answer Relevancy0.57

Output meets the answer relevancy threshold of 0.5.

Faithfulness0.72

Output meets the faithfulness threshold of 0.5.

Hallucination0.22

Output meets the hallucination threshold of 0.5.

Contextual Precision0.66

Output meets the contextual precision threshold of 0.5.