AI-powered search and recommendation features are increasingly common in SaaS products. Your product might use embeddings for semantic search, collaborative filtering for recommendations, or LLM-powered query understanding.

These features produce non-deterministic output. The same query might return different results based on user history, model state, or randomization. Traditional assertion-based testing ("assert results contain X") is brittle and unreliable.

Testing Patterns for Non-Deterministic Features

Relevance testing: Search for a known term that should always appear in results. If searching "pricing" never returns the pricing page, something is broken. This is a weak but reliable assertion.

Ranking testing: For known queries, verify that the most relevant result appears in the top N positions. Exact position may vary, but the top result should be consistently relevant.

Empty state testing: Search for nonsensical terms. Verify the empty state renders correctly and doesn't crash. AI-generated search often doesn't handle zero-result cases.

Injection testing: Search for SQL injection patterns, XSS payloads, and prompt injection attempts. Verify the search safely handles malicious input.

Performance testing: Measure search latency. AI-powered search can be slow if embeddings are computed on each request rather than pre-indexed.

Boundary testing: Search with extremely long queries, special characters, Unicode, and empty strings. Verify the system handles each gracefully.

TestSprite tests search and recommendation features as part of the full application test suite. The agent performs searches, verifies results are relevant, checks error handling for edge cases, and validates that the UI correctly displays both results and empty states.

For products with AI-powered features, testing the non-deterministic parts is as important as testing the deterministic ones.

Try TestSprite free →