BETTER-CONVEX

Filters

Full-text search and complex query patterns.

In this guide, we'll explore advanced querying in Convex. You'll learn to implement full-text search with relevance scoring, use streams for complex filtering with consistent pagination, and combine multiple queries with UNION and JOIN patterns.

Overview

Convex provides powerful querying capabilities beyond basic filtering:

FeatureDescription
Full-text searchSearch string fields with relevance scoring
StreamsComplex filtering with consistent pagination
UNION patternsCombine multiple queries into one paginated result
JOIN patternsPaginate through related documents
DISTINCTGet unique values efficiently

Let's explore each one.

Warning: Streams do NOT support withSearchIndex(). You cannot combine streams with full-text search - choose one approach based on your needs.

Full-Text Search

Full-text search enables searching string fields with relevance scoring. Each search index can search ONE field, with optional filter fields for equality filtering.

Schema Definition

Let's start by adding search indexes to your schema. Use the naming pattern search_<searchField>:

convex/functions/schema.ts
import { v } from 'convex/values';
import { defineEnt, defineEntSchema } from 'convex-ents';

const schema = defineEntSchema({
  articles: defineEnt({
    title: v.string(),
    content: v.string(),
    author: v.string(),
    category: v.string(),
  })
    // Search the content field, with category and author as filter fields
    .searchIndex('search_content', {
      searchField: 'content',
      filterFields: ['category', 'author'],
    })
    // Separate index for title search
    .searchIndex('search_title', {
      searchField: 'title',
    }),
});

Note: Each search index can only search ONE field. Add filterFields for fast equality filtering before text search.

Now let's implement a basic search query:

convex/functions/articles.ts
import { z } from 'zod';
import { zid } from 'convex-helpers/server/zod4';
import { publicQuery } from '../lib/crpc';

export const search = publicQuery
  .input(z.object({
    query: z.string(),
    limit: z.number().min(1).max(100).default(10),
  }))
  .output(z.array(z.object({
    _id: zid('articles'),
    title: z.string(),
    content: z.string(),
  })))
  .query(async ({ ctx, input }) => {
    return await ctx
      .table('articles')
      .search('search_content', (q) => q.search('content', input.query))
      .take(input.limit);
  });

Important: Always use .take() or .paginate() with search queries to limit results.

Search with Filters

Filter fields enable fast pre-filtering before text search. Only equality filters are supported:

convex/functions/articles.ts
export const searchByCategory = publicQuery
  .input(z.object({
    query: z.string(),
    category: z.string(),
    author: z.string().optional(),
  }))
  .output(z.array(z.object({
    _id: zid('articles'),
    title: z.string(),
    content: z.string(),
    category: z.string(),
  })))
  .query(async ({ ctx, input }) => {
    return await ctx
      .table('articles')
      .search('search_content', (q) => {
        let search = q.search('content', input.query).eq('category', input.category);
        if (input.author) {
          search = search.eq('author', input.author);
        }
        return search;
      })
      .take(20);
  });

Use .paginate() with the paginated builder for cursor-based pagination:

convex/functions/articles.ts
const ArticleSchema = z.object({
  _id: zid('articles'),
  _score: z.number().optional(),
  title: z.string(),
  content: z.string(),
  category: z.string(),
});

export const searchPaginated = publicQuery
  .input(z.object({
    query: z.string(),
    category: z.string().optional(),
  }))
  .paginated({ limit: 20, item: ArticleSchema })
  .query(async ({ ctx, input }) => {
    return await ctx
      .table('articles')
      .search('search_content', (q) => {
        let search = q.search('content', input.query);
        if (input.category) {
          search = search.eq('category', input.category);
        }
        return search;
      })
      .paginate({ cursor: input.cursor, numItems: input.limit });
  });

Tip: Search results include a _score field for relevance ranking. Results are automatically sorted by relevance.

Streams

Streams from convex-helpers enable complex filtering with consistent pagination. Unlike the built-in .filter() which causes variable page sizes, streams guarantee the requested number of items per page.

Installation

First, install convex-helpers:

bun add convex-helpers

Setup

Import the stream utilities:

import { stream, mergedStream } from 'convex-helpers/server/stream';
import schema from '../functions/schema';

Note: Stream initialization requires ctx.db as the first parameter (exception to the ctx.table rule). However, you SHOULD use ctx.table() inside stream callbacks for lookups.

filterWith - Consistent Page Sizes

Here's where streams shine. The built-in .filter() causes variable page sizes with pagination because it filters AFTER fetching. Use filterWith to filter BEFORE pagination for consistent pages:

convex/functions/characters.ts
import { stream } from 'convex-helpers/server/stream';
import schema from './schema';

const CharacterSchema = z.object({
  _id: zid('characters'),
  name: z.string(),
  categories: z.array(z.string()).nullable(),
  score: z.number(),
});

export const searchCharacters = publicQuery
  .input(z.object({
    category: z.string().optional(),
    minScore: z.number().optional(),
  }))
  .paginated({ limit: 20, item: CharacterSchema })
  .query(async ({ ctx, input }) => {
    return await stream(ctx.db, schema)
      .query('characters')
      .withIndex('private', (q) => q.eq('private', false))
      .filterWith(async (char) => {
        // Full TypeScript power - any logic you need
        if (input.category && !char.categories?.includes(input.category)) {
          return false;
        }
        if (input.minScore && char.score < input.minScore) {
          return false;
        }

        // Async lookups - use ctx.table() inside streams
        const author = await ctx.table('user').get(char.userId);
        return author !== null && !author.isBanned;
      })
      .paginate({ cursor: input.cursor, numItems: input.limit });
  });

Tip: filterWith supports any TypeScript logic including async lookups, array methods, and complex conditions that aren't possible with Convex's built-in filter expressions.

mergedStream - UNION Queries

Combine multiple queries into a single paginated stream. This is useful when you need results from different index ranges:

convex/functions/messages.ts
import { stream, mergedStream } from 'convex-helpers/server/stream';

const MessageSchema = z.object({
  _id: zid('messages'),
  text: z.string(),
  from: zid('user'),
  to: zid('user'),
  _creationTime: z.number(),
});

export const conversation = publicQuery
  .input(z.object({
    u1: zid('user'),
    u2: zid('user'),
  }))
  .paginated({ limit: 50, item: MessageSchema })
  .query(async ({ ctx, input }) => {
    // Stream of messages u1 → u2
    const sentMessages = stream(ctx.db, schema)
      .query('messages')
      .withIndex('from_to', (q) => q.eq('from', input.u1).eq('to', input.u2));

    // Stream of messages u2 → u1
    const receivedMessages = stream(ctx.db, schema)
      .query('messages')
      .withIndex('from_to', (q) => q.eq('from', input.u2).eq('to', input.u1));

    // Merge by creation time - both streams must be ordered by this field
    const allMessages = mergedStream(
      [sentMessages, receivedMessages],
      ['_creationTime']
    );

    return await allMessages.paginate({ cursor: input.cursor, numItems: input.limit });
  });

Note: All streams being merged MUST be ordered by the same fields specified in the merge. The second parameter ['_creationTime'] tells the merger how to interleave results.

flatMap - JOINs with Pagination

Join tables and paginate through the combined results:

convex/functions/messages.ts
const MessageWithFriendSchema = z.object({
  _id: zid('messages'),
  text: z.string(),
  from: zid('user'),
  to: zid('user'),
  fromBestFriend: z.boolean(),
});

export const messagesFromFriends = authQuery
  .input(z.object({}))
  .paginated({ limit: 20, item: MessageWithFriendSchema })
  .query(async ({ ctx, input }) => {
    // Start with friends list
    const friends = stream(ctx.db, schema)
      .query('friends')
      .withIndex('userId', (q) => q.eq('userId', ctx.userId));

    // For each friend, get their messages to the current user
    const messages = friends.flatMap(
      (friend) =>
        stream(ctx.db, schema)
          .query('messages')
          .withIndex('from_to', (q) =>
            q.eq('from', friend.friendId).eq('to', ctx.userId)
          )
          .map((message) => ({
            ...message,
            fromBestFriend: friend.isBest,
          })),
      ['from', 'to'] // Index fields of the inner stream
    );

    return await messages.paginate({ cursor: input.cursor, numItems: input.limit });
  });

distinct - Unique Values

Get unique values efficiently without loading all documents:

convex/functions/messages.ts
const UserSchema = z.object({
  _id: zid('user'),
  name: z.string(),
  email: z.string(),
});

export const messageRecipients = authQuery
  .input(z.object({}))
  .paginated({ limit: 20, item: UserSchema })
  .query(async ({ ctx, input }) => {
    // Get unique recipients from messages
    const recipients = await stream(ctx.db, schema)
      .query('messages')
      .withIndex('from_to', (q) => q.eq('from', ctx.userId))
      .distinct(['to']) // Only keep first message per recipient
      .map(async (message) => {
        const user = await ctx.table('user').get(message.to);
        return user!;
      })
      .paginate({ cursor: input.cursor, numItems: input.limit });

    return recipients;
  });

Search vs Streams

Understanding when to use each approach is critical. Here's a comparison:

FeatureSearchStreams
Text matchingFull-text search with relevanceNo text search
Complex filtersfilterFields only (equality)Full TypeScript
Range queriesNoYes
PaginationStandardConsistent page sizes
UNION/JOINNoYes (mergedStream, flatMap)
PerformanceOptimized for textScans documents
CombinedNot supportedN/A

Choosing the Right Approach

  1. Need text search? → Use search indexes
  2. Need complex filters with pagination? → Use streams
  3. Need range queries (greater than, less than)? → Use streams
  4. Need UNION or JOIN operations? → Use streams
  5. Need both text search AND complex filters? → See options below

Combining Search and Complex Filters

Since streams don't support search, you have three options.

Option 1: Add more filterFields (recommended)

Add equality filters to your search index:

.searchIndex('search_content', {
  searchField: 'content',
  filterFields: ['category', 'author', 'status', 'dateGroup'],
})

Option 2: Separate query paths

Use search when text is provided, streams otherwise:

export const searchOrFilter = publicQuery
  .input(z.object({
    query: z.string().optional(),
    category: z.string().optional(),
    startDate: z.number().optional(),
  }))
  .paginated({ limit: 20, item: ArticleSchema })
  .query(async ({ ctx, input }) => {
    if (input.query) {
      // Text search path - limited filtering
      return await ctx
        .table('articles')
        .search('search_content', (q) => {
          let search = q.search('content', input.query!);
          if (input.category) {
            search = search.eq('category', input.category);
          }
          return search;
        })
        .paginate({ cursor: input.cursor, numItems: input.limit });
    }

    // Stream path - full filtering power
    return await stream(ctx.db, schema)
      .query('articles')
      .withIndex('publishedAt')
      .filterWith((article) => {
        if (input.category && article.category !== input.category) return false;
        if (input.startDate && article.publishedAt < input.startDate) return false;
        return true;
      })
      .paginate({ cursor: input.cursor, numItems: input.limit });
  });

Option 3: Post-process search results (small datasets only)

// Only viable for small result sets
const searchResults = await ctx
  .table('articles')
  .search('search_content', (q) => q.search('content', query))
  .take(100); // Hard limit

const filtered = searchResults.filter((article) =>
  article.publishedAt >= startDate && article.publishedAt <= endDate
);

Performance Considerations

Understanding performance is critical for production apps.

Search Performance

  • Search indexes have overhead - don't over-index
  • Use filterFields to narrow results before text matching
  • Always limit results with .take() or .paginate()

Stream Performance

Streams scan documents until the page is full. For expensive filters, limit scanning:

const results = await stream(ctx.db, schema)
  .query('posts')
  .filterWith(async (post) => expensiveCheck(post))
  .paginate({
    cursor: input.cursor,
    numItems: input.limit,
    maximumRowsRead: 1000, // Fail if scanning too many documents
  });

Warning: Without maximumRowsRead, a filter that matches few documents could scan the entire table. Set a reasonable limit to fail fast instead of timing out.

Stream Cursor Fragility

Stream cursors only work with identical stream construction:

// These cursors are NOT compatible
const stream1 = stream(ctx.db, schema)
  .query('messages')
  .withIndex('time');

const stream2 = stream(ctx.db, schema)
  .query('messages')
  .withIndex('time')
  .filterWith((m) => m.read); // Different structure!

Memory Efficiency

Streams are lazy - documents are fetched on-demand:

  • Minimal memory overhead
  • Good for large datasets
  • Use .collect() sparingly (loads all matching documents)

Best Practices

Here are key practices to follow when using filters and search.

  1. Create appropriate indexes - Each search field needs its own search index
  2. Limit results - Always use .take() or .paginate()
  3. Use filterFields - Pre-filter with equality before text search
  4. Set maximumRowsRead - Prevent runaway scans with streams
  5. Normalize search terms - Consider lowercase for consistent matching
  6. Handle empty queries - Decide behavior when search query is empty
  7. Use ctx.table() in callbacks - Inside stream operations, use ctx.table() not ctx.db

Next Steps

Ents

Triggers

Aggregates

convex-helpers

Triggers

Database triggers for automatic side effects.

Aggregates

O(log n) counts and sums with TableAggregate.

On this page

OverviewFull-Text SearchSchema DefinitionBasic SearchSearch with FiltersPaginated SearchStreamsInstallationSetupfilterWith - Consistent Page SizesmergedStream - UNION QueriesflatMap - JOINs with Paginationdistinct - Unique ValuesSearch vs StreamsChoosing the Right ApproachCombining Search and Complex FiltersPerformance ConsiderationsSearch PerformanceStream PerformanceStream Cursor FragilityMemory EfficiencyBest PracticesNext Steps