Optimizing Context Delivery: Passing App Data to the LLM Engine

Managing codebase context for LLM-powered features is a balancing act between providing enough relevant information and avoiding noise. In the dyad project, we recently refined how the engine receives information about mentioned applications to improve payload accuracy and maintain performance.

The Context Problem

Previously, when a user mentioned an application, we would indiscriminately append codebase prefixes. This caused redundant information to be sent to the LLM, leading to cluttered prompts and increased token consumption. We needed a cleaner way to distinguish between direct mentions and general codebase context.

Refactoring the Payload

We updated the logic to handle mentioned applications as structured entities. By passing these apps—including their names and relevant file metadata—directly to the dyad-engine, we can now gate the inclusion of other codebase prefixes.

If the engine is enabled, the backend now prioritizes the specific mentioned apps rather than relying on global prefix injection. This results in a leaner, more precise request payload.

Implementation Example

Here is how we structured the payload transformation in TypeScript:

function prepareEnginePayload(options: EngineOptions, isEngineEnabled: boolean) {
  const payload: EnginePayload = {
    mentioned_apps: options.mentionedApps,
  };

  // Only include global context if the specialized engine is off
  if (!isEngineEnabled) {
    payload.otherCodebasePrefix = options.globalPrefix;
  }

  return payload;
}

This approach ensures that the engine receives exactly what it needs for the current task. By conditionally omitting otherCodebasePrefix, we significantly reduce unnecessary noise in the prompt window.

The Results

After refactoring the extractMentionedAppsCodebases utility to return both file paths and codebase information, we updated our end-to-end tests to snapshot the request payload. By asserting the exact structure of the dyad_options, we confirmed that we are successfully isolating the mentioned applications from the rest of the codebase.

The Takeaway

When working with LLM integrations, treat context as a limited resource. Instead of sending the "whole world" to the engine, explicitly pass only the files or applications the user has interacted with. Check your request payloads periodically to ensure you aren't leaking unnecessary context into your prompts.

Generated with Gitvlg.com