How to Give AI Agents Enough Context to Be Useful

The speaker’s core claim is that AI agents become materially more useful when they are given broad, evidence-grounded company context and a workflow that includes critique and prioritization. In Skyvern’s case, this is presented not as theory but as an operating system for a small company: agents are used across PM, marketing, support, and minor engineering tasks, and the company’s remote, heavily recorded communication habits make this easier. The talk’s strongest practical point is organizational rather than model-specific: if important business context is not captured in searchable systems, agents cannot use it well.

Key insights

1. Poor agent output is framed primarily as a context problem, not just a model problem: The speaker argues that agents often produce ‘slop’ because they follow instructions without having enough business-specific context. He compares this to onboarding a new employee: willingness to help is not enough without exposure to the company’s history, customer issues, internal decisions, and operating norms.

   Why it matters: This shifts the implementation focus away from prompt wording alone and toward building systems that expose company memory. For teams adopting agents, the bottleneck may be context availability rather than raw model capability.

2. High-value context includes operational exhaust across many systems: The talk explicitly names email, Slack, Notion, customer call recordings, customer communications, and even database access as context sources. The important idea is not any single tool, but that agents should be able to correlate signals across documentation, conversations, customer history, and product behavior.

   Why it matters: Useful agent behavior depends on cross-source retrieval and synthesis. Teams that keep knowledge fragmented or inaccessible should expect weaker results, especially for diagnosis, prioritization, and customer-specific work.

3. Remote-first communication creates an advantage because more context is naturally recorded: The speaker claims remote companies have an ‘unfair advantage’ because key business context is more often written or recorded in Slack messages and calls, whereas in-person context is frequently exchanged verbally and then lost.

   Why it matters: This is a strategic organizational insight: agent readiness is partly determined by how a company communicates. In-person companies may need deliberate process changes to capture context that remote companies record by default.

4. The PRD workflow uses retrieval, adversarial review, and prioritization to reduce low-quality output: For drafting product requirement documents, the agent searches call recordings, Slack, Notion, and customer communications on a topic, produces an evidence-grounded draft, then passes it through sub-agents for adversarial review and finally through a prioritization framework such as RICE to remove weak requirements.

   Why it matters: This shows a concrete mechanism for improving agent output quality: don’t stop at first-draft generation. Add evidence retrieval, structured critique, and an explicit prioritization filter to cut junk requirements.

5. Acceptance of agent output required iterative tuning against user skepticism: The speaker says early PRD drafts were dismissed internally as ‘slop,’ and he kept tuning until teammates found them usable as a starting point for feature drafting.

   Why it matters: This suggests successful deployment is iterative and social, not just technical. Teams should expect resistance and should measure success by whether outputs become practically reusable, not whether they are impressive in isolation.

6. Grounding documents in source material increases downstream usefulness: In the CAPTCHA-related example, the generated strategy document included links to specific recordings so the people implementing the feature could inspect the original customer evidence.

   Why it matters: This matters because it turns AI output from generic summary into auditable work product. Traceability to source evidence can improve trust, speed handoff, and reduce hallucination risk.

Strategic implications

• Companies that want useful agents may need to treat knowledge capture as core infrastructure. If important context remains implicit, verbal, or siloed, agent performance will likely stay shallow.
• Remote-native practices appear better aligned with agent adoption because they generate searchable records by default. In-person organizations may need compensating processes such as more written decisions, recorded meetings, and public channels.
• The most effective agent workflows described here are multi-stage: retrieve evidence, draft, critique, and prioritize. Single-pass generation is implicitly portrayed as insufficient for serious internal work products.
• Agent deployment can influence org design. Policies like limiting DMs and recording more conversations may improve machine usability of context, but they also change privacy, culture, and communication norms.

Signals to watch

• Whether more companies start redesigning communication practices specifically to make context machine-readable, not just human-readable.
• Whether teams move from isolated prompt usage to retrieval-plus-review workflows with explicit critique and prioritization stages.
• Whether source-linked AI outputs become a norm for internal documents, especially where trust and auditability matter.
• Whether remote companies continue to show disproportionate gains from agents because of better recorded context.

Caveats

• The transcript is a short talk and provides limited operational detail on tooling, architecture, evaluation, security, privacy controls, or failure modes.
• The revenue/run-rate and company performance claims are stated by the speaker and are not independently verified in the provided material.
• Some phrases in the transcript are noisy or repetitive, and the description is truncated, so finer details may be missing.