System prompt design for products

The task

You are designing instructions for a product assistant, not a one-off chat. The prompt must hold up across many users, edge cases, and tool calls. If you are new to the term, start with system prompts before designing the product version.

Why the old approach is outdated

Old templates often start with a persona and then pile on rules. Product assistants need operating boundaries first: what the assistant may do, what it must not do, when it should call tools, and when a human takes over.

For the tool side of that boundary, see tool use and function calling.

The current approach

Design a system prompt for a billing-support assistant.

Product facts:
- The assistant drafts replies for human agents.
- It can read ticket text and policy excerpts.
- It cannot issue refunds, change plans, or promise resolution timelines.

Return a system prompt with sections for task scope, source rules, tool-use rules,
handoff cases, and output format. Keep it short enough for a product team to maintain.

When to use a different model

Use ChatGPT when the system prompt needs API request examples or structured output design. Use Gemini when the product assistant handles media. Use Mistral when the product already targets Mistral’s API, open-weight models, or self-deployment path.

What to avoid

• Treating tone as the whole system prompt.
• Hiding product policy inside a long paragraph.
• Letting the assistant decide its own authority.
• Forgetting that server-side checks still matter when money, permissions, or private data are involved.