StayFresh

Static archive of workflow research and patterns

April 2026

What is Prompting: Operational Constraints

Prompting is not abstract instruction-writing. Prompting is a constrained pipeline where tokenization mechanics, token budgets, and control structures form the actual boundaries within which agent behavior operates.

Agents do not see "prompts." They see token sequences. Understanding prompting means understanding the mechanics that govern those sequences.

The Pipeline

Prompts follow a deterministic 7-stage process:

  1. Human input: raw text (words, images, other media)
  2. Tokenization: conversion to numerical token IDs (model-specific)
  3. Token IDs: numerical representation ready for processing
  4. LLM processing: core computation phase
  5. Output token IDs: numerical response
  6. Detokenization: conversion back to text tokens
  7. Human-readable output: final agent response

This pipeline is the mechanism. Everything that happens to the prompt happens within this process.

Three Hard Constraints

1. Tokenization (The Conversion Constraint)

Tokenization is the compilation step. Just as code must be compiled to machine instructions, prompts must be tokenized to numerical sequences.

This has three practical implications:

For agent specification: tokenization means agents do not see structural hints in formatting. They see token sequences. Decomposing specifications into structured sequences, not prose, respects how agents actually process input.

2. Max Tokens (The Working Memory Constraint)

Every LLM has a context window limit: the maximum tokens it can process in a single interaction. This is a fixed architectural constraint, not a soft preference.

Examples:

How it constrains agents: if max_tokens = 8,192 and your prompt uses 1,000 tokens, only 7,192 tokens remain for reasoning and output. Agents cannot reason deeper than the token budget allows. Token limits force agents to compress reasoning or fail requests that exceed capacity.

From a cost perspective: token limits directly translate to billing. Most LLM services charge (input_tokens + output_tokens) × price_per_token. Larger context windows mean higher costs if fully used.

The real insight: the context window is the agent's working memory. A 4K token limit is fundamentally more constrained than a 1M token limit. This affects what agents can hold in mind, what context they can reference, what reasoning chains they can execute.

3. Control Tokens (The Structure Constraint)

Control tokens are special tokens that organize prompt regions and guide LLM processing phases.

Examples:

These tokens are handled internally by modern APIs (you don't write them explicitly in OpenAI), but understanding them shows how prompts are actually structured. Control tokens segment reasoning phases. They tell the LLM where one phase ends and another begins. This explains why conversation state is preserved in message-based prompts but not in basic text prompts.

Prompt Types as Architectural Choices

The choice of how to structure prompts directly affects agent capability.

Basic Text Prompts

"Translate 'Hello' to French"

Messages Prompts

[
  { role: "user", content: "Translate 'Hello' to French" },
  { role: "assistant", content: "Bonjour" },
  { role: "user", content: "And 'goodbye'?" }
]

System Prompts

system: "You are a French translator. Be concise."
user: "Translate 'Hello'"

The choice between basic and message-based prompts determines whether an agent can maintain reasoning continuity across multiple requests. It is a fundamental architectural constraint, not a minor implementation detail.

Prompt Management (Version Control for Behavior)

As prompts evolve, they need versioning:

translation_openai_v1.0.0  # Initial version
translation_openai_v1.1.0  # Enhancement (minor version)
translation_openai_v2.0.0  # Major refactor (major version)

Why versioning matters:

The Design Implication

Agents cannot be understood independently of their prompting mechanism. The tokenization pipeline, token limits, and control structures are not implementation details. They are the operational constraints that determine what agents can do.

When designing agent specifications:

  1. Respect tokenization: structure specs as token sequences, not prose
  2. Budget tokens: reserve output space within context window limits
  3. Use system prompts: define agent boundaries globally, not per-request
  4. Choose prompt type carefully: messages-based for agents with memory, basic for stateless operations
  5. Version prompts: track which prompt versions produce which behaviors

Agents do not follow instructions abstractly. They operate within tokenized, token-budgeted, control-structured pipelines. Understanding prompting means understanding these constraints as first-class design elements.

Related Principles