Anti-pattern 1: calling Gemini Ultra for everything
Gemini Ultra (or GPT-4-class models) costs 10 to 30 times more per token than smaller models. Many teams default to the most capable model because it "just works" during prototyping, then never re-evaluate.
Fix: build a model router. Classify each incoming request by complexity. Simple lookups, short summaries, and classification tasks go to Gemini Flash or Haiku. Only complex reasoning, multi-step synthesis, and long-context tasks go to Pro or Ultra. In most production systems, 60-80% of requests can be served by the cheaper tier.
Anti-pattern 2: no context caching
Vertex AI supports prompt caching (as does the Anthropic API). A system prompt that is 10k tokens, sent with every request at $3/M tokens, costs $30 for every million calls before the user has typed a single word.
Fix: cache any context that is static or changes infrequently: system prompts, retrieved document sets, few-shot examples. Cache hits cost about 10% of full input price.
Anti-pattern 3: synchronous batch jobs
Teams run nightly document processing jobs synchronously, one document at a time, each blocked on the previous. This is slow and expensive because you pay for idle wait time between calls.
Fix: use the Vertex AI batch prediction API for jobs over ~1,000 documents. Batch jobs run asynchronously, are eligible for spot discounts, and typically cost 50% less per token than online serving.