---
title: "Token efficiency"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Token efficiency}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
Keep data and computation in R, and send models enough context to choose useful actions.
For inline work, an object around 4 GB can remain in the same R session across turns. The model
receives size-limited descriptions, previews and tool output that may include data values, while
its R tool computes on the object. No repeated export and
reload cycle is required; R copies, computation time and request latency still depend on the
workflow. Workers receive serialized context rather than sharing the parent workspace.
Four practices help:
- Refer to live objects with a compact description rather than printing large tables into a
prompt. The model can compute on those objects with the `r` tool.
- Compose several deterministic operations in one R call and return a bounded result.
- Reuse a session's stable prompt and growing conversation when the provider supports prompt
caching. Changing models, tools, compaction or rewind can change that prefix.
- Inspect usage and set budgets before starting a costly workflow.
The executed examples use fake providers. Their request counts and estimates demonstrate the
interfaces; they are not measurements of a real model's speed, quality or bill.
## Inspecting the prompt
``` r
library(gptr)
view = gptr_prompt(preset = "minimal")
attr(view, "tool_names")
#> [1] "read" "r" "edit" "write"
view$sections
#> name tier tokens
#> 1 preamble T0 42
#> 2 tools T0 86
#> 3 rules T0 348
#> 4 modes T0 84
#> 5 context T0 152
view$total_tokens
#> [1] 1846
```
`gptr_prompt()` inspects the system sections and tool definitions with estimated token counts;
printing it shows their text. It does not query a provider. The `standard` preset describes
most capabilities as functions in `peter$`, with the core model-visible `r`, `read`, `edit`
and `write` tools and `ask` when applicable. `view$total_tokens` already counts the tool array
and a preview of the first message; your prompt and the objects you attach add to it.
## Object descriptions and result size
``` r
writeLines(gptr_describe(iris))
#> 150 x 5, 6.8 KB
#> $ Sepal.Length min 4.3, median 5.8, max 7.9; no NA
#> $ Sepal.Width min 2, median 3, max 4.4; no NA
#> $ Petal.Length min 1, median 4.35, max 6.9; no NA
#> $ Petal.Width min 0.1, median 1.3, max 2.5; no NA
#> $ Species 3 levels, top: setosa (50), versicolor (50), virginica (50); no NA
writeLines(gptr_describe(iris, budget = 60))
#> 150 x 5, 6.8 KB
#> Sepal.Length:dbl Sepal.Width:dbl Petal.Length:dbl Petal.Width:dbl Species:fct
```
`gptr_describe()` summarizes class, shape and values within a budget (150 tokens by default).
Packages can register methods for their own classes. Generative-model context normally uses
these descriptions and names; **System 1 questions send their input states**, so their egress
and size rules differ. See [Typed decisions](system-one.html).
``` r
work = new.env()
session = peter("Compare mpg and weight by cylinder count.", mtcars,
model = fake, envir = work, mode = auto)
work$slopes
#> 4 6 8
#> -5.647025 -2.780106 -2.192438
```
The calculation runs in R once the model requests it. Return a small table, statistic or plot
rather than a full intermediate object. Long `r` output keeps a bounded head and tail; the
full spilled text can be read through `peter$out(id)` when an output ID was supplied.
## Reported usage and estimated components
``` r
gptr_usage(session, by = "model")
#> group requests input output cache_read cache_write cost
#> 1 fake/fake-1 2 3382 48 0 0 0
#>
#> # 1 row
#> 2 requests, 3.4k tokens in, $0.0000
ledger = gptr_usage(session, detail = TRUE)
aggregate(tokens ~ component, data = ledger, FUN = sum)
#> component tokens
#> 1 attached 132
#> 2 environment 150
#> 3 images 0
#> 4 other 156
#> 5 project 0
#> 6 t0 2720
#> 7 t1 618
#> 8 tool_results 53
#> 9 tools 1926
#> 10 transcript 70
#> 11 workspace 58
```
The usage summary contains provider-reported input, output, cache reads/writes and cost.
`by = "session"`, `"agent"`, `"model"` or `"route"` groups it; child requests are included
once. `gptr_usage()` without a session also includes the process-level System 1 log.
The detailed ledger estimates components such as tools, workspace, attached objects,
transcript and tool results. Its sum can differ from reported usage because tokenizers,
provider accounting, images and fake-provider behavior differ. Unknown reported usage or
cost stays `NA`, not zero. A local model's known zero metered charge does not imply zero
hardware cost or latency.
## Budgets and partial results
``` r
budget_error = tryCatch(
peter("Summarize mpg.", model = small, budget = list(tokens = 500)),
gptr_error_budget = function(e) e
)
class(budget_error)[1]
#> [1] "gptr_error_budget_tokens"
budget_error$session$status
#> [1] "budget"
```
The estimated input for the first request already exceeds 500 tokens, so this call stops
before that request. A top-level run's defaults are 2 million tokens and 5 US dollars;
`budget = list(tokens = ..., cost = ..., turns = ...)` supplies limits for that call. Change
settings deliberately when different defaults are needed:
``` r
gptr_config(budget = list(tokens = 20000, cost = 1, turns = 10), .scope = "session")
session = peter("Review this analysis.", budget = list(tokens = 12000, turns = 6))
```
Checks happen before requests using estimated next-input tokens and usage already recorded.
The next output or final provider charge can exceed a remaining allowance; unknown usage
cannot establish a hard monetary ceiling. A budget is a stopping policy, not a billing
reservation. At the console the UI can ask to extend it; without a person present the call
signals a budget condition carrying the partial session as `e$session`. Inspect that
session's status, transcript, workspace and usage before deciding whether to continue.
Teams and fan-outs share the root call's budget; launching more children does not give each
one a fresh root allowance. With `gptr_parallel()`, member `budget =`, `tools =` and per-run
`.opts` are not applied. Set the intended limits through the governing settings or enclosing
run, cap concurrency and inspect the returned children's statuses. See
[Teams and background work](teams-and-background.html).
## Compaction
At the chat prompt, `/compact` replaces older model context with a summary. Add a focus when
particular details must survive, for example `/compact retain the model comparison and next
steps`. Automatic compaction follows the configured context and cold-cache thresholds; see
[Configuration](configuration.html).
Compaction can make a model request and add usage. It keeps the R objects already in your
session, while the model no longer sees every older turn verbatim. Preserve important decisions
in project files and named R objects. A changed request prefix can also affect provider cache
reuse.
## Three different caches
| Mechanism | What is reused | Inspect |
|---|---|---|
| Provider prompt cache | A repeated request prefix | `gptr_usage(session)` |
| System 1 answer cache | Matching input, question and model | Metadata and `gptr_cache()` |
| Replay / System 2 cache | Recorded code and saved answer | `gptr_blocks(path)`, `gptr_cache()` |
A stable prefix makes provider cache reuse possible. Providers decide whether a cache hit
occurs, subject to their support, minimum size, expiry and pricing; gptr does not guarantee a
hit after the first request. Compaction, image elision, changed resources or rewind can break
the prefix. Document replay makes no model request for a usable recorded call but still runs
its R code. Clearing an answer cache changes what can be reused.
## Explicit ways to reduce context
These live-call examples assume a configured model. For unattended use of a remote provider,
save its egress acknowledgment first, as in [Language models](language-models.html).
`context = "none"` still requires that consent because project instructions and environment
details remain; offline providers and local loopback routes are exempt (Ollama only while
local-only inference is on).
``` r
peter("Summarize the model fit.", fit, .opts = list(context = "names"))
peter("Find the mean of x. Return one number.", .opts = list(context = "none"))
peter("Compute the mean.", x, .opts = list(preset = "minimal"))
```
`context = "names"` lists workspace names and uses headers for explicitly attached objects.
`"none"` omits both built-in workspace and attached-object descriptions; the R bindings still
exist and can be used by tool code, as `x` is in the second example. Prompt text, tool results,
images and other context blocks can still reach the model. System 1 input states use a separate
encoding route. [Execution model](execution-model.html) explains these boundaries.
`preset = "minimal"` reduces the tool/prompt surface; it does not authorize actions or remove
required safety checks.
`{identifier}` in a literal prompt can insert a short atomic value when that is sufficient.
For a long workflow, continue one session, keep intermediate data in R and request concise
reports between stages. Use a new session or an explicit fork when a separate analysis is
intended.