| Title: | AI Token Prices, Context Windows and Model Sizes |
|---|---|
| Description: | Provides five documented data frames for analysing artificial intelligence (AI) model application programming interface (API) prices, published context windows, and published or estimated parameter counts. Fixed October 2026 snapshots retain source URLs and qualifications and add typed dates and numeric quantities for reproducible comparisons and teaching. Data are from Pistachio Technology Inc. (2026), published at <https://tokenusage.dev/publications>: API prices <doi:10.5281/zenodo.23552904>, context windows <doi:10.5281/zenodo.23552975>, and model sizes <doi:10.5281/zenodo.23553072>. These are selected historical records; estimated sizes are distinguished from lab disclosures. |
| Authors: | Jack He [aut, cre], Pistachio Technology Inc. [cph] |
| Maintainer: | Jack He <[email protected]> |
| License: | CC BY 4.0 |
| Version: | 0.1.0 |
| Built: | 2026-11-19 22:29:16 UTC |
| Source: | https://github.com/cran/tokenusagedata |
Five documented historical data frames from tokenusage.dev research notes 002, 003 and 004, with source text, qualifications, typed dates and nominal numeric quantities.
The package contains token_prices,
token_prices_premium, context_windows,
model_sizes and model_size_estimates.
They derive from five CSV files in three fixed Zenodo version records.
The October 2026 snapshots are selected historical records, not a
complete market census or a live service.
Data are available offline. All tables are base R data frames with automatic row names. Empty CSV fields become missing character values. Original text retains its UTF-8 characters.
Dates use a leading YYYY-MM-DD when present; partial dates remain missing, and date qualifications are retained in the tables that have qualified date fields. Nominal context and parameter counts come from the first figure in the corresponding text label. Commas and an optional leading tilde are removed; k, M, B and T denote powers 3, 6, 9 and 12 of ten. This extraction does not reconcile conflicts, measure effective context, or establish the accuracy of an estimate.
Prices are US dollars per one million tokens. Published unrounded blends weight input and output 3:1. Context units and access conditions differ across models. Published parameter counts and attributed closed-model estimates remain separate. Consult each table's help before making comparisons.
Only the five identified CSV files are included. Excluded source rows, other downloads, article text, charts and images are not package data.
The compilation is credited to tokenusage.dev and was deposited by Pistachio Technology Inc. The data and this package are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. Reusers must credit tokenusage.dev, link the relevant note and licence, and identify changes. This package renames columns, converts dates and prices, adds documented nominal numeric and lab fields, and converts empty fields to missing values while preserving the source text.
Third-party source material keeps its own rights. Epoch AI's results
retain their own CC BY 4.0 licence and credit to Epoch AI and the
authors it names. The model-size help pages identify the relevant
credits. Use citation("tokenusagedata") for the data records.
Jack He, package author and maintainer.
Pistachio Technology Inc., data-record creator and compilation copyright holder.
https://tokenusage.dev/publications
data("token_prices", package = "tokenusagedata") c(records = nrow(token_prices), model_lines = length(unique(token_prices$line)))data("token_prices", package = "tokenusagedata") c(records = nrow(token_prices), model_lines = length(unique(token_prices$line)))
Selected published context-window records from tokenusage.dev research note 003, retaining access restrictions, units and qualifications.
context_windowscontext_windows
A data frame with 25 rows and 11 columns:
Character. Verbatim date or observation field, including any date qualification.
Date. The leading YYYY-MM-DD in date_text,
parsed with as.Date; missing when the field does not start
with a full date. The month-only MPT record is missing.
Character. Verbatim model field, including the lab in parentheses.
Character. Text inside the last parenthesised group of
model, without the parentheses. No lab names are normalized.
Character. Verbatim published window label.
Numeric. Nominal token count extracted from the
first figure in window_text. Commas and a leading tilde are
removed; suffixes k, M, B and T mean powers 3, 6, 9 and 12 of ten.
A label with no figure would give a missing value.
Character. Verbatim access description.
Character. Text before the first spaced middle
dot in access: Served, Limited or
On paper.
Character. Verbatim explanation of the unit, access or other limitation.
Character. Published source labels in order,
separated by | .
Character. Published source URLs in order, separated by spaces; archive targets and timestamps are retained.
The source is the October 2026 snapshot, researched through October 13. This is a selection of published records, not a continuous daily maximum. Training sequence lengths, input limits and input-plus-output totals remain different quantities. Tokenizers also differ, so equal token counts do not establish equal text capacity or effective recall. A served label includes public service or trained open weights; it does not assert unrestricted access to every hosted endpoint.
The numeric column is a reproducible extraction from the CSV label,
not an independently measured effective limit. In particular,
65k becomes 65000; Scout's 10M becomes 10000000 rather
than its configuration integer 10485760; and DeepSeek's 1M
becomes 1000000 rather than 1048576. A label describing a reduction
from 2000000 to 1000000 gives 2000000, its first figure. Always read
window_text, access and qualification alongside it.
Dates retain their published basis. GPT-3's derived date is its paper date, not its later API date. No day is invented for MPT. Grok 4.20's reduction is bounded by archive observations, and Grok 4.5's date retains its API-date qualification.
The CSV excludes the OpenRouter hosted listing for Llama 4 Scout and the LWM-Text-1M row containing a RULER result. The RULER comparison table is not included. The source selection also excludes research-only 10M and 100M claims, Pokee-Isaac's unreproduced claim, Mistral Large 4's unresolved window and early Kimi figures stated in Chinese characters. Excluded records are not reconstructed from the article or chart.
The source compilation is credited to tokenusage.dev and was deposited by Pistachio Technology Inc. It and this derived table are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. Reusers must credit tokenusage.dev, link the note and licence, and identify changes. This package renames columns, adds the derived date, lab, nominal token count and availability, and converts empty fields to missing values. The original text is retained. Third-party source material keeps its own rights.
https://tokenusage.dev/publications/oct-26-context-windows
Source file: oct-26-context-windows.csv,
version 2026-10 snapshot.
data("context_windows", package = "tokenusagedata") table(context_windows$availability) subset(context_windows, is.na(date), select = c(model, date_text, window_text))data("context_windows", package = "tokenusagedata") table(context_windows$availability) subset(context_windows, is.na(date), select = c(model, date_text, window_text))
Attributed third-party estimates and claims for closed models from tokenusage.dev research note 004. These are estimates, not lab disclosures.
model_size_estimatesmodel_size_estimates
A data frame with nine rows and ten columns:
Character. Verbatim model-date field, including grouped dates or other qualifications.
Date. The leading YYYY-MM-DD in date_text,
parsed with as.Date; missing if no full date leads the field.
A field with two dates contributes only its leading date.
Character. Verbatim model label, including grouped models.
Character. Verbatim claimed or inferred total label, retaining approximation and evidence-type text.
Numeric. Nominal total parameters extracted from
the first figure in total_text. Commas and a leading tilde are
removed; suffixes k, M, B and T mean powers 3, 6, 9 and 12 of ten.
A label with no figure gives a missing value.
Character. Verbatim active-per-token label, retaining any assumption.
Numeric. Parameters active per token, extracted
from active_text using the same first-figure rule.
Unknown becomes a missing value.
Character. Verbatim evidence and uncertainty, including confidence, prediction bands and alternative estimates.
Character. Published source labels in order,
separated by | .
Character. Published source URLs in order, separated by spaces.
This is the closed-model selection in the October 2026 snapshot, researched through October 12. The source CSV omits the GPT-4 estimate derived from SemiAnalysis. It must not be restored from the note, another source or a chart.
The numeric columns are label extractions, not measurements or confidence bounds. The PaLM 2 active count assumes a dense model. Serving-economics estimates, knowledge-probe estimates, industry estimates and a CEO claim retain their different evidence types. The Grok conference transcription remains unverified. Its grouped row is not split into separately dated observations.
The source research also omits the withdrawn gpt-3.5-turbo figure, an unchecked o1 relay, unreleased Olympus and MAI-1 training targets, snippet-only GPT-5.6 Sol and GPT-6 Astra estimates, and models without usable totals. These estimates are not part of the note's disclosed model-size chart. They do not establish equivalent capability, measured inference speed or measured task cost.
The source compilation is credited to tokenusage.dev and was deposited by Pistachio Technology Inc. It and this derived table are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. Reusers must credit tokenusage.dev, link the note and licence, and identify changes. This package renames columns, adds typed dates and nominal numeric counts, and converts empty fields to missing values. Original labels, uncertainty and source fields are retained.
Third-party source material keeps its own rights. Epoch AI's results retain their own CC BY 4.0 licence and credit to Epoch AI and the authors it names. In particular, credit Epoch AI and Ege Erdil for the GPT-4o and Claude 3.5 Sonnet serving-economics estimates, and retain their source URL. Removing a tilde for numeric extraction does not remove the estimate's uncertainty.
https://tokenusage.dev/publications/oct-26-model-sizes
Source file: oct-26-model-sizes-estimates.csv,
version 2026-10 snapshot.
data("model_size_estimates", package = "tokenusagedata") model_size_estimates[c("model", "total_text", "active_text")]data("model_size_estimates", package = "tokenusagedata") model_size_estimates[c("model", "total_text", "active_text")]
Selected published total and active-per-token parameter counts from tokenusage.dev research note 004, with status and qualifications.
model_sizesmodel_sizes
A data frame with 28 rows and 11 columns:
Character. Verbatim model-date field, including any qualification.
Date. The leading YYYY-MM-DD in date_text,
parsed with as.Date; missing if no full date leads the field.
Character. Model portion before the spaced em dash in the source Model / lab field.
Character. Lab portion after the spaced em dash in the
source Model / lab field. Joining model and lab with
that separator reconstructs the source field exactly.
Character. Verbatim published total-parameter label.
Numeric. Total parameters extracted from the first
figure in total_text. Commas and a leading tilde are removed;
suffixes k, M, B and T mean powers 3, 6, 9 and 12 of ten.
A label with no figure gives a missing value.
Character. Verbatim active-per-token label.
Numeric. Parameters active per token, extracted
from active_text using the same first-figure rule.
Unknown becomes a missing value.
Character. Verbatim Status and qualifications field, including architecture, confidence and discrepancies.
Character. Published source labels in order,
separated by | .
Character. Published source URLs in order, separated by spaces.
These 28 rows are the published-model selection in the October 2026
snapshot, researched through October 12. They are not the complete
118-model research appendix. No rows are omitted from this source CSV.
Third-party closed-model estimates are separate in
model_size_estimates.
Dates normally identify the earliest announcement naming a model. Later size disclosures, public weights and general availability remain separate qualifications. Weight status is fixed at the snapshot cutoff; an open-weight description can include a restrictive licence.
Totals retain lab headline figures. The numeric extraction does not resolve the differences recorded for DeepSeek-V3's MTP module, Kimi K2's later backbone count, Inkling's stored-weight count or Mistral Large 4's conflicting reports. Grok-1's active count remains qualified as a calculation from the published percentage. Unknown active counts remain missing. Read the numeric columns with their text labels and status.
The note's chart uses a disclosed dense-model record and five selected mixture-of-experts (MoE) examples with both total and active counts. Those MoE examples are not a size-record series. Switch-C is a qualified encoder-decoder research comparison outside the decoder-model sequence. Mistral Large 4 remains an API preview with weights pending. Parameter comparisons do not establish equivalent capability, measured inference speed or measured task cost.
The source compilation is credited to tokenusage.dev and was deposited by Pistachio Technology Inc. It and this derived table are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. Reusers must credit tokenusage.dev, link the note and licence, and identify changes. This package renames columns, splits the model/lab field losslessly, adds typed dates and nominal numeric counts, and converts empty fields to missing values. Original labels, qualifications and source fields are retained.
Third-party source material keeps its own rights. Epoch AI's results retain their own CC BY 4.0 licence and credit to Epoch AI and the authors it names. The Switch-C weight-status qualification cites Epoch AI; retain that credit and its source URL when reusing it.
https://tokenusage.dev/publications/oct-26-model-sizes
Source file: oct-26-model-sizes.csv,
version 2026-10 snapshot.
data("model_sizes", package = "tokenusagedata") known <- subset(model_sizes, !is.na(active_params)) head(data.frame( model = known$model, active_fraction = known$active_params / known$total_params ))data("model_sizes", package = "tokenusagedata") known <- subset(model_sizes, !is.na(active_params)) head(data.frame( model = known$model, active_fraction = known$active_params / known$total_params ))
Selected historical application programming interface (API) list prices from tokenusage.dev research note 002, in its October 2026 snapshot.
token_pricestoken_prices
A data frame with 88 rows and 12 columns:
Character. Selected model-line label, copied from the CSV.
Character. Model lab, copied from the CSV.
Date. The leading YYYY-MM-DD in the source date field,
parsed with as.Date; missing if no full date leads the field.
All dates in this table have a full leading date.
Character. Model represented by the line from this date, copied from the CSV.
Character. Published event label: start,
step or relabel.
Numeric. Published input price in US dollars per one million tokens, converted directly from its decimal CSV field.
Numeric. Published output price in US dollars per one million tokens, converted directly from its decimal CSV field.
Numeric. Published, unrounded 3:1 blended price in US dollars per one million tokens, converted directly from its decimal CSV field.
Character. Applicable context or prompt-length tier,
copied verbatim; an empty field becomes NA_character_.
Character. Published qualifications, copied verbatim;
an empty field becomes NA_character_.
Character. Published explanatory note, copied verbatim;
an empty field becomes NA_character_.
Character. Source URLs in their published order, separated by spaces. Archive targets and timestamps are preserved.
The snapshot covers research through October 12, 2026. It contains nine selected model lines from six labs, not every model or the cheapest available API on each day. A successor replaces the represented model within a line even when the previous model remains available. The records are ordered by line and then date.
There are 44 starts, 15 price steps and 29 relabels. A relabel records a different model at the same rate. The opening davinci record on January 1, 2022 is an opening state for an already available model, not a launch event.
Prices are for uncached requests at the lowest prompt-length tier. Batch and cache discounts, free tiers, credits and negotiated rates are excluded. The selected DeepSeek peak prices exclude off-peak discounts. Labelled introductory and promotional prices count on their recorded dates. Some historical per-character prices were converted to tokens; their conversion assumptions remain in the caveats and notes.
The blended price is (3 * input + output) / 4.
The published calculation uses integer micro-dollars before weighting.
This table preserves the unrounded result, rather than the article's
displayed cents. These are list-price comparisons, not measured task
costs or comparisons at equal model quality.
The Meta line uses Together AI's hosted Llama 70B prices. Its July 2024 step also changes the default endpoint from Reference (FP16) to Turbo (FP8). The represented line ends on April 5, 2025, exclusively; the CSV contains no end-marker row. Exclude it from cross-line calculations on or after that date. This boundary is not a retirement date for Llama 3.3 API access. Grok enters on April 9, 2025.
Inferred dates, inferred prices, preview status, secondary sources and
endpoint-specific prices remain qualified in the text. An archive
capture date is not necessarily a verified effective date. GPT-4.1 is
outside the selected OpenAI flagship sequence. Premium tiers are in
token_prices_premium. No rows are omitted from this source
CSV; narrative comparisons and article figures are not additional
package data.
The source compilation is credited to tokenusage.dev and was deposited by Pistachio Technology Inc. It and this derived table are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. Reusers must credit tokenusage.dev, link the note and licence, and identify changes. This package renames columns, converts dates and decimal prices to R types, and converts empty fields to missing values; it retains the published text and values. Third-party source material keeps its own rights.
https://tokenusage.dev/publications/oct-26-token-prices
Source file: oct-26-token-prices.csv, version 2026-10 snapshot.
data("token_prices", package = "tokenusagedata") table(token_prices$change) subset(token_prices, line == "OpenAI low-cost", select = c(date, model, blended_usd_per_mtok))data("token_prices", package = "tokenusagedata") table(token_prices$change) subset(token_prices, line == "OpenAI low-cost", select = c(date, model, blended_usd_per_mtok))