Parsing JPEG Markers

JPEG Overview

A JPEG file is a collection of markers which define blocks of data in the file. See Wikipedia to get an overview of the file format.

This vignette simply parses out all the headers and prints a description of each one.

JPEG markers

There is an included table of jpeg_markers in this vignette, but only the first 15 rows are displayed here.

This data consists of:

  • Hex the hexadecimal code that is the maker
  • Marker the codename for this marker
  • Name/Description more information about this marker
Hex Marker Name Description
ffc0 SOF0 Start of Frame 0 Baseline DCT
ffc1 SOF1 Start of Frame 1 Extended Sequential DCT
ffc2 SOF2 Start of Frame 2 Progressive DCT
ffc3 SOF3 Start of Frame 3 Lossless (sequential)
ffc4 DHT Define Huffman Table NA
ffc5 SOF5 Start of Frame 5 Differential sequential DCT
ffc6 SOF6 Start of Frame 6 Differential progressive DCT
ffc7 SOF7 Start of Frame 7 Differential lossless (sequential)
ffc8 JPG JPEG Extensions NA
ffc9 SOF9 Start of Frame 9 Extended sequential DCT, Arithmetic coding
ffca SOF10 Start of Frame 10 Progressive DCT, Arithmetic coding
ffcb SOF11 Start of Frame 11 Lossless (sequential), Arithmetic coding
ffcc DAC Define Arithmetic Coding NA
ffcd SOF13 Start of Frame 13 Differential sequential DCT, Arithmetic coding
ffce SOF14 Start of Frame 14 Differential progressive DCT, Arithmetic coding

Example JPEG file

library(ctypesio)

jpeg_file <- system.file("img", "Rlogo.jpg", package="jpeg")
jpeg <- jpeg::readJPEG(jpeg_file)
plot(as.raster(jpeg))

What does the binary data in the JPEG look like?

dim(jpeg)
#> [1]  76 100   3
dat <- readBin(jpeg_file, raw(), n = file.size(jpeg_file)) 
head(dat, 100)
#>   [1] ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 01 2c 01 2c 00 00 ff e1 00 80 45
#>  [26] 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 05 01 12 00 03 00 00 00 01 00 01
#>  [51] 00 00 01 1a 00 05 00 00 00 01 00 00 00 4a 01 1b 00 05 00 00 00 01 00 00 00
#>  [76] 52 01 28 00 03 00 00 00 01 00 02 00 00 87 69 00 04 00 00 00 01 00 00 00 5a

Process all chcunks

A JPEG file is just a sequence of chunks. A chunk consists of

  • A marker e.g. ffe0
  • A length (unsigned 16 bit integer in big-endian)
  • Data bytes to match the length

The following code first asserts that the JPEG file starts with the “Start of Image” marker ffd8, then:

  1. Reads the chunk marker (2-byte hex)
  2. Reads the chunk length (unsigned, 16bit integer)
  3. Reads the data for this chunk
  4. Repeats from Step (1) until the “Start of Scan” marker (ffda) is found.
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Open a connection, and tag the connection such that 
# values are read in **big endian** by default.
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
con <- file(jpeg_file, 'rb') |>
  set_endian('big')

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Read the first 2 bytes as HEX
# For regular JPEG files, this should be the "Start of Image (SOI)" marker
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
soi <- read_hex(con, n = 1, size = 2) # ffd8: SOI
stopifnot(soi == 'ffd8')

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Keep reading markers and the chunk data until we reach
# the 'Start of Scan' marker (ffda)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
marker <- read_hex(con, n = 1, size = 2)

while(length(marker) > 0 && nchar(marker) > 0) {
  
  # The relevant row from the 'jpeg_markers' data.frame
  info <- subset(jpeg_markers, jpeg_markers$Hex == marker)
  
  # It's possible there may be custom markers which aren't included
  # in my list of markers
  if (nrow(info) == 0) {
    cat("Unknown marker: ", marker, "\n")
    marker <- read_hex(con, n = 1, size = 2)
    next
  }
  
  # Read the length of data in this chunk and output the chunk info
  len <- read_uint16(con)
  msg <- sprintf("%s [%5i] [%s] [%s] [%s]\n", marker, len, info$Marker, info$Name, info$Description)
  cat(msg)
  
  # Check if we've reached the Start of Scan marker
  if (marker == 'ffda') {
    cat("Compressed image data until end of file\n")
    break
  }
  
  # Read the chunk data 
  # In JPEG the length of each chunk includes the 2 bytes which specify 
  # the chunk length, so read len-2 bytes from the current position
  chunk_data <- read_uint8(con, n = len - 2)
  
  # Process chunk data here
  
  
  # Read the next marker and continue
  marker <- read_hex(con, n = 1, size = 2)
}
#> ffe0 [   16] [APP0] [Application Segment 0] [JFIF – JFIF JPEG image AVI1 – Motion JPEG (MJPG)]
#> ffe1 [  128] [APP1] [Application Segment 1] [EXIF Metadata, TIFF IFD format, JPEG Thumbnail (160×120) Adobe XMP]
#> ffdb [   67] [DQT] [Define Quantization Table] [NA]
#> ffdb [   67] [DQT] [Define Quantization Table] [NA]
#> ffc0 [   17] [SOF0] [Start of Frame 0] [Baseline DCT]
#> ffc4 [   31] [DHT] [Define Huffman Table] [NA]
#> ffc4 [  181] [DHT] [Define Huffman Table] [NA]
#> ffc4 [   31] [DHT] [Define Huffman Table] [NA]
#> ffc4 [  181] [DHT] [Define Huffman Table] [NA]
#> ffda [   12] [SOS] [Start of Scan] [NA]
#> Compressed image data until end of file
close(con)