CD Request Pipeline

Scope

In Scope

This document covers the CD request pipeline provided by the CD library, meaning the path from submitting a request through to its completion. It is one of the core parts of the CD library file written by the Ape Escape developers.

Out of Scope

• Asset loading and minigame execution are covered in their own documents because they build requests instead of being part of the pipeline that fulfills them.
• How the Sony CD library works.
• How code that bypasses the pipeline controls the CD.

Overview

The CD request pipeline can be divided into 3 steps that requests flow through. Callers submit requests which are turned into a list of commands needed to complete them. The pipeline has to be advanced by either calling a blocking function or manually advancing it, typically every frame. As it advances the queued commands are executed using the Sony CD Library. The steps are described below.

Submit

The library lets callers submit the following requests:

• Seeking to the start of a file
• Streaming an STR file
• Loading a file from the CD
• Loading part of a file from the CD
• Loading the current asset file tracked by the library

In the submit phase this involves two main steps: finding where the file is on the CD and building the commands needed to fulfill the request into the buffer. These steps are handled by the request functions and a queuing function.

Request functions seem to follow a common template. These are:

• Abortion of an active stream or error recovery (not done by all).
• Address and metadata resolution
• Asking for the specific sequence of commands necessary to fulfill the request to be enqueued.
  The sequence of commands also follow the following template:
  - Configure the drive mode with a CdlSetMode.
  - Sets the target seek location with a CdlSetLoc.
  - A CdlNop (probably a delay for the hardware?).
  - The actual hardware operation (Read/Seek).

They typically figure out where files live on the CD using look up tables and caller passed indexes¹. There are two types of look up tables, one general table that contains the location and size of every file on the CD measured in sectors, and another XA streaming table whose entries contain additional metadata for file streaming.

The request functions then pass in what the enqueue function needs to enqueue commands into the command buffer. The buffer is a 16 entry circular queue with two indexes: one tracks the next command to executed, the other the next open slot and when the two are equal to each other the buffer is considered empty. Due to the buffer’s limited size the enqueue function has logic to make room for a command by executing one before enqueuing and once a command is enqueued the enqueue index is incremented.

The functions that make up this phase are listed below.

Request Functions:
cdlib_queue_seek
cdlib_queue_file_load
cdlib_queue_adpcm_play
cdlib_queue_partial_file_load
cdlib_queue_single_asset_load

Queuing Function:
cdlib_enqueue_cmd

Drive

In order to execute commands the request pipeline must be driven. This is done by calling cdlib_manager. More details about the function can be found in its function documentation but at a high level it’s a non blocking state machine. Each call takes one action, based on what the manager is trying to do represented by an IO state. In addition to driving reads or seeks, the state manager incorporates timeout, retry and error recovery logic. For normal reads it checks that the read hasn’t timed out and attempts to retry if it did, while for streams a dedicated XA manager is called to monitor progress. Actions are taken based on this progress, hardware status of the CD, and metadata in the XA streaming table. Some of the actions taken include looping, pausing, or drive recovery.

Callers can use the state machine to implement synchronous or asynchronous behavior. For synchronous behavior, the library actually provides a method called cdlib_process_until_idle that will block until the IO state equals idle. Callers can also implement their own busy wait loop based on their specific needs. For asynchronous operation the manager is called once per frame alongside other work.

Drive Functions:
cdlib_manager
cdlib_xa_manager

Execute

Execution of requests is handled by one function. Like above, a more in depth look of the function can be found in the documentation. This is the last step of the pipeline, here a single command is executed from the queue. Execution is divided into two parts:

• Executing our custom op code
• Executing all other op codes

The difference between the two is that the custom command op code branch writes to some global variables to track the progress of a normal read and registers a callback. The callback calls the registered function when the CD has loaded a sector into its buffer and the data is ready to be transferred to memory.

The other branch just executes the opcode and then goes through a switch case to decide how to update different global variables that track different aspects of the cd system based on the command op code. At the end of the case the queue exec index is incremented, meaning a command is considered completed.

Execute Function:
cdlib_execute_cmd

Callback:
cdlib_ready_callback

IO States

The library tracks 7 constants for its state machine, these are:

x00000: Idle - Used to take no action
x10000: ReadS - Used to check status of a stream
x20000: Seek - Used to record the seek head reached target location
x40000: ReadN - Used to check if read is taking too long
x50000: Ready - Used to execute a command
xE0000: Retry - Used to retry a read
xF0000: Error - Used to recover from an error

The most important thing to know about these values is that they are NOT used to track the state of the hardware. Instead they track the intent of the manager. Specifically what the manager is trying to do with the hardware. The hardware has its own condition in the moment which can be polled by CdSync to decide how to proceed based on the IO state, but the manager uses the state machine values to know what job it’s doing and what action it should take next. The documentation for cdlib_manager provides more detail on how these values are used.

Footnotes

1. Only exception is cdlib_queue_single_asset_load which uses cd library state that track the next asset to load, more detail in the Asset Loading System section. ↩