Cboe Order Book Management Guide

Cboe Order Book Management Guide.md

Cboe Order Book Management Guide

Introduction

This document provides comprehensive guidance for developers building and maintaining an accurate order book using the Cboe Australia Titanium Multicast Depth of Book (PITCH) protocol. It details the message processing rules, data structures, and implementation considerations necessary for reliable order book reconstruction and management.

Order Book Fundamentals

The Cboe order book represents the current state of all active orders in the market, organized by:

• Security symbol
• Price level
• Time priority within price level
• Side (buy/sell)

A properly implemented order book should:

1. Maintain the full depth of orders for each symbol
2. Process updates in the correct sequence
3. Support rapid lookup by Order ID
4. Handle special order types properly
5. Recover gracefully from data gaps or disconnections

Data Structures for Order Book Management

Order Entry

Each order in the book should contain:

struct Order {
    uint64_t orderId;           // Unique identifier for the order
    char symbol[6];             // Security symbol
    char side;                  // 'B' for buy, 'S' for sell
    uint32_t quantity;          // Current quantity
    uint64_t price;             // Price in Cboe binary price format
    char participantId[4];      // PID (if attributed)
    uint64_t timestamp;         // Entry/update timestamp
    bool isUndisclosed;         // True for undisclosed orders
}

Price Level

Price levels aggregate orders at the same price:

struct PriceLevel {
    uint64_t price;             // Price point
    uint32_t totalQuantity;     // Sum of all order quantities at this price
    vector<Order*> orders;      // Orders at this price in time priority
}

Symbol Book

Each symbol has its own book of buy and sell orders:

struct SymbolBook {
    char symbol[6];             // Security symbol
    map<uint64_t, PriceLevel> bids;  // Buy orders by price (descending)
    map<uint64_t, PriceLevel> asks;  // Sell orders by price (ascending)
    map<uint64_t, Order*> ordersById; // Fast lookup by Order ID
    char tradingStatus;         // Current trading status
}

Message Processing Rules

The order book must process PITCH messages according to strict rules to maintain accuracy. Below are the detailed processing steps for each relevant message type.

Initial Setup

1. Start with an empty order book at the beginning of each trading day
2. Process Unit Clear messages if received (rare)
3. Track Trading Status for each symbol

Add Order (0x37)

When an Add Order message is received:

1. Create a new Order object with the following fields:

   • Order ID from the message
   • Symbol from the message
   • Side from the message
   • Quantity from the message (zero for undisclosed orders)
   • Price from the message
   • PID from the message (if present)
   • Timestamp from the message

2. Add the order to the symbol's ordersById map for fast lookup

3. If the order is not undisclosed (quantity > 0):

   • Find or create the appropriate price level in bids or asks (based on side)
   • Add the order to the end of that price level's orders list (preserving time priority)
   • Update the total quantity at that price level

4. Special handling for undisclosed orders:

   • If quantity is zero, flag the order as undisclosed
   • Store the order in ordersById but do not add to visible price levels

Example:

void processAddOrder(AddOrderMessage msg) {
    Order* order = new Order();
    order->orderId = msg.orderId;
    order->symbol = msg.symbol;
    order->side = msg.side;
    order->quantity = msg.quantity;
    order->price = msg.price;
    order->participantId = msg.pid;
    order->timestamp = msg.timestamp;
    order->isUndisclosed = (msg.quantity == 0);
    
    SymbolBook* book = getBookForSymbol(msg.symbol);
    book->ordersById[order->orderId] = order;
    
    if (!order->isUndisclosed) {
        PriceLevel* level = getPriceLevel(book, order->side, order->price);
        level->orders.push_back(order);
        level->totalQuantity += order->quantity;
    }
}

Order Executed (0x38)

When an Order Executed message is received:

1. Retrieve the order from ordersById using the Order ID
2. Subtract the executed quantity from the order's current quantity
3. Subtract the executed quantity from the price level's total quantity
4. If the order's remaining quantity is zero:
   • Remove the order from its price level
   • Remove the order from ordersById
   • Delete the price level if it's now empty

Example:

void processOrderExecuted(OrderExecutedMessage msg) {
    Order* order = findOrderById(msg.orderId);
    if (!order) return; // Order not found
    
    uint32_t executedQty = msg.executedQuantity;
    PriceLevel* level = getPriceLevel(book, order->side, order->price);
    
    order->quantity -= executedQty;
    level->totalQuantity -= executedQty;
    
    if (order->quantity == 0) {
        removeOrderFromPriceLevel(level, order);
        book->ordersById.erase(order->orderId);
        delete order;
        
        if (level->orders.empty()) {
            removePriceLevel(book, order->side, order->price);
        }
    }
}

Order Executed at Price (0x58)

When an Order Executed at Price message is received (for auctions):

1. Retrieve the order from ordersById using the Order ID
2. Subtract the executed quantity from the order's current quantity
3. Subtract the executed quantity from the price level's total quantity
4. If the order's remaining quantity is zero:
   • Remove the order from its price level
   • Remove the order from ordersById
   • Delete the price level if it's now empty
5. Note: The execution price in the message may differ from the order's original price

Reduce Size (0x39)

When a Reduce Size message is received:

1. Retrieve the order from ordersById using the Order ID
2. Subtract the cancelled quantity from the order's current quantity
3. Subtract the cancelled quantity from the price level's total quantity
4. If the order's remaining quantity is zero:
   • Remove the order from its price level
   • Remove the order from ordersById
   • Delete the price level if it's now empty

Example:

void processReduceSize(ReduceSizeMessage msg) {
    Order* order = findOrderById(msg.orderId);
    if (!order) return; // Order not found
    
    uint32_t cancelledQty = msg.cancelledQuantity;
    PriceLevel* level = getPriceLevel(book, order->side, order->price);
    
    order->quantity -= cancelledQty;
    level->totalQuantity -= cancelledQty;
    
    if (order->quantity == 0) {
        removeOrderFromPriceLevel(level, order);
        book->ordersById.erase(order->orderId);
        delete order;
        
        if (level->orders.empty()) {
            removePriceLevel(book, order->side, order->price);
        }
    }
}

Modify Order (0x3A)

When a Modify Order message is received:

1. Retrieve the order from ordersById using the Order ID
2. If the price has changed:
   • Remove the order from its current price level
   • Subtract the old quantity from that price level's total quantity
   • Find or create the new price level
   • Add the order to the end of the new price level's orders list (losing time priority)
   • Add the new quantity to the new price level's total quantity
3. If only the quantity has changed:
   • Calculate the difference between old and new quantities
   • Update the price level's total quantity accordingly
   • Update the order's quantity
4. Delete any empty price levels

Example:

void processModifyOrder(ModifyOrderMessage msg) {
    Order* order = findOrderById(msg.orderId);
    if (!order) return; // Order not found
    
    PriceLevel* oldLevel = getPriceLevel(book, order->side, order->price);
    
    // If price changed, remove from old level and add to new
    if (order->price != msg.price) {
        removeOrderFromPriceLevel(oldLevel, order);
        oldLevel->totalQuantity -= order->quantity;
        
        order->price = msg.price;
        order->quantity = msg.quantity;
        order->timestamp = msg.timestamp; // Reset timestamp (lose priority)
        
        PriceLevel* newLevel = getPriceLevel(book, order->side, order->price);
        newLevel->orders.push_back(order);
        newLevel->totalQuantity += order->quantity;
        
        if (oldLevel->orders.empty()) {
            removePriceLevel(book, order->side, order->oldPrice);
        }
    }
    // If only quantity changed
    else {
        int32_t qtyDiff = msg.quantity - order->quantity;
        oldLevel->totalQuantity += qtyDiff;
        order->quantity = msg.quantity;
    }
}

Delete Order (0x3C)

When a Delete Order message is received:

1. Retrieve the order from ordersById using the Order ID
2. Remove the order from its price level
3. Subtract the order's quantity from the price level's total quantity
4. Remove the order from ordersById
5. Delete the price level if it's now empty

Example:

void processDeleteOrder(DeleteOrderMessage msg) {
    Order* order = findOrderById(msg.orderId);
    if (!order) return; // Order not found
    
    if (!order->isUndisclosed) {
        PriceLevel* level = getPriceLevel(book, order->side, order->price);
        level->totalQuantity -= order->quantity;
        removeOrderFromPriceLevel(level, order);
        
        if (level->orders.empty()) {
            removePriceLevel(book, order->side, order->price);
        }
    }
    
    book->ordersById.erase(order->orderId);
    delete order;
}

Trade (0x3D)

For on-exchange executions of undisclosed orders:

1. Retrieve the order from ordersById using the Order ID
2. Update the order's quantity (subtracting the executed amount)
3. If the order's remaining quantity is zero:
   • Remove the order from ordersById
   • Delete the order

For off-exchange trades, no order book update is needed.

Trading Status (0x3B)

When a Trading Status message is received:

1. Update the trading status for the symbol
2. Handle special status transitions:
   • When transitioning to "C" (Closed), consider clearing all orders for that symbol
   • When transitioning to "H" (Halted) or "S" (Suspended), flag the symbol accordingly
   • When transitioning to "O" (Pre-Open) or "E" (Pre-Close), prepare for auction processing

Example:

void processTradingStatus(TradingStatusMessage msg) {
    SymbolBook* book = getBookForSymbol(msg.symbol);
    book->tradingStatus = msg.tradingStatus;
    
    // Special handling for status changes
    if (msg.tradingStatus == 'C') {
        // Optional: Clear all orders at end of day
    }
}

Unit Clear (0x97)

When a Unit Clear message is received:

1. Clear all orders for all symbols in the specified unit
2. Reset all price levels and order maps

Example:

void processUnitClear(UnitClearMessage msg, int unitId) {
    vector<SymbolBook*> booksInUnit = getBooksForUnit(unitId);
    
    for (SymbolBook* book : booksInUnit) {
        // Delete all orders
        for (auto& pair : book->ordersById) {
            delete pair.second;
        }
        
        // Clear all collections
        book->bids.clear();
        book->asks.clear();
        book->ordersById.clear();
    }
}

Special Order Types Handling

Undisclosed Orders

Undisclosed orders require special handling:

1. Add to ordersById but not to visible price levels
2. Process executions via Trade messages rather than Order Executed
3. Track them separately from visible orders
4. Remove them when a Delete Order is received or when fully executed

Iceberg Orders

Iceberg orders appear as:

1. Initial Add Order with visible quantity
2. Order Executed messages for visible portion executions
3. Trade messages for hidden portion executions
4. When visible portion is exhausted, a new Add Order with a new Order ID

Example sequence:

1. Add Order: ID=100, Side=B, Qty=50, Symbol=ABC, Price=10.00
2. Order Executed: ID=100, Qty=50 (visible portion exhausted)
3. Trade: Symbol=ABC, Qty=50, Price=10.00 (hidden portion execution)
4. Add Order: ID=101, Side=B, Qty=25, Symbol=ABC, Price=10.00 (replenishment)

Multi-Day Orders

Multi-day orders (GTC/GTD) will:

1. Appear as new Add Order messages each morning
2. Maintain time priority within price levels
3. May be purged overnight due to corporate actions

Auction Processing

Order Book During Auctions

During auction periods:

1. Trading Status changes to "O" (Pre-Open) or "E" (Pre-Close)
2. Auction Update messages show indicative price/volume
3. Order books may overlap (bid price ≥ ask price)
4. Process all orders normally during auction call periods

Auction Execution

When an auction executes:

1. Process Order Executed at Price messages for visible orders
2. Process Trade messages for undisclosed orders
3. Process Reduce Size or Delete Order for affected orders
4. Process Auction Summary for final statistics

Example auction sequence:

1. Trading Status: Symbol=ABC, Status=O (Pre-Open)
2. Auction Update: Symbol=ABC, Type=O, BuyShares=1000, SellShares=800, Price=10.00
3. [Multiple order updates during pre-auction period]
4. Order Executed at Price: OrderID=100, Qty=500, Price=10.00
5. Trade: Symbol=ABC, Qty=300, Price=10.00, Type=O (auction trade for undisclosed order)
6. Auction Summary: Symbol=ABC, Type=O, Price=10.00, Shares=800
7. Trading Status: Symbol=ABC, Status=T (Trading)

Order Book Integrity and Recovery

Sequence Gap Detection

Monitor sequence numbers to detect gaps:

1. Track the last sequence number received for each unit
2. Verify each new message has the expected next sequence
3. If a gap is detected, buffer subsequent messages

Example:

void processMessage(PitchMessage msg) {
    if (msg.sequence != expectedSequence[msg.unit]) {
        if (msg.sequence > expectedSequence[msg.unit]) {
            // Gap detected
            bufferMessage(msg);
            requestGap(msg.unit, expectedSequence[msg.unit], 
                      msg.sequence - expectedSequence[msg.unit]);
            return;
        }
        else {
            // Duplicate or out-of-order message
            return;
        }
    }
    
    // Process message normally
    dispatchMessage(msg);
    expectedSequence[msg.unit]++;
    
    // Process any buffered messages that are now in sequence
    processBufferedMessages(msg.unit);
}

Gap Recovery

When a gap is detected:

1. Request missing messages via Gap Request Proxy
2. Apply recovered messages in sequence
3. Process buffered messages once recovery is complete

For large gaps, use the Spin Server:

1. Request a spin of the current order book
2. Apply the spin data to establish your base order book
3. Apply buffered messages once the spin is complete

Intraday Recovery

To recover the order book intraday:

1. Connect to the Spin Server
2. Request a spin using the last known sequence number
3. Buffer all real-time messages during the spin
4. Clear existing order book data
5. Apply spin data to rebuild the book
6. Apply buffered messages in sequence

Performance Considerations

Memory Efficiency

Optimize memory usage:

1. Pre-allocate memory for high-volume sessions
2. Use appropriate data structures (hash maps for Order ID lookups)
3. Properly clean up deleted orders to prevent memory leaks

Computational Efficiency

Optimize processing:

1. Use hash tables for constant-time Order ID lookups
2. Use balanced trees (e.g., Red-Black trees) for price level organization
3. Implement efficient price level management
4. Optimize gap detection and recovery

Feed Arbitration

When using multiple feeds (A and B):

1. Process messages from either feed based on sequence number
2. Discard duplicate messages
3. Use the feed that delivers the next expected sequence first

Debugging and Verification

Order Book Validation

Validate order book integrity:

1. Sum of quantities at each price level should match individual orders
2. Order counts should match across all data structures
3. Order IDs should be unique
4. Price levels should be properly sorted (bids descending, asks ascending)

Example validation:

void validateOrderBook(SymbolBook* book) {
    // Validate bid price levels
    for (auto& bidPair : book->bids) {
        PriceLevel& level = bidPair.second;
        uint32_t sumQty = 0;
        
        for (Order* order : level.orders) {
            sumQty += order->quantity;
            assert(order->price == level.price);
            assert(order->side == 'B');
        }
        
        assert(sumQty == level.totalQuantity);
    }
    
    // Similar validation for asks
    // ...
}

State Logging

Implement logging for troubleshooting:

1. Log all incoming messages with sequence numbers
2. Log order book state changes
3. Log gap detection and recovery events
4. Log error conditions and unexpected messages

Appendix: Data Type Conversion

Binary Price Conversion

Convert binary price to decimal:

double convertPrice(uint64_t binaryPrice) {
    return binaryPrice / 10000000.0;
}

Timestamp Conversion

Convert binary UTC timestamp to readable form:

string convertTimestamp(uint64_t timestamp) {
    time_t seconds = timestamp / 1000000000;
    uint32_t nanoseconds = timestamp % 1000000000;
    
    struct tm* timeinfo = gmtime(&seconds);
    char buffer[30];
    strftime(buffer, 30, "%Y-%m-%d %H:%M:%S", timeinfo);
    
    char result[40];
    sprintf(result, "%s.%09u", buffer, nanoseconds);
    return string(result);
}

Appendix: Market State Management

Market Phase Transitions

Track market phases for each symbol:

1. Closed → Pre-market
2. Pre-market → Pre-Open (auction call)
3. Pre-Open → Trading (after opening auction)
4. Trading → Pre-Close (auction call)
5. Pre-Close → Post-market (after closing auction)
6. Post-market → Closed

Example state machine:

void updateMarketPhase(SymbolBook* book, char newStatus) {
    char oldStatus = book->tradingStatus;
    book->tradingStatus = newStatus;
    
    if (oldStatus == 'O' && newStatus == 'T') {
        // Transition from Pre-Open to Trading
        // Opening auction completed
        logMarketPhaseChange(book->symbol, "Opening auction completed");
    }
    else if (oldStatus == 'E' && newStatus == 'P') {
        // Transition from Pre-Close to Post-market
        // Closing auction completed
        logMarketPhaseChange(book->symbol, "Closing auction completed");
    }
    // Handle other transitions
}

Appendix: Sample Order Book Output

Example of order book display:

Symbol: ZVZT  Status: T (Trading)
Time: 2025-03-21 10:15:30.123456789

BID                          ASK
Price    | Quantity | Orders  Price    | Quantity | Orders
10.0500  | 1,500    | 3       10.0600  | 2,000    | 2
10.0400  | 3,200    | 5       10.0700  | 1,500    | 3
10.0300  | 5,000    | 4       10.0800  | 3,000    | 1
10.0200  | 2,300    | 2       10.0900  | 4,200    | 4
10.0100  | 1,800    | 3       10.1000  | 2,500    | 2

Market Depth: Bid 5 levels, Ask 5 levels
Total Bid Volume: 13,800
Total Ask Volume: 13,200