Unsupported opcode 5A (PHY) error during InitZobristTables execution

[barryw]

Description

When running tests that call InitZobristTables (which fills 1536+ bytes of random data using a PRNG loop), the simulator reports an unsupported opcode error:

The OpCode 5A @ address 44F6 is not supported.

Steps to Reproduce

1. Build the Chess project: make build

2. Run any test that calls InitZobristTables, e.g.:

   docker run --rm -v "$PWD":/code ghcr.io/barryw/sim6502:v3.6.0 -s /code/tests/ai_zobrist.6502
   

3. The first suite (Zobrist PRNG) passes, but when Zobrist Tables suite runs and calls InitZobristTables, the error occurs.

Analysis

• Opcode $5A is PHY (Push Y to stack) - a 65C02 instruction
• The code at address $44F6 in the symbol file is inside ZobristPRNG
• The actual assembled code at that location is valid 6502 (lda $fb = $A5 $FB)
• Tests that don't call InitZobristTables work fine
• Tests that call only ZobristPRNG (without the full table init loop) also work

This suggests the issue may be related to:

1. Execution reaching an unexpected memory location after many iterations
2. Some state corruption during long-running loops
3. Address calculation issue in the simulator

Expected Behavior

InitZobristTables should complete successfully - it's standard 6502 code with no 65C02 instructions.

Environment

• sim6502 version: v3.6.0
• Platform: macOS (ARM64, running via Docker with linux/amd64 emulation)
• Source code: Standard 6502 assembly (no 65C02 mode)

Workaround

Tests that need Zobrist functionality can use ZobristPRNG directly or work with pre-seeded values rather than calling the full InitZobristTables.

[barryw]

Initial Analysis

Thanks for the detailed report. I've investigated and here's what I've found:

Key Observations

1. Opcode $5A: On the 6502, this is an illegal/undocumented opcode. On 65C02, it's PHY. In 6502 mode, encountering $5A should indeed throw an unsupported opcode error.

2. The Real Question: Why is the PC reaching memory containing $5A when the assembled code at $44F6 should be lda $fb ($A5 $FB)?

Possible Causes

Since this only happens during the long-running InitZobristTables loop (1536+ iterations), potential causes include:

1. Memory Corruption: The loop or PRNG code may be inadvertently writing to address $44F6
2. Index Overflow: If using indexed addressing with X or Y registers, an overflow could cause writes to unexpected locations
3. Stack Issues: Deep nesting or stack corruption could cause incorrect return addresses

Debugging Suggestions

1. Add trace output to see the execution path before the error:

   test("debug-zobrist", "Trace execution") {
     trace = true
     // ... your test setup
   }
   

2. Check memory at $44F6 before and during execution:

   assert(peekbyte($44F6) == $A5, "Code should be LDA opcode")
   

3. Watch for writes to code region: If your PRNG or table-filling code uses indexed addressing, verify the index bounds don't overflow into the code region.

Upgrade Recommendation

I also recommend trying the latest version (just pushed) which includes multi-processor support and additional test coverage. The core execution hasn't changed, but it may help isolate whether this is a simulator issue or something in the test code.

Would you be able to share the relevant assembly source for InitZobristTables and ZobristPRNG? This would help identify if there's an addressing calculation that could overflow.

[barryw]

Thanks for looking into this! Here's the complete source code for the Zobrist functions:

ai/zobrist.asm

#importonce

// Zobrist Hashing for Position Identification
// Uses xorshift32 PRNG seeded at startup

*=* "AI Zobrist"

//
// 16-bit Galois LFSR PRNG with improved mixing
// State stored in $fb-$fc (zero page for speed)
// Returns 8-bit result in A, advances state
//
// Uses polynomial x^16 + x^14 + x^13 + x^11 + 1
// Period: 65535 (2^16 - 1)
// Taps: bits 16, 14, 13, 11 -> feedback mask $B400
//
// Runs 8 LFSR cycles per byte for better randomness
//
ZobristPRNG:
  ldx #$08          // 8 iterations per output byte

!prng_loop:
  // Galois LFSR step: shift right, XOR if LSB was 1
  lda $fb
  lsr               // Shift LSB into carry
  ror $fc           // Rotate high byte right through carry
  ror $fb           // Rotate low byte right through carry

  bcc !no_xor+      // If carry clear (LSB was 0), skip XOR
  // XOR with feedback polynomial $B400
  lda $fc
  eor #$B4
  sta $fc
!no_xor:

  dex
  bne !prng_loop-

  // Return low byte XOR high byte for better distribution
  lda $fb
  eor $fc

  // Store last result for testing
  sta ZobristLastRandom

  rts

//
// Seed the PRNG with a fixed non-zero value
// Call once at startup
// IMPORTANT: LFSR must never be all zeros!
//
ZobristSeed:
  lda #$CE
  sta $fb
  lda #$A7
  sta $fc
  rts

// Storage for last random value (for testing)
ZobristLastRandom:
  .byte $00

//
// Initialize all Zobrist tables with random values
// Call once at game startup
// Uses $f9/$fa as pointer (temp)
//
InitZobristTables:
  // Seed PRNG
  jsr ZobristSeed

  // Fill piece-square table (1536 bytes = 12 pieces x 64 squares x 2 bytes)
  lda #<ZobristPieces
  sta $f9
  lda #>ZobristPieces
  sta $fa

  ldx #$00        // Page counter (6 pages = 1536 bytes)
  ldy #$00

!fillpieces:
  jsr ZobristPRNG
  sta ($f9), y
  iny
  bne !fillpieces-

  // Next page
  inc $fa
  inx
  cpx #$06        // 6 pages for pieces (1536 bytes)
  bne !fillpieces-

  // Fill side to move (2 bytes)
  jsr ZobristPRNG
  sta ZobristSide
  jsr ZobristPRNG
  sta ZobristSide + 1

  // Fill castling rights (8 bytes = 4 flags x 2 bytes)
  ldy #$00
!castloop:
  jsr ZobristPRNG
  sta ZobristCastling, y
  iny
  cpy #$08
  bne !castloop-

  // Fill en passant files (16 bytes = 8 files x 2 bytes)
  ldy #$00
!eploop:
  jsr ZobristPRNG
  sta ZobristEnPassant, y
  iny
  cpy #$10
  bne !eploop-

  rts

*=* "Zobrist Tables"

// Piece-square table: 12 pieces x 64 squares x 2 bytes = 1536 bytes
ZobristPieces:
  .fill 1536, $00

// Side to move (2 bytes)
ZobristSide:
  .word $0000

// Castling rights: 4 flags x 2 bytes = 8 bytes
ZobristCastling:
  .fill 8, $00

// En passant file: 8 files x 2 bytes = 16 bytes
ZobristEnPassant:
  .fill 16, $00

// Current position hash (2 bytes for better collision resistance)
ZobristHash:
  .word $0000

Memory Layout from Build

According to the build output, the Zobrist code is placed at:

• $45b8-$4622 AI Zobrist (the functions)
• $4623-$4d63 Zobrist Tables (the data)

The error message says the problematic opcode is at $44F6, which is actually in the Timer Routines segment ($4453-$45b7), not in the Zobrist code itself.

Additional Context

The full codebase is at: https://github.com/barryw/Chess (on the feature/chess-ai branch in .worktrees/chess-ai/)

The test file that triggers this is tests/ai_enhancements.6502. The failing test simply does:

jsr([InitZobristTables], stop_on_rts = true, fail_on_brk = true)

Zero Page Usage

The Zobrist functions use these zero page addresses:

• $fb-$fc - PRNG state (16-bit LFSR)
• $f9-$fa - Pointer for filling tables

Let me know if you need any other files or information!

[barryw]

Additional Finding: Error Address Analysis

The error address $44F6 falls within the Timer Routines segment ($4453-$45b7), not the Zobrist code ($45b8-$4622).

Here's the timer routines code (timers.asm):

// Timer Library
// Manages up to 8 single-shot or continuous timers with callbacks

*=* "Timer Routines"

ClearTimers:
  FillMI($00, c64lib_timers, TIMER_STRUCT_BYTES)
  rts

SetupTimers:
  // Timer 0: Flash piece (continuous, starts disabled)
  StoreWord(r0, PIECE_FLASH_SPEED)
  StoreWord(r1, FlashPieceCallback)
  lda #TIMER_CONTINUOUS
  sta r2L
  lda #TIMER_FLASH_PIECE
  sta r2H
  lda #DISABLE
  sta r3L
  jsr CreateTimer
  // ... more timer setup ...
  rts

CreateTimer:
  lda r2H
  mult8               // pseudo-command: asl asl asl
  tax
  stb r3L:c64lib_timers, x
  stb r2L:c64lib_timers + $01, x
  stb r0L:c64lib_timers + $02, x
  stb r0H:c64lib_timers + $03, x
  stb r0L:c64lib_timers + $04, x
  stb r0H:c64lib_timers + $05, x
  stb r1L:c64lib_timers + $06, x
  stb r1H:c64lib_timers + $07, x
  rts

UpdateTimers:
  // ... timer update logic ...
  rts

EnableTimer:
  mult8
  tax
  stb #ENABLE:c64lib_timers, x
  rts

DisableTimer:
  mult8
  tax
  stb #DISABLE:c64lib_timers, x
  rts

Memory Layout (full)

$0801-$080c Basic
$080e-$0828 Main
$08c0-$0ac6 Keyboard Scan Routine
$0c00-$0e59 Sprites
$0e5a-$0f38 Board Storage
$0f39-$1e07 Routines
$1e08-$1f2d Raster Routine
$1f2e-$2560 Variable Storage
$2561-$2bb6 String Storage
$2bb7-$2d33 Opening Book
$3000-$37ff Custom Characters
$3800-$3bfe Display
$3bff-$3cca Input
$3ccb-$3fdf Game
$3fe0-$4452 Menus
$4453-$45b7 Timer Routines      <- $44F6 is HERE
$45b8-$4622 AI Zobrist          <- InitZobristTables is HERE
$4623-$4d63 Zobrist Tables
$5b00-$a4a7 Generated Opening Book
$a4a8-$a675 Piece-Square Tables
$a676-$acdc AI MoveGen
$acdd-$b1ec AI Eval
$b1ed-$b301 Transposition Table
$b302-$bb75 AI Search
$bb76-$bd8d AI Rules

Key Observation

The test calls jsr([InitZobristTables]) which is at $45b8+. But the error occurs at $44F6 in Timer Routines. The test doesn't call any timer code directly.

However, InitZobristTables uses these zero-page addresses:

• $fb-$fc - PRNG state
• $f9-$fa - Pointer for table filling

Pseudo-commands Used

The code uses KickAssembler pseudo-commands like stb (store byte) and mult8 (asl asl asl). These expand to standard 6502 instructions at assembly time.

[barryw]

Found the Bug! 🐛

The issue is a register clobber causing an infinite loop that corrupts memory.

The Problem

In InitZobristTables, you use X as a page counter:

  ldx #$00        // Page counter (6 pages = 1536 bytes)
  ldy #$00

!fillpieces:
  jsr ZobristPRNG   // <-- THIS CLOBBERS X!
  sta ($f9), y
  iny
  bne !fillpieces-

  inc $fa
  inx               // X was clobbered, so this doesn't work as expected
  cpx #$06        
  bne !fillpieces-

But ZobristPRNG also uses X as its internal loop counter:

ZobristPRNG:
  ldx #$08          // Sets X to 8
!prng_loop:
  ...
  dex
  bne !prng_loop-   // X ends up as 0
  ...
  rts               // Returns with X = 0

What Happens

1. InitZobristTables sets X = 0 (page counter)
2. Calls ZobristPRNG → X becomes 0 (after PRNG loop)
3. Fills 256 bytes (Y loop)
4. inc $fa → pointer high byte incremented
5. inx → X = 1
6. cpx #$06 → not equal, continue
7. Calls ZobristPRNG → X becomes 0 again!
8. Fills another 256 bytes
9. inx → X = 1 (not 2!)
10. Loop never terminates because X never reaches 6

The Result

The loop runs forever, incrementing $fa (the pointer high byte) indefinitely. This writes random data throughout memory, eventually corrupting the Timer Routines code at $44F6, replacing $A5 (LDA) with $5A (random garbage that happens to be the 65C02 PHY opcode).

The Fix

Preserve X across the jsr ZobristPRNG call. Options:

Option 1: Use a zero-page variable for the page counter

  lda #$00
  sta PageCount     // Use a ZP variable instead of X
  
!fillpieces:
  jsr ZobristPRNG
  sta ($f9), y
  iny
  bne !fillpieces-

  inc $fa
  inc PageCount
  lda PageCount
  cmp #$06
  bne !fillpieces-

Option 2: Push/pull X around the JSR

!fillpieces:
  txa
  pha               // Save X
  jsr ZobristPRNG
  pla
  tax               // Restore X
  sta ($f9), y     // Oops - A is now X, not the random value!

Actually Option 2 won't work easily because you need A for the random value. Option 1 is cleaner.

Option 3: Make ZobristPRNG preserve X

ZobristPRNG:
  txa
  pha               // Save caller's X
  ldx #$08
!prng_loop:
  ...
  dex
  bne !prng_loop-
  
  lda $fb
  eor $fc
  sta ZobristLastRandom
  
  pla
  tax               // Restore caller's X
  lda ZobristLastRandom  // Return random value in A
  rts

This is probably the best approach since it makes ZobristPRNG a well-behaved subroutine that doesn't clobber the caller's registers (except A, which is the return value).

[barryw]

Fixed! Thank you!

You nailed it! The X register clobber was exactly the problem. I've pushed a fix that preserves X in ZobristPRNG:

ZobristPRNG:
  txa
  pha               // Preserve X register for callers
  ldx #$08          // 8 iterations per output byte

!prng_loop:
  // ... LFSR logic ...
  dex
  bne !prng_loop-

  // ... result logic ...

  pla
  tax               // Restore X register

  lda ZobristLastRandom  // Return random value in A
  rts

Results

After the fix, the first 11 tests now pass:

• ✅ PST tables (3 tests)
• ✅ MVV-LVA capture ordering (6 tests)
• ✅ Quiescence search (2 tests)

New Issue with TT Tests

The Transposition Table tests still fail with a different error:

OpCode B2 @ address B208 is not supported

I suspect this might be related to memory banking. The TT code is located at $B1ED-$B301, which is inside the BASIC ROM address range ($A000-$BFFF). When TTClear does:

lda $01
and #$fe    // Bank out BASIC ROM
sta $01

...I wonder if there's something going on with how sim6502 handles that banking while code is executing from that same address range.

But that's likely a separate issue - or possibly my code's fault for placing TT routines in the $Axxx-$Bxxx range. Let me investigate further.

Thanks again for the quick and accurate diagnosis! 🎉

[barryw]

Root Cause Found: Code Self-Destruction

After your comment about sim6502 being a generic 6502 emulator (not C64-specific), I realized the real issue.

The Problem

Looking at the memory map:

$a4a8-$a675 Piece-Square Tables
$a676-$acdc AI MoveGen
$acdd-$b1ec AI Eval
$b1ed-$b301 Transposition Table (CODE)  <-- TT functions live here
$b302-$bb75 AI Search

And TTClear does this:

TTClear:
  // ... setup ...
  lda #<$A000
  sta $f0
  lda #>$A000
  sta $f1
  
!tt_clear_page:
  sta ($f0), y      // Write $00
  iny
  bne !tt_clear_page-
  inc $f1
  lda $f1
  cmp #$E0          // Loop until $f1 == $E0 (64 pages, $A000-$DFFF)
  bne !tt_clear_page-

We're zeroing $A000-$DFFF, which includes $B1ED where our own code is running! 💥

Why This Works on Real C64

On a real C64:

1. Program loads into RAM (code at $B1ED goes into RAM)
2. BASIC ROM overlays $A000-$BFFF, hiding that RAM
3. When we bank out BASIC ROM (lda $01; and #$fe; sta $01), we access the underlying RAM which is separate storage
4. Writing to $A000-$BFFF for TT data doesn't touch our code - it's writing to the "under-ROM RAM"
5. Our code at $B1ED remains intact in the main RAM layer

Why This Fails in sim6502

sim6502 has flat 64KB RAM with no ROM overlay. Address $B1ED is just $B1ED - there's no separate "under-ROM" storage. So when TTClear writes zeros to $B1ED, it destroys the very code that's executing.

Potential sim6502 Enhancement?

This might be an interesting case for sim6502 to handle. Options could include:

• C64 memory model mode: Emulate the ROM/RAM banking behavior of $01
• Memory protection: Warn when code writes to addresses in the currently-executing code range
• Documentation note: Warn users that C64 banking tricks won't work in flat RAM mode

For my code, the fix is straightforward - I need to either relocate my TT functions outside the $A000-$BFFF range, or use a different TT_BASE address when testing in sim6502.

Thanks for the rubber-duck debugging session! 🦆

[barryw]

System Emulation Framework Proposed

Based on your discovery that the real issue is C64 memory banking (writing "under ROM" to $A000-$DFFF), we've designed a system emulation framework that would properly support this.

See Issue #6 for the full specification: #6

Key Points for Your Chess Engine

1. New syntax: system(c64) instead of processor(6510)
2. Proper banking: Writing to $A000-$DFFF with BASIC ROM banked out will correctly write to underlying RAM
3. ROM support: rom("kernal", "kernal.bin") to load your own legally-owned ROMs
4. Banking via $01: Your code can set $01 = $35 to bank out all ROM and access full 64KB RAM

Example Test File

system(c64)
rom("kernal", "kernal.901227-03.bin")
rom("basic", "basic.901226-01.bin")
load("chess-engine.prg")

test("init_transposition_table", "Should clear $A000-$DFFF without corrupting code") {
    ; Bank out BASIC ROM to access RAM underneath
    poke($01, $35)
    
    jsr(init_zobrist_tables)
    
    ; Verify RAM was cleared, not code corrupted
    assert(peek($A000) == 0)
    assert(peek($DFFF) == 0)
}

We'd love your feedback on Issue #6 - does this design cover your needs? Any C64 edge cases we should consider?

---

🤖 Generated with Claude Code

[barryw]

Good news! Implementation is complete. 🎉

The system emulation framework you requested has been implemented and is ready for testing.

Your requirements have been addressed:

1. Writes always go to RAM "underneath" - Even when ROM is visible, writes go to underlying RAM
2. Banking via $01 port - Full support for LORAM, HIRAM, CHAREN bits
3. All banking modes - $34 (all RAM), $35, $36, $37 etc.

How to use it:

suite("Chess Engine Test") {
  system(c64)
  rom("kernal", "path/to/kernal.rom")  ; Optional
  rom("basic", "path/to/basic.rom")    ; Optional
  
  test("under-rom-write", "Writing under ROM works") {
    ; Write to RAM under BASIC ROM at $A000
    $A000 = $42
    
    ; Bank out ROMs to see our write
    $01 = $35
    assert(peekbyte($A000) == $42, "RAM visible")
  }
}

The implementation is on master branch. Please give it a try and let us know if it meets your chess engine's needs!

See Issue #6 for full implementation details.