diff --git a/README.md b/README.md index 14bd6d98..e8dd9cf8 100644 --- a/README.md +++ b/README.md @@ -147,7 +147,7 @@ Hardware, testing, and development references: | Area | Documentation | |---|---| | Machine profiles | [Indigo2 IP22](docs/indigo2-ip22.md), [Indigo2 IMPACT IP28](docs/indigo2-ip28.md), and [GR2 XZ/Extreme](docs/indy-xz-elan.md), and the [SGI O2 (IP32)](docs/o2-ip32.md). | -| Graphics and peripherals | [REX3](docs/Indy/rex3.md), [HAL2 audio](docs/Indy/hal2.md), [WD33C93A SCSI](docs/Indy/wd33c93a.md), [interrupt map](docs/interrupt_map.md), and [Ultra64](docs/Indy/ultra64.md) / [N64 GIO specifications](docs/Indy/n64_gio_specs.md). | +| Graphics and peripherals | [REX3](docs/Indy/rex3.md), [HAL2 audio](docs/Indy/hal2.md), [WD33C93A SCSI](docs/Indy/wd33c93a.md), [QLogic ISP1040 SCSI (Octane)](docs/Octane/qla1040.md), [interrupt map](docs/interrupt_map.md), and [Ultra64](docs/Indy/ultra64.md) / [N64 GIO specifications](docs/Indy/n64_gio_specs.md). | | CPU test suite | [Suite README](cpu-tests/README.md), [current status](cpu-tests/docs/status.md), [findings](cpu-tests/docs/findings.md), [gotchas](cpu-tests/docs/gotchas.md), and [emulator support](docs/cpu-test-support.md). | | CPU test infrastructure | [Toolchain](cpu-tests/docs/toolchain.md), [memory map](cpu-tests/docs/memory-map.md), [oracle overview](cpu-tests/docs/oracle.md), [hardware oracle](cpu-tests/oracle/README.md), [R4600 notes](cpu-tests/docs/r4600.md), and [original test plan](cpu-tests/PLAN.md). | | Benchmarks and memory tests | [Benchmark suite](bench/README.md), [prebuilt assets](bench/prebuilt/README.md), [reference results](data/bench_reference.README.md), [GUI benchmark design](clanker/plans/gui-benchmark-plan.md), and [memory stress utility](test/memstress/README.md). | diff --git a/docs/Octane/qla1040.md b/docs/Octane/qla1040.md new file mode 100644 index 00000000..3416b013 --- /dev/null +++ b/docs/Octane/qla1040.md @@ -0,0 +1,1085 @@ +# QLogic ISP1040 (QL1040B) SCSI controller on the Octane + +The Octane's two "Integral SCSI controller" ports are QLogic ISP1040B chips +(SGI calls the part QL1040B) sitting on the IP30 board's Bridge at PCI +slots 0 and 1. To boot IRIX on an emulated Octane, iris has to present a +chip that the PROM's and IRIX's `ql` drivers accept: PCI configuration +space, a 256-byte register block, the mailbox protocol of the chip's +firmware, and DMA'd request/response queues in host memory. This document +is the register map, the firmware interface, and a description of how +each OS we have source for (and IRIX, which we don't) actually drives it. + +Nothing executes on the real chip's RISC processor in an emulator. What +the drivers see is the *firmware's* behaviour, so this document describes +the chip as the firmware makes it look, not the internal RISC. + +## Sources + +Everything here was derived from these, read side by side: + +| Source | Licence | What it gave | +|---|---|---| +| OpenBSD `sys/dev/ic/qlw.c`, `qlwreg.h`, `sys/dev/pci/qlw_pci.c` (d82e6535c6, the last OpenBSD/sgi tree) | ISC | The simplest complete ISP1040 driver, and the one that ran on real Octanes (`#ifdef __sgi__`: initiator ID 0). | +| FreeBSD `stable/12` `sys/dev/isp/isp.c`, `ispreg.h`, `ispmbox.h`, `ispvar.h` (Matthew Jacob) | BSD-2 | The full register map (BIU/DMA/SXP/RISC blocks), the mailbox in/out register tables, every completion code. FreeBSD 13 dropped parallel SCSI, so use stable/12. | +| NetBSD `sys/dev/ic/isp*.c`, `sys/dev/pci/isp_pci.c`, `sys/dev/microcode/isp/asm_1040.h` | BSD | Same driver family; PCI attach; the 4.66.00 initiator firmware image. | +| Linux `drivers/scsi/qla1280.c`, `qla1280.h`; linux-firmware `qlogic/1040.bin` | GPL-2 | QLogic's own driver lineage: reset dance, 64-bit (A64) IOCBs, firmware 7.65. | +| IRIX 6.5.30 `/usr/include/sys/ql.h` (shipped header) | — | SGI's driver structures: the register struct as IRIX sees it, IOCB layouts, mailbox command/status enums, IP30 compile options. | +| IRIX 6.5.30 `/var/sysgen/boot/ql.o` (shipped kernel module, with symbols), `unix`, PROM 4.17 image | — | Disassembled for interoperability: the exact register sequence IRIX performs, which mailbox commands it issues and in what order, the firmware images it carries. Strings only from the PROM. | +| QLogic ISP1040B / ISP1040C data sheets (bitsavers `pdf/qlogic/`) | — | Block diagram, FIFO sizes, clock. No register-level manual for the 1040 has surfaced; the data sheets are marketing-level. | +| SGI Bridge ASIC spec rev 4.0 (`docs/Octane/`), IRIX `sys/PCI/{bridge,pciio,pcibr}.h`, OpenBSD `arch/sgi/xbow/xbridge*.{c,h}`, Linux `arch/mips/pci/pci-xtalk-bridge.c` | — | How the Octane's Bridge presents the chip: byte swapping, 64-bit DMA attributes, devio windows. | + +No leaked IRIX source was used. IRIX behaviour comes from its shipped +header and from disassembling the shipped binaries. + +## 1. The chip + +The ISP1040 is a PCI SCSI I/O processor: a 16-bit RISC core running +firmware out of external SRAM, a SCSI sequencer (the "SXP") and a +dual-channel DMA engine (command DMA for IOCBs/status, data DMA for SCSI +data) behind a PCI bus interface unit (BIU). The host talks to the +firmware through eight 16-bit mailbox registers and through request and +response queues of 64-byte I/O control blocks (IOCBs) in host memory. + +### Variants + +| Name | PCI device ID | PCI revision ID | CONF0 bits 3:0 | Clock | Notes | +|---|---|---|---|---|---| +| ISP1020 | 0x1020 | 0 | 1 | 40 MHz | Fast SCSI | +| ISP1020A | 0x1020 | 1 | 2 | 40 MHz | some Ultra-capable (RISC PSR bit 7) | +| ISP1040 | 0x1020 | 2 | 3 | 60 MHz | Ultra (Fast-20) | +| ISP1040A | 0x1020 | 3 | 4 | 60 MHz | "busted FIFO": drivers leave the FIFO threshold at 0 | +| ISP1040B | 0x1020 | 4 | 5 | 60 MHz | | +| ISP1040B rev 2 / "1040C" | 0x1020 | 5 | 6 | 60 MHz | **the Octane's chip** | +| ISP1240 | 0x1240 | | | 60 MHz | dual bus, 1080-style register banks | +| ISP1080 / 1280 / 12160 | 0x1080 / 0x1280 / 0x1216 | | | 100 MHz | Ultra2/Ultra3, different DMA/SXP banking | + +The PCI revision table is OpenBSD's and IRIX's (`REV_ISP1040BV2 = 5`, +printed by `hinv` as "QL1040B (rev. 2)"); the CONF0 column is what +FreeBSD and Linux decode from the hardware revision field, which runs one +ahead of the PCI revision ID. The real Octane chip's CONF0 value has not +been read back; `6` is what the tables imply for PCI revision 5. + +PCI vendor ID is 0x1077. Only the `0x1020` family matters for the Octane; +the 1080/1240/12160 differences are noted where a driver code path +branches on them so the reader knows what to ignore. + +### What the Octane has + +From `hinv -v` on an Octane running IRIX 6.5.30: + +``` +Integral SCSI controller 0: Version QL1040B (rev. 2), single ended +Integral SCSI controller 1: Version QL1040B (rev. 2), single ended + PCI Adapter ID (vendor 0x1077, device 0x1020) PCI slot 0 + PCI Adapter ID (vendor 0x1077, device 0x1020) PCI slot 1 +``` + +Both chips are devices 0 and 1 on the Bridge at xtalk widget 15 (the IP30 +board), beside the IOC3 (device 2) and RAD audio (device 3). Controller 0 +is the internal bus holding the system disks; controller 1 is the +external connector. They are single-ended, and the host adapter uses +SCSI ID 0 (SGI convention; `INITIATOR_SCSI_ID 0` in `ql.h`). The onboard +chips have no NVRAM: IRIX never touches the NVRAM register, and OpenBSD's +driver falls back to defaults when the NVRAM reads back invalid. + +## 2. PCI configuration space + +| Offset | Register | Value / use | +|---|---|---| +| 0x00 | Vendor / Device ID | 0x1077 / 0x1020 | +| 0x04 | Command / Status | drivers set I/O space, memory space, bus master, MWI, parity and SERR enables | +| 0x08 | Revision ID, class | revision 5 on the Octane; class 0x01 0x00 (SCSI storage) | +| 0x0C | Cache line size, latency timer | IRIX writes 0x80 / 0x40 (`pciio_config_set` of 1 byte at 12 and 13); NetBSD and OpenBSD set 0x10 / 0x80 | +| 0x10 | BAR0: I/O space | 256 bytes of registers. **IRIX uses this one** (`USE_IOSPACE` for IP30, `PCIIO_SPACE_WIN(0)`). | +| 0x14 | BAR1: memory space | the same 256 registers; Linux, NetBSD prefer it | +| 0x2C | Subsystem vendor / ID | supported by the 1040B | +| 0x30 | Expansion ROM BAR | BIOS ROM on PC adapters; drivers disable it (clear bit 0) | +| 0x3C / 0x3D | Interrupt line / pin | pin A | +| 0x40 | "ISP_Config_1" | vendor-specific; named in IRIX's `PCI_REG` struct, not written by any driver read here | + +IRIX reads vendor/device (2 bytes at 0), the revision byte (at 8), reads +and writes the command register (2 bytes at 4, OR-ing in bit 0), writes +the latency timer and cache line size, then allocates Bridge read +response buffers for the slot (`pcibr_rrb_alloc`; the PROM prints "Cannot +allocate %d RRBs for ql %d" when that fails). + +## 3. Register map + +All registers are 16 bits wide at even offsets from the BAR. Offsets are +PCI offsets; see §8 for how they appear to a big-endian CPU through the +Bridge. Unless stated, bit meanings come from FreeBSD's `ispreg.h` and +Linux's `qla1280.h`, which agree. + +### 3.1 Bus interface unit, 0x00–0x1F + +| Offset | Name | Access | Bits | +|---|---|---|---| +| 0x00 | ID_LO | R | bus ID low | +| 0x02 | ID_HI | R | bus ID high | +| 0x04 | CONF0 | R | bits 3:0 hardware revision (table above) | +| 0x06 | CONF1 | RW | bit 2 `BURST_ENABLE`; bit 3 `SXP` register select (maps the SXP registers over 0x80–0xFF); bits 6:4 PCI FIFO threshold: 0x10 = 16 B, 0x20 = 32 B, 0x30 = 64 B, 0x40 = 128 B; bits 1:0 the old SBus FIFO threshold; bits 9:8 (1080 family only) DMA/SXP bank select (0x300) | +| 0x08 | ICR | RW | bit 0 `SOFT_RESET`; bit 1 `ENABLE_ALL_INTS`; bit 2 `ENABLE_RISC_INT`; bit 3 `ENABLE_SXP_INT`; bit 4 `ENABLE_CDMA_INT`; bit 5 `ENABLE_DMA_INT`; bit 8 flash enable, bit 9 flash upper bank (Linux) | +| 0x0A | ISR | R | bit 1 interrupt pending (global); bit 2 `RISC_INT`; bit 3 SXP int; bit 4 CDMA int; bit 5 DMA int; bit 14 "PCI 64-bit slot" (Linux) | +| 0x0C | SEMA | RW | bit 0 `LOCK`: the outgoing mailboxes hold a mailbox completion or async event; bit 1 status | +| 0x0E | NVRAM | RW | bit 0 clock, bit 1 chip select, bit 2 data out (host → EEPROM), bit 3 data in; bit 7 busy on later chips | +| 0x10 | FLASH_BIOS_DATA | RW | (OpenBSD/Linux) | +| 0x12 | FLASH_BIOS_ADDR | RW | | + +Writes of 0 to ISR (IRIX does this in its parity routine) are harmless. + +### 3.2 DMA channels, 0x20–0x63 (1040 layout) + +The 1080/1240/12160 move these to a bank at 0x80 selected by CONF1 +bits 9:8; on the 1040 they are fixed here. "RW\*" means the RISC must be +paused. + +| Offset | Name | Bits | +|---|---|---| +| 0x20 | CDMA_CONF (RW\*) | bit 0 direction (1 = host → FIFO), bit 1 `BURST_ENABLE`, bit 2 enable interrupts to RISC, bit 3 enable SXP → DMA | +| 0x22 | CDMA_CONTROL (RW\*) | bit 0 strobe (start), bit 1 reset interrupt, bit 2 clear FIFO, bit 3 clear channel (and reset DMA registers), bit 4 suspend | +| 0x24 | CDMA_STATUS (R) | 0x8000 interrupt, 0x4000 retry status, 0x3000 channel status (0x1000 active to host, 0x2000 suspended, 0x3000 transfer in progress), 0x0100 FIFO overrun, 0x0080 FIFO underrun, 0x0040 bus error, 0x0020 bus parity, 0x0010 clear pending, 0x0008 terminal count, 0x0004 suspended, 0x0003 pipeline state | +| 0x26 | CDMA_FIFO_STS (R) | 0x200 overrun, 0x100 underrun, 0xFF byte count | +| 0x28 | CDMA_COUNT (RW\*) | transfer count | +| 0x2C–0x32 | CDMA_ADDR0..3 (RW\*) | DMA address words 0..3 | +| 0x40 | DDMA_CONF (RW\*) | as CDMA_CONF | +| 0x42 | DDMA_CONTROL (RW\*) | as CDMA_CONTROL | +| 0x44 | DDMA_STATUS (R) | as CDMA_STATUS | +| 0x46 | DDMA_FIFO_STS (R) | | +| 0x48 / 0x4A | DDMA_COUNT_LO / HI | | +| 0x4C / 0x4E | DDMA_ADDR0 / 1 | | +| 0x50 / 0x52 | DDMA_ADDR2 / 3 | 1040A and later | +| 0x60 / 0x62 | DFIFO_COMMAND / DFIFO_DATA | FIFO access ports | + +Drivers only touch these to clear the channels after reset (`CONTROL = +CLEAR_CHAN | RESET_INT`, IRIX writes 0x0E = clear channel + clear FIFO + +reset int) and to set the burst-enable bits. The firmware programs them +internally; an emulator need not model the DMA engine itself, only +return sane values. + +### 3.3 Mailboxes, 0x70–0x7E + +| Offset | Register | +|---|---| +| 0x70 | MAILBOX0 — command (in) / status (out) | +| 0x72 | MAILBOX1 | +| 0x74 | MAILBOX2 | +| 0x76 | MAILBOX3 | +| 0x78 | MAILBOX4 — doubles as the request queue in-pointer (host writes) and out-pointer (firmware writes) | +| 0x7A | MAILBOX5 — doubles as the response queue in-pointer (firmware writes) and out-pointer (host writes) | +| 0x7C | MAILBOX6 | +| 0x7E | MAILBOX7 | + +Each mailbox is really two registers: a write lands in the *incoming* +mailbox the RISC reads, a read returns the *outgoing* mailbox the RISC +wrote. Writing MAILBOX4 does not disturb what the host reads back from +MAILBOX4. This matters for the queue pointers (§4.4). + +### 3.4 RISC processor registers, 0x80–0xBF (CONF1 bit 3 = 0) + +| Offset | Name | Notes | +|---|---|---| +| 0x80 | ACC | accumulator | +| 0x82–0x9E | R1–R15 | general registers | +| 0xA0 | PSR | 0x8000 force true, 0x4000 loop count done, 0x2000 RISC int, 0x1000 timer rollover, 0x0800 ALU overflow, 0x0400 ALU MSB, 0x0200 carry, 0x0100 zero, **0x0080 PCI Ultra capable**, 0x0020 SBus Ultra, 0x0010 DMA int, 0x0008 SXP int, 0x0004 host int, 0x0002 int pending, 0x0001 force false | +| 0xA2 | IVR | interrupt vector | +| 0xA4 | PCR | processor control | +| 0xA6 / 0xA8 | RAR0 / RAR1 | RAM address registers | +| 0xAA | LCR | loop counter | +| 0xAC | PC | program counter (R) | +| 0xAE | MTR | memory timing; FreeBSD writes 0x1212, or 0x1313 for "fast memory timing" | +| 0xB0 | EMB | external memory boundary (1040); 0x8 = dual bank (1080) | +| 0xB2 | SP | stack pointer | +| 0xBE | HRL | hardware revision level (R) | +| 0xC0 | **HCCR** | host command and control, below | +| 0xC2 / 0xC4 | BP0 / BP1 | breakpoints | +| 0xC6 / 0xC8 | TCR / TMR | test control / test mode (W) | +| 0xCC / 0xCE | GPIO data / enable | 1080-class parts (termination control). `ql.h` documents GPIO use for SGI's XIO "MSCSI" boards, not IP30. | +| 0xF2 | SCSI control pins | (Linux `scsiControlPins`) | + +All RISC registers except HCCR and the breakpoints are only accessible +while the RISC is paused (HCCR PAUSE, HCCR read bit 0x20 set). + +**HCCR (0xC0)** — a write is a command in bits 15:12, a read is status: + +| Write | Command | +|---|---| +| 0x0000 | NOP | +| 0x1000 | reset RISC | +| 0x2000 | pause RISC | +| 0x3000 | release RISC (from pause or reset) | +| 0x4000 | single step | +| 0x5000 | set host interrupt (tells the RISC a mailbox command is waiting) | +| 0x6000 | clear host interrupt | +| 0x7000 | clear RISC interrupt (acknowledge; the host does this after every interrupt) | +| 0x8000 | change breakpoint enables | +| 0x9000 \| bit 0 | write BIOS enable (0x9000 = disable BIOS so the RISC runs from RAM) | +| 0xA000 \| bits 8,9 | write SRAM parity enable, bit 8 = bank 0, bit 9 = bank 1 (IRIX writes 0xA100) | +| 0xE000 | generate a parity error | +| 0xF000 | test mode | + +| Read bit | Meaning | +|---|---| +| 0x0080 | host interrupt set (a mailbox command is pending) | +| 0x0040 | RISC reset in progress | +| 0x0020 | RISC paused | +| 0x0100 / 0x0200 | parity enabled on bank 0 / 1 | +| 0x0400 | parity error detected (1040; IRIX `HCCR_SRAM_PARITY_ERROR_DETECTED_1040`) | + +### 3.5 SXP (SCSI executive processor) registers, 0x80–0xFF (CONF1 bit 3 = 1) + +Set CONF1 bit 3 while the RISC is paused and the SXP bank replaces the +RISC bank. Only two are read by hosts, but the full map (FreeBSD) is: + +| Offset | Name | Offset | Name | +|---|---|---|---| +| 0x80 | PART_ID | 0xC0 | SCSI_ID (0x0F00 (re)selecting id, 0x000F select id) | +| 0x82 | CONFIG1 (0xF000 async setup, 0x0600 selection timeout, 0x00E0 clock factor, 0x000F SCSI id) | 0xC2 | DEV_CONFIG1 (sync hold/setup/offset) | +| 0x84 | CONFIG2 (0x40 disable filter, 0x20 REQ/ACK pullups, 0x10 data pullups, 0x08 config autoload, 0x02 reselect, 0x01 select) | 0xC4 | DEV_CONFIG2 (0x400 wide, 0x200 parity, 0x100 block mode) | +| 0x86 | CONFIG3 | 0xC8 | PHASE_PTR | +| 0x8C | INSTRUCTION | 0xCC | BUF_PTR | +| 0x90 | RETURN_ADDR | 0xD0 | BUF_CTR | +| 0x94 | COMMAND (0x300B = reset SCSI bus) | 0xD2–0xD8 | BUFFER, BUF_BYTE, BUF_WD, BUF_WD_TRAN | +| 0x98 | INTERRUPT (0x8000 parity err, 0x4000 gross err, 0x2000 function abort, 0x1000 condition failed, 0x0800 FIFO empty, 0x0400 buf counter zero, 0x0200 xfer zero, 0x0080 int pending, 0x0040 cmd running, 0x000F return code) | 0xDA–0xE0 | FIFO, FIFO_STATUS, FIFO_TOP, FIFO_BOTTOM | +| 0x9C | SEQUENCE | 0xE4 | TRAN_REG | +| 0x9E | GROSS_ERR | 0xE8–0xEE | TRAN_CNT_LO/HI, TRAN_CTR_LO/HI | +| 0xA0 | EXCEPTION enables | 0xF0 | ARB_DATA | +| 0xA4 | OVERRIDE (0x0008 FIFO reset, 0x0004 terminate, 0x0002 reset regs, 0x0001 reset module, …) | 0xF2 | PINS_CTRL (0x8000 phase valid, 0x100 REQ, 0x80 ACK, 0x40 RST, 0x20 BSY, 0x10 SEL, 0x08 ATN, 0x04 MSG, 0x02 C/D, 0x01 I/O) | +| 0xA8 | LIT_BASE | 0xF4 | PINS_DATA | +| 0xAC / 0xB0 / 0xB4 | USER_FLAGS / USER_EXCEPT / BREAKPOINT | 0xF6 | **PINS_DIFF**: 0x0200 DIFFSENS, **0x0100 differential mode**, 0x0080 enable output, 0x007C diff control pins, 0x0002 target, 0x0001 initiator. Ultra2 adds 0x1000 LVD, 0x0800 HVD, 0x0400 SE. | + +Hosts read `PINS_DIFF` to learn the bus type: FreeBSD prints +"Differential Mode" if bit 8 is set; IRIX's `ql_diffsense_sp` pauses the +RISC, sets CONF1 bit 3, reads 0xF6 and reports "SCSI Adapter %x in bus +mode : %d" (1 = single-ended, 2 = HVD, 4 = LVD). For an Octane return +bit 8 clear. FreeBSD also reads `RISC_PSR` (0xA0) bit 7 to decide the +part is Ultra capable and runs the clock at 60 MHz. + +## 4. Principles of operation + +### 4.1 Reset + +Every driver begins the same way (Linux `qla1280_chip_diag`, FreeBSD +`isp_reset`, OpenBSD `qlw_softreset`, IRIX `ql_download_fw` / +`ql_reset_interface`): + +1. Disable interrupts (ICR = 0), pause the RISC (HCCR 0x2000) if the + driver wants to read SXP/RISC registers first. +2. ICR = `SOFT_RESET` (bit 0). The bit clears itself when the chip is + back (Linux waits up to 5 s polling it; FreeBSD too). The chip does + not respond on PCI during the reset — Linux notes a read-back right + after the write machine-checks on some hosts, so emulate the bit + clearing after a short delay rather than instantly if you want to be + faithful, but instantly works for every driver read here. +3. Clear the DMA channels: CDMA_CONTROL and DDMA_CONTROL = `CLEAR_CHAN | + RESET_INT` (FreeBSD) or 0x0E (IRIX, 1040 only). +4. CONF1 = 0 (FreeBSD/OpenBSD), then HCCR 0x1000 (reset RISC), SEMA = 0, + HCCR 0x3000 (release). Linux does it in one write: HCCR = + `RESET | RELEASE | DISABLE_BIOS` = 0x1000|0x3000|0x9000 — the chip + takes that as the individual commands. +5. Wait for the ROM firmware's self test: MAILBOX0 reads 4 (`MBOX_BUSY`) + while it runs, then 0 (`FIRMWARE_ALIVE`; 1 = checksum error, 2 = + shadow load error). The other mailboxes then hold the product ID: + MAILBOX1 = 0x4953 "IS", MAILBOX2 = 0x5020 "P " (0x0000 on some old + parts), MAILBOX3 = 0x2020 " ", MAILBOX4 = 0x0001. Linux and OpenBSD + check these; IRIX does not. IRIX polls HCCR bit 0x40 (reset in + progress) clear instead, then waits while MAILBOX0 == 4. +6. CONF1 = burst enable | FIFO threshold (FreeBSD/OpenBSD 0x34 = 64 B, + IRIX and Linux 0x44 = 128 B), and the CDMA/DDMA_CONF burst bits. + On a 1040A the threshold bits are left 0. +7. HCCR 0x9000 (BIOS disable), enable interrupts (ICR = 6, bits 1 and 2), + then a mailbox NOP (FreeBSD/OpenBSD) and a MAILBOX REGISTER TEST + (everyone; see §4.2) to prove the ROM firmware answers. + +After this the chip runs its boot ROM, which services the small set of +mailbox commands needed to load firmware (0x00–0x08, 0x09/0x0A, 0x40–0x42). +Anything else returns 0x4001 (invalid command). + +The RAM contents survive ICR soft reset and HCCR RISC reset. IRIX relies +on that: `ql_download_fw` loads the firmware once, then every +`ql_reset_interface` (initial setup and every error recovery) resets the +chip and issues EXECUTE FIRMWARE again *without reloading*. An emulator +must keep the loaded image across resets. + +### 4.2 The mailbox protocol + +A mailbox command is at most eight 16-bit words. The host: + +1. writes the parameters to the incoming mailboxes (MAILBOX0 = command + code), +2. writes HCCR 0x5000 (set host interrupt). HCCR reads back with bit 0x80 + until the RISC takes the command, +3. waits for an interrupt (or polls): ISR bit 2 set **and** SEMA bit 0 set + means the outgoing mailboxes hold a mailbox completion or an async + event. MAILBOX0 ≥ 0x8000 is an async event (§4.6), otherwise it is + the command's status, +4. reads the outgoing mailboxes it cares about, writes SEMA = 0, and + writes HCCR 0x7000 (clear RISC interrupt). + +Only one mailbox command is outstanding at a time; while the semaphore is +held the firmware will not post another completion. Response-queue +interrupts (§4.4) arrive with SEMA bit 0 *clear*; that is how drivers tell +the two apart. OpenBSD and NetBSD/FreeBSD share this logic +(`qlw_read_isr`, `isp_pci_rd_isr`), Linux's `qla1280_isr` and IRIX's +`ql_service_interrupt` do the same. + +Status codes in MAILBOX0: + +| Code | Meaning | +|---|---| +| 0x4000 | command complete | +| 0x4001 | invalid command | +| 0x4002 | host interface error | +| 0x4003 | test failed | +| 0x4005 | command error (MAILBOX1 = subcode) | +| 0x4006 | command parameter error | +| 0x4007 | bus configuration error (IRIX `ql.h`) | +| 0x4010 | checksum error (FC firmware; the 1040 ROM reports a bad checksum as a failed VERIFY CHECKSUM) | + +Drivers wait between 0.4 s (OpenBSD, polled during attach) and 20 s +(Linux) for a completion; IRIX retries after each 1 s by writing HCCR 0x7000 and prints +"Mailbox timeout: ... resetting ISR and retrying". + +FreeBSD keeps a table of which mailboxes each command reads and writes; +it is reproduced here (bit n = MAILBOXn), with the parameter meanings +collected from all the drivers. + +| Code | Name | In | Out | Parameters | +|---|---|---|---|---| +| 0x00 | NO-OP | 0x01 | 0x01 | | +| 0x01 | LOAD RAM | 0x1F | 0x01 | 1 = RISC address, 2/3 = host address 31:16 / 15:0, 4 = word count. DMA from host to RISC RAM. | +| 0x02 | EXECUTE FIRMWARE | 0x03 | 0x01 | 1 = start address (0x1000) | +| 0x03 | DUMP RAM | 0x1F | 0x01 | as LOAD RAM, RISC → host | +| 0x04 | WRITE RAM WORD | 0x07 | 0x07 | 1 = address, 2 = data | +| 0x05 | READ RAM WORD | 0x03 | 0x07 | 1 = address → 2 = data | +| 0x06 | MAILBOX REGISTER TEST | 0x3F | 0x3F | echoes 1..5 (FreeBSD checks 1..5, Linux 1..7, IRIX writes 1..7 with 0x1111..0x7777, the PROM compares) | +| 0x07 | VERIFY CHECKSUM | 0x07 | 0x07 | 1 = start address. The 16-bit sum of words [start, start + len) where len = word[start+3] must be 0. Returns the sum in 1 (OpenBSD prints it, the PROM prints "theirs %x, mine %x"). | +| 0x08 | ABOUT FIRMWARE | 0x01 | 0x0F | → 1 = major, 2 = minor, 3 = subminor; 6 = attributes (0 on 1040) | +| 0x09 | LOAD RAM A64 (ROM) | 0xDF | 0x01 | as LOAD RAM plus 6/7 = host address 63:48 / 47:32. Served by the boot ROM. | +| 0x0A | DUMP RAM A64 (ROM) | | | | +| 0x0E | CHECK FIRMWARE | 0x01 | 0x05 | | +| 0x10 | INIT REQUEST QUEUE | 0x1F | 0x1F | 1 = entries, 2/3 = address 31:16 / 15:0, 4 = initial in-pointer (0) → 4 = firmware's out-pointer | +| 0x11 | INIT RESPONSE QUEUE | 0x3F | 0x3F | 1 = entries, 2/3 = address, 5 = initial out-pointer (0) → 5 = firmware's in-pointer | +| 0x12 | EXECUTE IOCB | 0x0F | 0x0F | 2/3 = address of a single IOCB | +| 0x13 | WAKE UP | 0x03 | 0x03 | | +| 0x14 | STOP FIRMWARE | 0x01 | 0x3F | | +| 0x15 | ABORT | 0x0F | 0x0F | 1 = (target << 8) \| lun (bit 15 = channel), 2/3 = handle | +| 0x16 | ABORT DEVICE | 0x03 | 0x03 | 1 = target/lun; outstanding commands return completion 5 | +| 0x17 | ABORT TARGET | 0x07 | 0x07 | | +| 0x18 | BUS RESET | 0x07 | 0x07 | 1 = reset delay (s), 2 = channel. Async 0x8001 follows. | +| 0x19–0x1C | STOP / START / SINGLE STEP / ABORT QUEUE | 0x03 | 0x07 | 1 = target/lun | +| 0x1D | GET DEVICE QUEUE STATUS | 0x03 | 0x4F | | +| 0x1F | GET FIRMWARE STATUS | 0x01 | 0x07 | → 2 = maximum outstanding commands (OpenBSD caps at 512, allocates half as many response entries) | +| 0x20–0x2A | GET … | 0x01/0x03 | | readback twins of the SET commands below; 0x28 GET TARGET PARAMETERS: 1 = target<<8 → 2 = flags, 3 = offset<<8 \| period, 6 = PPR info (12160) | +| 0x30 | SET INITIATOR SCSI ID | 0x03 | 0x03 | 1 = id \| channel << 7 | +| 0x31 | SET SELECTION TIMEOUT | 0x07 | 0x07 | 1 = ms (bus 0), 2 = ms (bus 1); 250 | +| 0x32 | SET RETRY COUNT | 0xC7 | 0xC7 | 1 = count, 2 = delay (bus 0); 6/7 bus 1 | +| 0x33 | SET TAG AGE LIMIT | 0x07 | 0x07 | 1/2 per bus; 8 | +| 0x34 | SET CLOCK RATE | 0x03 | 0x03 | 1 = MHz (40 or 60; Linux writes 40 even for a 1040) | +| 0x35 | SET ACTIVE NEGATION STATE | 0x07 | 0x07 | 1/2 per bus: bits 4 and 5 enable REQ/ACK and data-line active negation (drivers disagree which is which; everyone sets both) | +| 0x36 | SET ASYNC DATA SETUP TIME | 0x07 | 0x07 | 1/2 per bus, in clocks: 6 at 40 MHz, 8 or 9 at 60 | +| 0x37 | SET PCI CONTROL PARAMETERS | 0x07 | 0x07 | 1 = data DMA channel, 2 = command DMA channel; bit 1 = burst enable | +| 0x38 | SET TARGET PARAMETERS | 0x4F | 0x4F | 1 = target << 8 \| channel << 15, 2 = flags (§4.7), 3 = sync offset << 8 \| sync period | +| 0x39 | SET DEVICE QUEUE PARAMETERS | 0x0F | 0x0F | 1 = target << 8 \| channel << 15 \| lun, 2 = max queue depth, 3 = execution throttle | +| 0x3A | SET RESET DELAY PARAMETERS | 0x07 | 0x07 | 1/2 per bus | +| 0x40–0x42 | RETURN BIOS BLOCK ADDR, WRITE FOUR RAM WORDS, EXEC BIOS IOCB | | | BIOS support | +| 0x45 / 0x46 | SET / GET SYSTEM PARAMETER | 0x03 / 0x01 | 0x03 | 1 = parameter word (IRIX writes 1, Linux 0 on a 1040) | +| 0x48 / 0x49 | GET / SET SCAM CONFIGURATION | | | | +| 0x4A / 0x4B | SET / GET FIRMWARE FEATURES | 0x03 | 0x03 | 1 = bit 0 fast posting, bit 1 report LVD bus transitions, bit 5 disable synchronous backoff, bit 8 disable reselection fairness, bit 9 "SGI cache flush" (IRIX) | +| 0x50 / 0x51 | LOAD / DUMP RAM A64 | 0xDF | 0xDF | firmware-mode versions of 0x09/0x0A | +| 0x52 | INIT REQUEST QUEUE A64 | 0xDF | 0xFF | as 0x10 plus 6/7 = address 63:48 / 47:32 | +| 0x53 | INIT RESPONSE QUEUE A64 | 0xEF | 0xFF | as 0x11 plus 6/7 | +| 0x54 | EXECUTE IOCB A64 | 0xCF | 0x01 | | +| 0x55 / 0x56 | ENABLE TARGET MODE / GET TARGET STATUS | | | | +| 0x5A / 0x5B | SET / GET DATA OVERRUN RECOVERY MODE | 0x03 | 0x03 | 1 = 0: pad data until the target leaves data phase; 1: hang the bus and post 0x800C; 2: reset the bus, post 0x800D and return all I/O (IRIX and Linux use 2) | +| 0x5C / 0x5D | SET / GET HOST DATA | | | | + +### 4.3 Firmware images and loading + +The RISC runs code from external SRAM, 16-bit words addressed 0x0000– +0xFFFF. Drivers download an image to 0x1000 (`ISP_CODE_ORG`, +`QLW_CODE_ORG`) and start it there. The image is an array of 16-bit words +with a header the loaders rely on: + +| Word | Meaning | +|---|---| +| 0 | first instruction (0x0078 in the QLogic-released images, 0x0804 in some) | +| 1 | image identifier: 0x1041 initiator firmware, 0x103A initiator/target firmware (NetBSD's 7.65, Linux's 1040.bin), 0x102D the PROM's self-test image | +| 2 | 0 | +| 3 | **length in words**, including the header | +| 4 | 0 | +| 5… | ASCII copyright banner, stored as big-endian pairs: " COPYRIGHT 1991,1992,1993,1994 QLOGIC CORPORATION …" | +| 30… | "ISP1020 Firmware Version 02.55 Customer No. 00 Product No. 00 $" (word 30 in every initiator image inspected) | +| len−1 | a checksum word that makes the 16-bit sum of words [0, len) zero | + +Linux's `qlogic/1040.bin` has a 3-word file header in front of this +(word 2 = load address 0x1000); NetBSD/OpenBSD embed the bare array. + +Images seen: + +| Image | ID | Words | Banner version | +|---|---|---|---| +| IRIX 6.5.30 `ql.o` `risc_1040_code01` | 0x1041 | 11143 | ISP1020 Firmware Version 02.55 | +| IP30 PROM 4.17 (two copies, at 0x1C398 and 0xB4998 in the 1 MB image) | 0x1041 | 11190 | ISP1020 Firmware Version 02.55 | +| IP30 PROM 4.17, at 0xB3D30 | 0x102D | 1335 | ISP1020 Self Test Firmware Version 01.00 | +| NetBSD/OpenBSD `asm_1040.h` (initiator) | 0x1041 | 11472 | 04.66 (dated Sep 2000) | +| NetBSD `asm_1040.h` with `ISP_TARGET_MODE`, linux-firmware `1040.bin` | 0x103A | 16471 / 16728 | 07.65 | + +ABOUT FIRMWARE returns the banner's version: IRIX prints "Firmware +version: %d.%d.%d" from mailboxes 1–3 (2.55.0 for its image), FreeBSD +reports "loaded F/W Revision 4.66.0". An emulator that does not +run the RISC can parse the banner at word 30 of the loaded image to +answer this, and should answer GET FIRMWARE STATUS with a plausible +command limit (256 is safe: IRIX and Linux size their rings at 256/255 +without asking, OpenBSD divides this value by 3 for its CCB count). + +Two ways to load: + +- **PIO**: WRITE RAM WORD (4) for every word. The 1040 path in Linux + (`qla1280_load_firmware_pio`), OpenBSD (`qlw_load_firmware_words`), + FreeBSD/NetBSD for SCSI chips, and **IRIX for the 1040** + (`ql_download_fw`: 11143 mailbox commands at boot). The PROM reports + "QL: write data failure at %d word" for this path. +- **DMA**: LOAD RAM (1, or the A64 form 9 with the upper address words in + mailboxes 6/7), 1000 words per command from a bounce buffer (Linux + for 1080+, IRIX for 1080/1240/12160; the PROM prints "mailbox command + for dma load"). Linux's `DUMP_IT_BACK` debug option reads it back with + DUMP RAM; the PROM does read back ("Firmware Data Mismatch [0x%x]: his + = %x, mine = %x"), so READ RAM WORD / DUMP RAM must return what was + loaded. + +Then VERIFY CHECKSUM (7) with mailbox 1 = 0x1000 (Linux, FreeBSD, +OpenBSD, the PROM — "ISP Firmware Checksum mismatch: theirs %x, mine = +%x"; IRIX's kernel skips it), EXECUTE FIRMWARE (2) with mailbox 1 = +0x1000, a 250 ms pause in OpenBSD, then ABOUT FIRMWARE. From here the +full command set is live and the ROM-only commands may stop working +(nobody issues them again except IRIX's parity routine, §5.1). + +### 4.4 Request and response queues + +Both queues are rings of 64-byte entries in host memory, physically +contiguous, set up with INIT REQUEST QUEUE / INIT RESPONSE QUEUE (or the +A64 versions when the ring may live above 4 GB — which on the Octane it +always does, see §8). Sizes used: IRIX 256/256, Linux 255/63, OpenBSD +"max commands" (from GET FIRMWARE STATUS, capped 512) and half that, +FreeBSD `RQUEST_QUEUE_LEN`/`RESULT_QUEUE_LEN`. + +Queue pointers live in mailboxes 4 and 5, using the in/out duality: + +| | Host writes (incoming) | Host reads (outgoing) | +|---|---|---| +| MAILBOX4 | request **in**-pointer: index of the next free entry after the ones the host just filled | request **out**-pointer: how far the firmware has consumed | +| MAILBOX5 | response **out**-pointer: how far the host has consumed | response **in**-pointer: index after the last entry the firmware posted | + +To issue commands the host fills entries at its in-pointer, then writes +the new in-pointer to MAILBOX4 — no HCCR host interrupt. The firmware +polls the incoming MAILBOX4 and processes entries from its out-pointer up +to it, advancing the outgoing MAILBOX4. Linux computes free space from +the outgoing MAILBOX4 (`qla1280_req_pkt`); OpenBSD and IRIX track it +themselves (IRIX marks a free entry by writing 0xDEAD into its handle +word). + +When a command completes the firmware writes a status entry at the +response in-pointer, advances the outgoing MAILBOX5, and raises the RISC +interrupt *without* the semaphore. The host walks entries from its +out-pointer to the new in-pointer, then writes its out-pointer to +MAILBOX5 and HCCR 0x7000. The firmware does not overwrite entries the +host has not released. + +Entry header (first four bytes of every IOCB): + +| Byte | Field | +|---|---| +| 0 | entry type | +| 1 | entry count (1 + number of continuation entries that follow) | +| 2 | system defined (drivers put a sequence number or ring index; firmware ignores) | +| 3 | flags, written by the firmware in responses: 0x01 continuation, 0x02 ring full / busy, 0x04 bad header, 0x08 bad payload, 0x10 bad count, 0x20 bad order | + +Types: + +| Type | Entry | +|---|---| +| 1 | command, 32-bit addresses, 4 data segments | +| 2 | continuation, 7 × 32-bit segments | +| 3 | status (response queue) | +| 4 | marker | +| 5 | extended command (88-byte CDB) | +| 6 / 7 | ATIO / CTIO (target mode) | +| 8 | SCAM | +| 9 | command, 64-bit addresses, 2 data segments — **what IRIX and 64-bit Linux use** | +| 0xA | continuation, 5 × 64-bit segments | +| 0xB–0xF | target-mode entries (enable LUN, modify LUN, immediate notify, notify ack, CTIO1) | +| 0x10 | status continuation (60 more sense bytes) | + +Everything in an IOCB is little-endian. IRIX builds IOCBs in big-endian +structs and byte-swaps each 32-bit word (`munge()`) before DMA so the +memory image is the layout below. + +**Command entry (type 1):** + +| Offset | Size | Field | +|---|---|---| +| 0 | 4 | header | +| 4 | 4 | handle — returned in the status entry; IRIX packs channel (bit 12), target (bits 11:8) and a 0–255 slot index | +| 8 | 1 | LUN | +| 9 | 1 | target ID; bit 7 = channel on dual-bus parts | +| 10 | 2 | CDB length | +| 12 | 2 | control flags (below) | +| 14 | 2 | reserved | +| 16 | 2 | timeout, seconds (0 = none) | +| 18 | 2 | data segment count (over this entry and its continuations) | +| 20 | 12 | CDB | +| 32 | 4 × 8 | data segments: address (32), length (32) | + +**Command entry A64 (type 9):** identical through the CDB; offsets 32–39 +reserved; then two segments of {address 31:0, address 63:32, length} +at 40 and 52. + +**Continuation (type 2):** header, 4 bytes reserved, seven {address, +length} pairs from offset 8. **Continuation A64 (type 0xA):** header, +then five {lo, hi, length} triples from offset 4. + +**Control flags:** + +| Bit | Meaning | +|---|---| +| 0x0001 | no disconnects | +| 0x0002 / 0x0004 / 0x0008 | head-of-queue / ordered / simple queue tag | +| 0x0010 | target routine (target mode) | +| 0x0020 | data in (read) | +| 0x0040 | data out (write) | +| 0x0080 | no fast posting for this command (OpenBSD) | +| 0x0100 | disable auto request sense | +| 0x0200 | force asynchronous | +| 0x0400 | force synchronous | +| 0x0800 | initiate wide negotiation | +| 0x1000 | disable data parity | +| 0x2000 | stop queue on error | +| 0x4000 | extended sense supported | +| 0x8000 | priority | + +**Marker (type 4):** header; 4 bytes reserved; LUN at 8; target at 9 +(bit 7 channel); modifier at 10: 0 = sync this ID/LUN, 1 = sync this ID, +2 = sync all. After a SCSI bus reset (async 0x8001, or the host's own BUS +RESET) the firmware expects a marker before it will run new commands; +every driver sets a "send marker" flag on 0x8001 and queues a SYNC ALL +marker ahead of the next command. IRIX also sends one after an ABORT +DEVICE. + +**Status entry (type 3):** + +| Offset | Size | Field | +|---|---|---| +| 0 | 4 | header | +| 4 | 4 | handle of the completed command | +| 8 | 2 | SCSI status: low byte is the SCSI status byte; bit 9 (0x200) = sense data valid (`RQCS_SV`) | +| 10 | 2 | completion status (below) | +| 12 | 2 | state flags: 0x0100 got bus, 0x0200 got target, 0x0400 sent CDB, 0x0800 transferred data, 0x1000 got status, 0x2000 got sense, 0x4000 transfer complete | +| 14 | 2 | status flags: 0x0001 disconnected, 0x0002 synchronous, 0x0004 parity error, 0x0008 bus reset, 0x0010 device reset, 0x0020 aborted, 0x0040 timeout, 0x0080 negotiation | +| 16 | 2 | time | +| 18 | 2 | request sense length | +| 20 | 4 | residual byte count | +| 24 | 8 | reserved | +| 32 | 32 | sense data (auto request sense) | + +Completion status codes (1x00 family; IRIX `SCS_*`, FreeBSD `RQCS_*`): + +| Code | Meaning | Code | Meaning | +|---|---|---|---| +| 0x00 | complete | 0x12 | device reset message failed | +| 0x01 | incomplete (selection timeout when state flags lack "got target") | 0x13 | identify message failed | +| 0x02 | DMA error | 0x14 | unexpected bus free | +| 0x03 | transport error | 0x15 | data underrun (residual valid) | +| 0x04 | reset occurred | 0x16 | command underrun (IRIX) | +| 0x05 | aborted | 0x18 | queued transaction with disconnect not set | +| 0x06 | timeout | 0x19 | queued transaction to target routine | +| 0x07 | data overrun | 0x1A | queued command when queueing disabled | +| 0x08 | command overrun | 0x1B | invalid IOCB entry type | +| 0x09 | status overrun | 0x1C | queue full | +| 0x0A | bad message | 0x1D | SCSI phase skipped | +| 0x0B | no message out | 0x1E | auto request sense failed | +| 0x0C | extended identify failed | 0x1F | wide negotiation failed | +| 0x0D | initiator detected error rejected | 0x20 | sync negotiation failed | +| 0x0E | abort message failed | 0x21 | LVD bus error | +| 0x0F | message reject rejected | 0x22 | data CRC (IRIX) | +| 0x10 | NOP rejected | | | +| 0x11 | message parity error rejected | | | + +Drivers treat 0x1F/0x20 as "fall back to narrow/async and retry", 0x15 +and 0x07 as good with a residual, 0x01 without "got target" as selection +timeout, 0x04/0x05 as requiring a marker. + +### 4.5 Fast posting + +With firmware feature bit 0 set, a command that completed with good +status is reported as async event 0x8020 with the handle in MAILBOX1 +(low) and MAILBOX2 (high) instead of a status entry (events 0x8021/0x8022 +and 0x8031–0x8035 batch 32- and 16-bit handles on later firmware). Linux's +default NVRAM settings enable it; OpenBSD, FreeBSD (for 1x40) and IRIX +leave it off. An emulator must honour the bit if Linux is a target; it can +ignore it for IRIX. + +### 4.6 Asynchronous events + +Posted in MAILBOX0 with the semaphore held: + +| Event | Meaning | Extra | +|---|---|---| +| 0x8001 | SCSI bus reset seen | MAILBOX6 = channel (dual bus) | +| 0x8002 | system error (firmware crash) | MAILBOX1–3 = RISC state; IRIX re-initialises the chip | +| 0x8003 | request queue transfer error | | +| 0x8004 | response queue transfer error | | +| 0x8005 | request queue wake-up (space freed) | | +| 0x8006 | execution timeout reset | | +| 0x8007 | bus device reset | | +| 0x800A | extended message underrun | | +| 0x800B | SCAM interrupt | | +| 0x800C | bus hung after data overrun (recovery mode 1) | | +| 0x800D | bus reset by the ISP after data overrun (mode 2) | | +| 0x800E | bus mode transition (SE/HVD/LVD) | MAILBOX1/2 = mode | +| 0x800F | target transfer rate reduced | | +| 0x8010 | no memory | | +| 0x8020 | fast-post completion | MAILBOX1/2 = handle | +| 0x8021 / 0x8022 | RIO: one / two 32-bit handles | MAILBOX1/2, 6/7 | +| 0x8031–0x8035 | RIO: one to five 16-bit handles | MAILBOX1–5 | + +### 4.7 Target parameters and defaults + +SET TARGET PARAMETERS mailbox 2 flags (OpenBSD `QLW_TARGET_*`, Linux +`TP_*`, IRIX capability byte << 8): + +| Bit | Meaning | +|---|---| +| 0x8000 | disconnect allowed | +| 0x4000 | parity checking | +| 0x2000 | wide | +| 0x1000 | synchronous | +| 0x0800 | tagged queuing | +| 0x0400 | auto request sense | +| 0x0200 | stop queue on check condition | +| 0x0100 | renegotiate on error | +| 0x0080 | narrow (negotiate down) | +| 0x0040 | asynchronous (negotiate down) | +| 0x0020 | PPR (12160) | + +OpenBSD starts every target at `SAFE` (0xC500: disconnect, parity, auto +sense, renegotiate) plus narrow and async, and raises it to 0xFD00 after +INQUIRY; Linux starts at renegotiate | auto sense | tags | wide | parity | +disconnect; IRIX's defaults from `ql.h` are everything on (0xFD), +sync period 25 (100 ns) or 12 (Fast-20, 50 ns), offset 8, execution +throttle 255, 8 LUNs per target, 256 queue depth. Mailbox 3 carries the +sync period in the low byte (in 4 ns units: 25 = 10 MHz, 12 = 20 MHz) and +the offset in the high byte; 0 means asynchronous. + +Common chip-wide defaults: selection timeout 250 ms, tag age limit 8, +retry count/delay 0/0 (IRIX) or 0/1 (Linux), bus reset delay 3 s, async +data setup 6 clocks at 40 MHz / 8–9 at 60 MHz, both active-negation +enables on, both DMA burst enables on, clock rate 60 for a 1040. + +### 4.8 NVRAM + +Add-in QLA1040 boards carry a 128-byte serial EEPROM (93C46) reached +through register 0x0E. Reading word *n* (OpenBSD `qlw_read_nvram`, +FreeBSD `isp_rdnvram_word`): write SELECT, then SELECT|CLOCK; clock out +9 bits MSB first — the read opcode 6 in bits 8:6 and the 6-bit word +address — each bit as three writes (SELECT[|DATA_OUT], then with CLOCK, +then without); then clock in 16 result bits MSB first, sampling DATA_IN +after each SELECT|CLOCK write; finally write 0. The 1080 family uses a +10-bit request (opcode << 8 | address) for a 256-byte part. + +Layout of the 1040 image (FreeBSD `ISP_NVRAM_*`): + +| Byte | Field | +|---|---| +| 0–3 | "ISP " | +| 4 | version (OpenBSD requires ≥ 2) | +| 5 | bits 1:0 FIFO threshold, bit 2 BIOS disable, bit 3 HBA enable, bits 7:4 initiator ID | +| 6 | bus reset delay | +| 7 / 8 | retry count / delay | +| 9 | bits 3:0 async data setup, bit 4 REQ/ACK active negation, bit 5 data active negation, bit 6 data DMA burst, bit 7 command DMA burst | +| 10 | tag age limit | +| 11 | termination and clock bits (bit 3 "enable 60 MHz", bit 4 SCSI reset disable, bit 6 FIFO threshold 128) | +| 12–13 | selection timeout (LE) | +| 14–15 | max queue depth (LE) | +| 16 | bus size/type, clock speed, soft termination, flash present | +| 28 + 6·t | per target t: byte 0 flags (bit 0 renegotiate, 1 stop queue, 2 auto sense, 3 tags, 4 sync, 5 wide, 6 parity, 7 disconnect), byte 1 execution throttle, byte 2 sync period, byte 3 bits 3:0 sync offset, bit 4 device enable, bit 5 LUN disable | +| 127 | checksum: the byte sum of all 128 bytes is 0 | + +The Octane's chips have none. OpenBSD's probe reads garbage, fails the +"ISP " / checksum test, prints "nvram corrupt" and uses defaults, with +initiator ID 0 on sgi. Returning DATA_IN = 0 for every bit is enough. +Linux does not read a 1040's NVRAM at all (`driver_setup.no_nvram = 1`). + +## 5. How each OS programs it + +### 5.1 IRIX 6.5.30 `ql` (kernel) + +The driver's view of the registers is `ql.h`'s `ISP_REGS`, a struct of +`u_short`s laid out at **PCI offset XOR 2** ("These register definitions +assume a 32 bit PCI interface (for PIO) on a big endian machine. For 32 +bit PCI little endian, macros used to access them need to XOR the address +with 2"). So in the disassembly a `sh` to `0xC2(base)` is HCCR (0xC0), +`0x72(base)` is MAILBOX0, `0x7A(base)` is MAILBOX5, `0x0A(base)` is ICR and +`0x08(base)` is ISR. §8 explains why. + +Compile-time choices for IP30 (`ql.h`): `USE_IOSPACE` (BAR0), +`QL_SRAM_PARITY 1`, `BUS_RESET_ON_INIT 0`, `LOAD_VIA_DMA` (used for the +1080 family only), `PROBE_ALL_LUNS`, request and response depth 256, +`IOCB_SEGS 2`, `CONTINUATION_SEGS 5` (type 9 / 0xA entries), up to 254 +continuation entries per command. + +`qlattach`: map BAR0 with `pciio_pio_addr(dev, NULL, PCIIO_SPACE_WIN(0), +0, 256, &map, 0)`; set PCI command bit 0, latency timer 0x40, cache line +0x80; `pcibr_rrb_alloc`; interrupt via `pciio_intr_alloc(..., INT A)`; +then `ql_init_board` → `ql_init_isp` → `ql_download_fw` and +`ql_reset_interface`. + +`ql_init_board` identifies the chip from PCI vendor 0x1077 and device: +0x1020 → clock 60, one channel, firmware `risc_1040_code01` +(`risc_1040_code_addr01` = 0x1000, length 11143); 0x1240 → 60 MHz, two +channels; 0x1080 → 100; 0x1280 → 100, two channels; 0x1216 → 80, two +channels. It records the Bridge revision (`pcibr_asic_rev`) for the +`BRIDGE_B_PREFETCH_DATACORR_WAR` workaround. + +**`ql_download_fw`** (register offsets are PCI offsets): + +1. ICR (0x08) = 0x0005 (soft reset + RISC int enable); 30 µs. +2. HCCR = 0x1000; HCCR = 0x3000; 30 µs. Poll HCCR bit 0x40 clear, up to + 10000 × 100 µs; failure → error 11. +3. Set `AFLG_CHIP_DOWNLOAD_FIRMWARE | AFLG_CHIP_SRAM_PARITY_ENABLE` and run + `ql_enable_sram_parity(chip, check = 1)`: + - HCCR = 0xA100 (parity enable, bank 0). + - HCCR = 0x2000 (pause). With the RISC paused write R3 (0x86) = + 0xBEEF, R4 (0x88) = 0x8000, R8 (0x90) = 0x0016, R9 (0x92) = 0x0449, + RAR1 (0xA8) = 0x1000, CDMA_ADDR0..3 (0x2C, 0x2E, 0x30, 0x32) = 0x43A4, + 0x8421, 0x08C2, 0x097A. HCCR = 0x3000; 30 µs. + - MAILBOX1 = 0x0016, MAILBOX0 = 0x0002 (EXECUTE FIRMWARE **at address + 0x16**, i.e. a routine in the chip's boot ROM that initialises the + SRAM parity bits), HCCR = 0x5000. Poll MAILBOX0 == 0x4000 for up to + 4000 ms (`QL_COMPUTE_PARITY_MEMORY`). No interrupt is used. + - SEMA = 0, ISR = 0 (harmless), HCCR = 0x7000, HCCR = 0x3000; 30 µs. + - 32 × READ RAM WORD (5) at 0x1000, 0x1001, …; after each, read HCCR + and fail if bit 0x400 (parity error) is set or the command failed. + On failure: ICR = 5, HCCR = 0x1000, restore HCCR, HCCR = 0x7000, + 0x3000, print "Q-logic sram parity error on bank 0 hccr=%x" and + continue without parity (error 7 if the chip is unusable). + An emulator should answer EXECUTE FIRMWARE at 0x0016 with 0x4000, + stay in ROM mode, keep READ RAM WORD working, and never set HCCR bit + 0x400. +4. `ql_enable_intrs`: ICR = 0x0006. +5. MAILBOX REGISTER TEST with 0x1111, 0x2222, 0x3333, 0x4444, 0x5555, + 0x6666, 0x7777 in mailboxes 1–7 (8 in, 8 out). Failure → error 1. The + kernel only checks that the command completed. +6. HCCR = 0x2000; CONF1 (0x06) = 0x0044 (128-byte FIFO, burst); 100 µs; + SEMA = 0; HCCR = 0x3000; 30 µs. +7. For device 0x1020: WRITE RAM WORD (4) for each of the 11143 words, + mailbox 1 = 0x1000 + i, mailbox 2 = word (3 in, 3 out). Failure → + error 4. Other chips: copy the image to a bounce buffer, byte-swap it + 16-bit-wise (`munge16`), map it with `pciio_dmatrans_addr(…, + PCIBR_VCHAN1 | PCIIO_BYTE_STREAM | PCIIO_DMA_A64 | PCIIO_DMA_DATA)` and + issue LOAD RAM A64 (9) with mailbox 1 = 0x1000, 2/3 = address 31:0, + 4 = length, 6/7 = address 63:32. +8. EXECUTE FIRMWARE (2), mailbox 1 = 0x1000 (2 in, 1 out). Failure → + error 5. No VERIFY CHECKSUM. +9. If `showconfig`: ABOUT FIRMWARE (1 in, 3 out), print "Firmware + version: %d.%d.%d". + +**`ql_reset_interface(chip, bus_reset)`**, run at init and on every +recovery: + +1. Flush queued requests; zero `request_in/out`, `response_in/out`; mark + all 256 request entries free (handle word = 0xDEAD). +2. ICR = 0; HCCR = 0x2000; poll HCCR bit 0x20 set (10000 × 100 µs; error + 1). +3. ICR = 0x0001 (soft reset); 30 µs; HCCR = 0x1000; HCCR = 0x3000; 30 µs; + poll HCCR bit 0x40 clear (error 2). +4. Device 0x1020: CDMA_CONTROL (0x22) = 0x000E, DDMA_CONTROL (0x42) = + 0x000E. (1080 family: CONF1 |= 0x300, the banked DMA control + registers at 0x80/0xA0/0xC0 = 0xE, then OR 0x2 into the bank's + config registers and HCCR.) +5. Wait while MAILBOX0 == 0x0004 (busy), up to 1000 × 1 ms. +6. `ql_enable_sram_parity(chip, 0)`: HCCR = 0xA100 if parity was enabled. +7. CONF1 = 0x0044; SEMA = 0; HCCR = 0x3000; 30 µs; ICR = 0x0006. +8. EXECUTE FIRMWARE at 0x1000 again (error 3) — the image loaded earlier + must still be in RAM. +9. SET PCI CONTROL PARAMETERS (0x37): 1 = 2, 2 = 2 (error 4). +10. SET CLOCK RATE (0x34): 60 (error 5). +11. Zero the 16 KB response ring, map it with `pciio_dmatrans_addr(…, + PCIBR_VCHAN1 | PCIIO_BYTE_STREAM | PCIIO_DMA_A64 | PCIIO_DMA_CMD)` and + INIT RESPONSE QUEUE A64 (0x53): 1 = 256, 2/3 = address 31:0, 4 = 0, + 5 = response_in (0), 6/7 = address 63:32 (8 in, 8 out; error 6). +12. Same for the request ring with INIT REQUEST QUEUE A64 (0x52): 1 = + 256, 4 = request_in (0), 5 = 0 (error 7). +13. If `bus_reset` (from `ql_bus_reset_on_boot`, default off for IP30): + BUS RESET (0x18) per channel, 1 = that bus's reset delay, 2 = channel + (error 8). +14. SET INITIATOR SCSI ID (0x30): 1 = (channel << 7) | (hostid & 0xF), + per channel (error 9). +15. SET SELECTION TIMEOUT (0x31) 250/250 (error 10); SET RETRY COUNT + (0x32) (error 11); SET TAG AGE LIMIT (0x33) 8/8 (error 12); SET + ACTIVE NEGATION (0x35) 0x30/0x30 (error 13); SET ASYNC DATA SETUP + TIME (0x36) 8/8 (error 14); SET DATA OVERRUN RECOVERY MODE (0x5A) 2 + (error 15); SET RESET DELAY PARAMETERS (0x3A) (error 16); SET SYSTEM + PARAMETER (0x45) 1 (error 17); SET FIRMWARE FEATURES (0x4A) 0x0002 + for the 1040, 0x0322 for the 12160 (error 18). +16. Set `AFLG_CHIP_INITIALIZED`. + +Then `ql_probe_bus` does, per target, SET TARGET PARAMETERS (0x38) with +1 = target << 8 | channel << 15, 2 = capability flags << 8, 3 = offset << +8 | period, and SET DEVICE QUEUE PARAMETERS (0x39) with 1 = target << 8 | +channel << 15 | lun, 2 = 256, 3 = throttle, for each of 8 LUNs; then +INQUIRY commands through the request queue. GET TARGET PARAMETERS (0x28) +and ABORT DEVICE (0x16) are used later. + +**Mailbox command path** (`ql_mbox_cmd(chip, out[], nin, nout, cmd, +m1, m2, m3, m4, m5, m6, m7)`): writes mailboxes 7, 6, 5 (skipped for 0x38 +and 9), 4 (skipped for 0x32, 0x53, 0x38), 3, 2, 1, 0 in that order, then +HCCR = 0x5000. In interrupt, dump, watchdog or parity-check context it +polls: ISR bit 2, then SEMA bit 0; if the semaphore is clear it treats +the interrupt as a response-queue update (reads MAILBOX5, sets +`AFLG_CHIP_ASYNC_RESPONSE_IN`) and writes HCCR = 0x7000. 100 retries of +1 s; on each timeout it prints the ISR/SEMA/HCCR/ICR values and writes +HCCR = 0x7000. Otherwise it sleeps on a semaphore that +`ql_service_mbox_interrupt` posts. + +**Interrupt path** (`qlintr` → `ql_service_interrupt`): loop while ISR +bit 2: if SEMA bit 0, `ql_service_mbox_interrupt` — MAILBOX0 == 0x4000 +copies mailboxes 0–7 to the chip's save area and wakes the waiter; 0x8001 +→ "SCSI bus RESET seen", set the send-marker flag; 0x8002 → +"Unrecoverable SCSI bus state/condition (mbox1=0x%x mbox2=0x%x)" and +`ql_reset_interface`; 0x800E → record bus mode from MAILBOX1; others +print their message; then SEMA = 0. Then read MAILBOX5 (response in) and +walk status entries: `munge()` the 64 bytes, check header flags 0x0E +("Detected busy condition / a bad header / a bad payload"), require type +3 ("Invalid driver command issued" otherwise), decode the handle (channel += bit 12, target = bits 11:8, index = bits 7:0), write 0xDEAD into the +entry's handle, complete the request; finally write MAILBOX5 = new +response out and HCCR = 0x7000. + +**Request path** (`ql_start_scsi`): a type 9 entry (type at byte 0 after +munge, count, sequence, handle, LUN, target | channel << 7, CDB length, +control flags 0x20 read / 0x40 write plus tag bits 0x2/0x4/0x8 and +0x200/0x400/0x800 for forced async/sync/wide, timeout in seconds, segment +count, CDB, two 64-bit segments), type 0xA continuations with five +segments each, then MAILBOX4 = new request in-pointer. A SYNC ALL marker +(modifier 2) precedes the next command when the send-marker flag is set. +"Transfer length (0x%x) too large; spans more than %d continuation +entries" caps a command at 254 continuations. + +What IRIX never does on the Octane: read NVRAM, VERIFY CHECKSUM, LOAD RAM +by DMA (1040), enable fast posting, read CONF0 or RISC_PSR (it trusts the +PCI IDs), issue BUS RESET at boot. + +### 5.2 IRIX IP30 PROM 4.17 + +Only strings and embedded images were examined; the PROM's driver is in +the flash image twice (main and recovery halves). It carries the same +firmware 2.55 as the kernel (11190 words — a slightly different build) +and a 1335-word "ISP1020 Self Test Firmware Version 01.00" (image ID +0x102D). Messages show the sequence: "QLISP1040" probe; mailbox register +test ("ql_init_isp - mailbox compare error" — it checks the echo); +self-test load via WRITE RAM WORD ("QL: write data failure at %d word"), +EXECUTE FIRMWARE ("QL: EXECUTE FIRMWARE Command Failed.") and result +check ("SCSI self test failed (mbox val %x)."); then the 2.55 load by +DMA ("mailbox command for dma load"), a read-back compare ("Firmware +Data Mismatch [0x%x]: his = %x, mine = %x"), VERIFY CHECKSUM ("ISP +Firmware Checksum mismatch: theirs %x, mine = %x"), SET SELECTION TIMEOUT +("ql_init_isp - SET_SELECTION_TIMEOUT Command Failed"), BUS RESET +("qlreset - SCSI BUS RESET Command Failed", "ql_mbox_cmd - ISP SCSI bus +RESET seen"). What the self-test image is expected to leave in the +mailboxes is not known from the strings; the first run of the PROM +against an emulated chip (`IRIS_IP30_TRACE=1`) will show which value it +waits for after EXECUTE FIRMWARE of that image. Expect it to want 0x4000 +in MAILBOX0 and possibly a result word in MAILBOX1. + +### 5.3 Linux `qla1280` + +`qla1280_initialize_adapter`: SEMA = 0, HCCR 0x7000, HCCR 0x6000; +`qla1280_chip_diag` (ICR soft reset, wait up to 5 s; CONF1 = 0; HCCR = +0x1000|0x3000|0x9000; wait MAILBOX0 != 4; check product ID 0x4953 / +0x5020-or-0 / 0x2020 / 1; enable interrupts; register test with 0xAAAA +0x5555 0xAA55 0x55AA 0xA5A5 0x5A5A 0x2525). For a 1040: PIO load from +`qlogic/1040.bin` (`fw_data[2]` = 0x1000 start, code from byte 6); 1080+: +LOAD RAM A64 1000 words at a time through the request ring buffer. +VERIFY CHECKSUM, EXECUTE FIRMWARE. `qla1280_nvram_config` (defaults on a +1040): CONF1 = FIFO threshold << 4 | burst << 2 (threshold 3 → 0x30 on a +1040 unless hardware revision 4 = 1040A), CDMA/DDMA_CONF |= burst; SET +SYSTEM PARAMETER 0; SET CLOCK RATE **40** (`IS_ISP1x40`); SET FIRMWARE +FEATURES (fast posting on, sync backoff disabled → 0x21); SET RETRY COUNT +0/1; SET ASYNC DATA SETUP 6; SET ACTIVE NEGATION 0x30; SET DATA OVERRUN +RECOVERY 2; SET PCI CONTROL 2/2; SET TAG AGE LIMIT 8; SET SELECTION +TIMEOUT 250; per bus SET INITIATOR ID 7 and per target SET TARGET +PARAMETERS (renegotiate | auto sense | tags | wide | parity | disconnect, +sync disabled until INQUIRY) and 8 × SET DEVICE QUEUE (32, throttle 16). +INIT REQUEST/RESPONSE QUEUE A64 with 255/63 entries. BUS RESET per bus +(twice before declaring the bus dead). Commands are type 9/0xA on 64-bit +kernels (`QLA_64BIT_PTR`), type 1/2 otherwise; `qla1280_isr` handles +0x8020 fast-post completions (handle = MAILBOX2 << 16 | MAILBOX1). + +### 5.4 FreeBSD / NetBSD `isp` + +`isp_reset`: pause; read CONF0 for the revision and SXP_PINS_DIFF / +RISC_PSR for differential and Ultra; ICR soft reset; CDMA/DDMA CONTROL +clear; wait ICR bit 0 clear; CONF1 = 0; HCCR 0x1000; SEMA = 0; CONF1 = +burst (and FIFO unless 1040A); CDMA/DDMA_CONF burst; RISC_MTR = 0x1212 +(or 0x1313 with HCCR STEP for "fast MTTR" boards); HCCR 0x3000; request / +response pointers = 0 (mailboxes 4 and 5); HCCR 0x9000; wait MAILBOX0 != +4; NOP; register test (0xdead 0xbeef 0xffff 0xa5a5 0x5a5a); PIO load with +WRITE RAM WORD; VERIFY CHECKSUM; EXECUTE FIRMWARE; ABOUT FIRMWARE; GET +FIRMWARE STATUS (MAILBOX2 = max commands). `isp_scsi_init`: SET RETRY +COUNT, ASYNC DATA SETUP, ACTIVE NEGATION, TAG AGE LIMIT, SELECTION +TIMEOUT, per channel SET INITIATOR ID and SET TARGET PARAMETERS (safe +defaults, narrow+async when firmware was loaded) and SET DEVICE QUEUE for +8 LUNs, INIT RESPONSE then REQUEST QUEUE (32-bit forms on a 1040), SET +FIRMWARE FEATURES only for Ultra2+. NetBSD's `isp_pci.c` keeps 1040 DMA +segments from crossing a 16 MB boundary (`dbound = 1 << 24`) and leaves +RQSTYPE_A64 disabled. Interrupt: ISR & RISC_INT, SEMA & LOCK, MAILBOX0. + +### 5.5 OpenBSD `qlw` (ran on real Octanes) + +`qlw_attach`: if mailboxes 1–3 don't read "ISP ", HCCR release; HCCR +pause; `qlw_softreset` (ICR = 0; ICR = 1; wait clear; CONF1 = 0; HCCR +0x1000; SEMA = 0; HCCR 0x3000; mailboxes 4/5 = 0; ICR = 6; HCCR 0x9000; +NOP); defaults (CONF1 0x34, initiator **0 on sgi**, 7 elsewhere); NVRAM +read (fails on the Octane → defaults); PIO firmware load + VERIFY +CHECKSUM; EXECUTE FIRMWARE; 250 ms; ABOUT FIRMWARE; GET FIRMWARE STATUS; +CONF1 = config; CDMA/DDMA_CONF burst; SET FIRMWARE FEATURES only if LVD +notify wanted; SET CLOCK RATE 60; SET RETRY COUNT; SET ASYNC DATA SETUP +9; SET ACTIVE NEGATION; SET TAG AGE LIMIT; SET SELECTION TIMEOUT; per +bus SET INITIATOR ID and per target SET TARGET PARAMETERS (safe | narrow +| async) + 8 × SET DEVICE QUEUE; INIT REQUEST then RESPONSE QUEUE +(32-bit); BUS RESET per bus; wait reset delay. Commands are type 1/2 +with a SYNC ALL marker after a reset; no fast posting. + +## 6. Minimal emulation model + +What a device model has to do for the PROM, IRIX, Linux, the BSDs: + +**State**: PCI config (IDs, command, BARs, revision 5); 64K words of RISC +RAM (persistent across soft and RISC resets); the 16-bit registers; +incoming and outgoing mailboxes (8 each); SEMA; ICR/ISR; HCCR status bits +(host int, reset, pause, parity enables); "ROM mode" vs "firmware +running"; per-queue base address, length, in/out indices; per-target and +per-device parameters (store and echo); send-marker expectation +(optional). + +**Reset** (ICR bit 0 or HCCR 0x1000 → 0x3000): clear ISR/SEMA, outgoing +mailboxes = 0, 0x4953, 0x5020, 0x2020, 0x0001, back to ROM mode, queues +undefined, HCCR bit 0x40 reads clear (or clears after a few µs). Keep RAM. + +**HCCR**: 0x5000 → process the mailbox command (immediately or after a +short delay), set the host-int status bit until processed; 0x2000/0x3000 +toggle the pause bit; 0x7000 clears the RISC interrupt (ISR bit 2); +0xA000 stores the parity enables; 0x4000, 0x6000, 0x8000, 0x9000, 0xE000, +0xF000 accepted and ignored. + +**Mailbox completion**: write outgoing mailboxes, SEMA |= 1, ISR |= 4, +raise the PCI interrupt if ICR bits 1 and 2 are set. The host clears SEMA +and writes HCCR 0x7000. Do not post a response-queue interrupt while +SEMA bit 0 is held. + +**ROM mode**: commands 0–8, 9/0xA, 0x40–0x42 answer; EXECUTE FIRMWARE at +0x1000 (or any address other than 0x0016) switches to firmware mode; +EXECUTE at 0x0016 answers 0x4000 and stays in ROM mode; everything else +0x4001. + +**Firmware mode**: all SET/GET commands succeed and echo their inputs; +ABOUT FIRMWARE from the banner; GET FIRMWARE STATUS → 256; INIT queues +record base/length and reset the pointers (outgoing MAILBOX4/5 = +incoming); BUS RESET completes, then posts async 0x8001 (MAILBOX6 = +channel) after the delay; ABORT DEVICE completes outstanding commands of +that LUN with status 0x05. + +**Request processing**: on a write to incoming MAILBOX4 (and after an +INIT REQUEST QUEUE), read entries from the out index to the new in index +(wrap at length), handle types 1, 2, 4, 9, 0xA; for each command run the +SCSI command against the target model using the data segments (64-bit +addresses on the Octane), then post a type 3 status entry with the +handle, SCSI status, completion 0 (or 0x15 with the residual on underrun, +0x01 with state flags 0 for no such target), auto-sense data with bit 9 +of the SCSI status set when the target returned CHECK CONDITION; advance +the outgoing MAILBOX5 and raise ISR bit 2 (without SEMA). Advance the +outgoing MAILBOX4 as entries are consumed. Markers and unknown entry +types are consumed silently (or answered with flags 0x04 for bad header). + +**Timing**: IRIX waits up to 1 s per mailbox command, 4 s for the parity +routine, 1 s for MAILBOX0 to leave 4; Linux 5 s for reset bits; OpenBSD +polls for 400 ms during attach. Completing everything instantly is fine. + +**Things that bite**: mailbox 4/5 in/out duality; RAM surviving reset; +EXECUTE at 0x16; READ RAM WORD after load; the type 9 segment layout +(address low, address high, length); the status entry's sense-valid bit; +IRIX requiring type 3 entries only; the PROM comparing the register-test +echo and the checksum; Linux's SET CLOCK RATE 40; the Bridge byte swap +(next section) making every IRIX register access appear at offset ^ 2. + +## 7. Endianness summary + +- Register PIO: the chip is little-endian PCI. The Bridge is configured + by the PROM and IRIX so that a 16-bit access from the big-endian CPU to + byte address A reaches PCI address A ^ 2 with the value unchanged + (word-swapped, "data invariant"). That is why `ql.h` lays the struct out + at offset ^ 2, and why IRIX writes HCCR at 0xC2. Linux and OpenBSD + instead clear the Bridge's swap bits and swap in software, so they + address 0xC0 directly. The device model should be written against PCI + offsets and let the Bridge model do the swap. +- IOCBs and firmware DMA: little-endian in memory, IRIX swaps them in + software (`munge`, `munge16`) and maps the memory `PCIIO_BYTE_STREAM` + (no Bridge swap), so DMA bytes are exact. +- Firmware images in C sources and in `ql.o` are arrays of 16-bit words; + on disk `1040.bin` is little-endian words. The banner text is stored + big-endian within words (hence "C PORYGITH" when a little-endian file + is dumped as bytes). + +## 8. The Octane side: Bridge, DMA addresses, interrupts + +- **Location**: Bridge at xtalk widget 15, PCI devices 0 and 1. Their + registers are reached through the Bridge's per-device "devio" windows + (2 MB each for devices 0 and 1) in PCI I/O space; IRIX maps BAR0 with + `PCIIO_SPACE_WIN(0)`. +- **Byte swapping**: the Bridge has per-device PIO swap (`BRIDGE_DEV_DEV_SWAP`, + bit 13 of the device(x) register) and large-window swap bits + (`BRIDGE_CTRL_IO_SWAP`/`MEM_SWAP`, bits 23/22 of widget control), plus + DMA swap bits `BRIDGE_DEV_SWAP_DIR` (bit 19, direct window) and + `BRIDGE_DEV_SWAP_PMU` (bit 20, translated/ATE). IRIX's register + struct shows PIO swapping is on for the QL devices. OpenBSD clears the + large-window swap bits ("IOC serial console relies on this") and uses + software-swapping bus_space tags for the QLogic but not for IOC3/IOC4/ + RAD; it sets the DMA swap bits for everything but those SGI parts. + Linux clears the large-window swap bits, sets `SWAP_DIR` for every slot + and then clears it again for the IOC3. For the IOC3 the natural mode is + no swap; for the QLogic it is swap. +- **DMA addresses**: IRIX maps queues and data with `PCIIO_DMA_A64`, so + the ISP issues 64-bit PCI addresses (dual address cycles). The Bridge + treats bits 63:48 as attributes and 47:0 as the xtalk address: bits + 63:60 target widget (8 = HEART, i.e. memory; Linux uses + `masterwid << 60`), bit 59 prefetch, bit 58 precise, bit 57 virtual + channel 1 (IRIX passes `PCIBR_VCHAN1` for the QL, so this is set), bit + 56 barrier (Linux sets `PCI64_ATTR_BAR`), bit 55 byte swap (clear for + the QL: `PCIIO_BYTE_STREAM`), bits 55:48 "RMF". The address is the + physical address as the CPU sees it (Linux: `dma = baddr + paddr`, + `paddr = dma & ~(0xff << 56)`). The 32-bit direct window + (0x8000_0000–0xFFFF_FFFF on PCI, via the Bridge direct-map register) + and the ATE-translated window (0x4000_0000–0x7FFF_FFFF) are what + OpenBSD and Linux's IOC3 use; the IRIX QL driver does not. +- **Read response buffers**: both the PROM and IRIX allocate Bridge RRBs + for the QL slots and fail the controller if they cannot; the Bridge + model needs working `even/odd response` buffer registers. +- **Interrupts**: pin A of devices 0 and 1 → Bridge interrupt bits 0 and + 1 → the Bridge's interrupt destination (HEART), as configured by the + PROM/IRIX in the Bridge INT_ADDR/INT_ENABLE registers. IRIX asks for + one line (`pciio_intr_alloc(dev, desc, 1, …)`). +- **Bus type**: `PINS_DIFF` bit 8 clear (single-ended) for both + controllers; initiator ID 0; three disks at IDs 1–3 on controller 0 in + the reference machine. + +## 9. Open items + +- The real chips' CONF0, ID_LO/HI, RISC_HRL and SXP_PART_ID values have + not been captured from the Octane (needs a user-space PCI PIO on the + running machine, or a trace from the PROM once the emulator gets that + far). The tables give the values the drivers expect. +- The PROM self-test firmware's result protocol (§5.2). +- What IRIX passes in SET RESET DELAY PARAMETERS and the exact 12160 + branches are not decoded; neither matters for a 1040 model. +- Firmware timing (reselection, tag queueing order) is the firmware's + business; a model that completes commands in order is enough for IRIX.