COSC 6385 Computer Architecture - Tomasulos Algorithm Fall 2008 Analyzing a short code-sequence DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 1
Analyzing a short code-sequence 3 True data dependencies DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 Analyzing a short code-sequence 3 True data dependencies DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 2
Analyzing a short code-sequence 3 True data dependencies DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 Analyzing a short code-sequence Anti-dependencies (WAR hazards) DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 3
Analyzing a short code-sequence Output dependency (WAW DIV.D F0, F2, F4 hazard) ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 Analyzing a short code-sequence DIV.D F0,F2, F4 ADD.D S, F0, F8 S.D S, 0(R1) SUB.D T, F10, F14 MUL.D F6,F10, T Renaming some registers can remove the WAR and WAW hazards Any subsequent use of F8 must be replaced by T 4
Tomasulo s Algorithm Register renaming is provided by reservation stations Buffer the operands of instructions waiting to being issued Fetches an operand as soon as available Eliminates the need to get an operand from register Pending instructions designate the reservation station providing the input For overlapping successive writes: only the last one will be executed Tomasulo s Algorithm Typically more reservation stations than registers Hazard detection is distributed (instead of centralized as in the Scoreboard) Results are passed directly from reservation stations to functional units using a common data bus (CDB) Each reservation station holds the opcode for the pending instruction and either operand values or names of reservation stations that will provide them Load and store buffers hold data and addresses for memory access 5
From instruction unit Instruction queue FP registers Common data bus Store buffers LOAD-STORE OPERATIONS Address unit Load buffers FP OPERATIONS Data Address 4 3 2 1 Reservation stations 4 3 2 1 Memory unit FP adders FP multipliers Tomasulo s Algorithm Load store buffers: Hold components of effective address Hold destination memory address ( = effective address) Hold value 6
Tomasulo s Algorithm Only three steps per instruction each step can take an arbitrary number of cycles Issue: get next instruction from FIFO instruction queue Search matching empty reservation station If found: issue instruction with operand values If not found: structural hazard-> instruction stalls If operands not in register: keep track of functional units producing operands Tomasulo s Algorithm Execute: If operands not available: monitor common data bus When all operands available: execute Write result: Write data on CDB and from there into registers 7
Data fields for reservation stations Q p : operation to perform on source operands S1 and S2 Q j, Q k : reservation stations producing the operands V j, V k : value for each operand A: holds information for memory address calculation (immediate field, effective address) Busy: indicates occupied functional units/reservation stations Q i : number of the reservation station who will produce the data to be stored in this register The same example as for scoreboarding L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Following slides are based on a lecture by Jelena Mirkovic, University of Delaware http://www.cis.udel.edu/~sunshine/courses/f04/cis662/class12.pdf Assumption: ADD and SUB take 2 clock cycles MULT takes 10 clock cycle DIV takes 40 clock cycles 2 Load/Store, 3 ADD and 2 Mult functional units/reservation stations 8
Time=1 Issue first load L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Load Regs[R2] 34 Add2 Mult1 Time=2 First load calc. address. Second load issued L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Load Regs[R2] +34 Yes Load Regs[R3] 45 Add2 Mult1 9
Time=3 First load read from mem. Second load calc address. Mult is issued L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Load Regs[R2]+34 Yes Load Regs[R3] +45 Add2 Mult1 Yes Mult Regs[F4] Mult1 Time=4 First load write res. Second load read mem. Mult stalled, Sub issued L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Load Regs[R3]+45 Yes Sub Mem[34+Regs[R2]] Add2 Mult1 Yes Mult Regs[F4] Mult1 10
Time=5 Second load write res. Mult stalled, Sub stalled, Div. issued L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Sub Mem[34+Regs[R2]] Mem[45+Regs[R3]] Add2 Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Time=6 Mult executes (1/10), Sub executes (1/2), Div. stalled, Add issued L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Sub Mem[34+Regs[R2]] Mem[45+Regs[R3]] Add2 Yes Add Mem[45+Regs[R3]] Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Add2 11
Time=7 Mult executes (2/10), Sub executes (2/2), Div. stalled, Add stalled L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Yes Sub Mem[34+Regs[R2]] Mem[45+Regs[R3]] Add2 Yes Add Mem[45+Regs[R3]] Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Add2 Time=8 Mult executes (3/10), Sub writes res., Div. stalled, Add stalled L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Yes Add Mem[34+Regs[R2]]- Mem[45+Regs[R3]] Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Mem[45+Regs[R3]] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Add2 12
Time=9 Mult executes (4/10), Div. stalled, Add executes (1/2) L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Yes Add Mem[34+Regs[R2]]- Mem[45+Regs[R3]] Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Mem[45+Regs[R3]] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Add2 Time=10 Mult executes (5/10), Div. stalled, Add executes (2/2) L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Yes Add Mem[34+Regs[R2]]- Mem[45+Regs[R3]] Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Mem[45+Regs[R3]] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Add2 13
Time=11 Mult executes (6/10), Div. stalled, Add writes result L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Mult1 Yes Mult Mem[45+Regs[R3]] Regs[F4] Yes Div Mem[34+Regs[R2]] Mult1 Mult1 Time=16 Mult writes result, Div. stalled L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Mult1 Yes Div Mem[45+Regs[R3]] * Regs[F4] Mem[34+Regs[R2]] 14
Time=17 Div. Executed (1/40) L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Mult1 Yes Div Mem[45+Regs[R3]] * Regs[F4] Mem[34+Regs[R2]] Time=57 Div. Writes result L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 ADD.D F6, F8, F2 Add2 Mult1 15
Some remarks To preserve exception behavior, no instruction is allowed to initiate execution until all branches preceding the instruction have completed Load and store can be executed in different order if they access different addresses Not easy to verify, since 100(R3) can point to the same effective address as 0(R5)! -> A load must wait for any uncompleted stores to the same effective memory address -> A store must wait until there are no unexecuted loads/stores to the same memory address Some remarks (II) Effective memory address calculation has to be executed in order For a load operation: Calculate effective memory address Check for conflicts with all active (=pending) store buffers If conflict: load stalls Bypassing memory and taking data from the store buffer directly to the load buffer often done Else: execute load For a store operation: Similarly checking for conflicts with both active load and store buffers 16
A loop based example Loop: LD MULTD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 F4, 0(R1) R1, R1,#8 R1, Loop This time assume Multiply takes 4 clocks Assume 1st load takes 8 clocks total (1 effective address + 7 mem. Access) (L1 cache miss), 2nd load takes 1 clock (hit) To be clear, will show clocks for SUBI, BNEZ Reality: integer instructions ahead of Fl. Pt. Instructions Show 2 iterations Slide based on a lecture by David A. Patterson, University of California, Berkley http://www.cs.berkeley.edu/~pattrsn/252s01 Time=1 Issue first load L.D F0, 0(R1) 1 MUL.D F4, F0, F2 S.D F4, 0(R1) L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1] 0 Store1 Store2 Mult1 17
Time=2 first load effective address calc., Issue mult L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1] +0 Store1 Store2 Mult1 Yes Mult Regs[F2] Mult1 Time=3 first load mem. access(1/7), mult stalled, Issue store L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1]+0 Store1 Yes Store Regs[R1] Mult1 0 Store2 Mult1 Yes Mult Regs[F2] Mult1 18
Time=4 first load ex (2/7)., mult stall, store eff. addr, Calc SUBI (not shown) L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1]+0 Store1 Yes Store Regs[R1] Mult1 +0 Store2 Mult1 Yes Mult Regs[F2] Mult1 Time=5 first load exec (3/7)., mult stall, store stall, BNEZ (not shown) L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1]+0 Store1 Yes Store Mult1 Regs[R1] +0 Store2 Mult1 Yes Mult Regs[F2] Mult1 19
Time=6 first load exec (4/7)., mult stall, store stall, issue load L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 MUL.D F4, F0, F2 S.D F4, 0(R1) Yes Load Regs[R1]+0 Yes Load Regs[R1] 0 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Mult1 Yes Mult Regs[F2] Mult1 Time=7 first load ex (5/7)., mult stall, store stall, load2 eff. Add., issue mult2 L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 MUL.D F4, F0, F2 7 S.D F4, 0(R1) Yes Load Regs[R1]+0 Yes Load Regs[R1] +0 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Mult1 Yes Mult Regs[F2] Yes Mult Regs[F2] 20
Time=8 first load ex (6/7)., mult, store, mult2 stall, load2 ex., issue store2 L.D F0, 0(R1) 1 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 MUL.D F4, F0, F2 7 S.D F4, 0(R1) 8 Yes Load Regs[R1]+0 Yes Load Regs[R1]+0 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1] 0 Mult1 Yes Mult Regs[F2] Yes Mult Regs[F2] Time=9 first load exec (7/7)., mult, store, mult2 stall, load2 exec., store2 L.D F0, 0(R1) 1 9 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 MUL.D F4, F0, F2 7 S.D F4, 0(R1) 8 Yes Load Regs[R1]+0 Yes Load Regs[R1]+0 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1] +0 Mult1 Yes Mult Regs[F2] Yes Mult Regs[F2] 21
Time=10 first load write res. mult, store, mult2 stall, load2 finish, store2 stal L.D F0, 0(R1) 1 9 10 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 10 MUL.D F4, F0, F2 7 S.D F4, 0(R1) 8 Yes Load Regs[R1]+0 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1]+0 Mult1 Yes Mult Mem[] Regs[F2] Yes Mult Regs[F2] Time=11 L.D F0, 0(R1) 1 9 10 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 10 11 MUL.D F4, F0, F2 7 S.D F4, 0(R1) 8 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1]+0 Mult1 Yes Mult Mem[] Regs[F2] Yes Mult Mem[] Regs[F2] Load 2 write res, Mult1 (1/4), mult2, store1, store2 stalled 22
Time=14 L.D F0, 0(R1) 1 9 10 MUL.D F4, F0, F2 2 14 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 10 11 MUL.D F4, F0, F2 7 S.D F4, 0(R1) 8 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1]+0 Mult1 Yes Mult Mem[] Regs[F2] Yes Mult Mem[] Regs[F2] Mult1 (4/4), (3/4), store1, store2 stalled Time=15 L.D F0, 0(R1) 1 9 10 MUL.D F4, F0, F2 2 14 15 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 10 11 MUL.D F4, F0, F2 7 15 S.D F4, 0(R1) 8 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1]+0 Mult1 Yes Mult Mem[] Regs[F2] Mult1 write res., (4/4), store1 exec, store2 stalled 23
Time=16 store1, store2 exec L.D F0, 0(R1) 1 9 10 MUL.D F4, F0, F2 2 14 15 S.D F4, 0(R1) 3 L.D F0, 0(R1) 6 10 11 MUL.D F4, F0, F2 7 15 16 S.D F4, 0(R1) 8 Store1 Yes Store Mult1 Regs[R1]+0 Store2 Yes Store Regs[R1]+0 Mult1 Tomasulo s Algorithm Please note: F0 never sees data from the first load Register File completely detached from computation First and Second iteration overlap completely Assuming two Mult units, we could not have issued a third mult operation for the next iteration of the loop -> no third store instruction could be issued In order issue, out-of-order execution, out-of-order completion Slide based on a lecture by David A. Patterson, University of California, Berkley http://www.cs.berkeley.edu/~pattrsn/252s01 24
Why can Tomasulo overlap iterations of loops? Register renaming Multiple iterations use different physical destinations for registers (dynamic loop unrolling). s Permit instruction issue to advance past integer control flow operations Also buffer old values of registers - totally avoiding the WAR stall that we saw in the scoreboard. Other perspective: Tomasulo building data flow dependency graph on the fly. Slide based on a lecture by David A. Patterson, University of California, Berkley http://www.cs.berkeley.edu/~pattrsn/252s01 25