Hakim Weatherspoon CS 3410 Computer Science Cornell University

Transcription

1 Hakim Weatherspoon CS 3410 Computer Science Cornell University The slides are the product of many rounds of teaching CS 3410 by Professors Weatherspoon, Bala, Bracy, McKee, and Sirer.

2 memory inst register file alu PC new pc offset target imm control extend =? cmp addr d in d out memory

3 Advantages Single cycle per instruction make logic and clock simple Disadvantages Since instructions take different time to finish, memory and functional unit are not efficiently utilized Cycle time is the longest delay Load instruction Best possible CPI is 1 (actually < 1 w parallelism) However, lower MIPS and longer clock period (lower clock frequency); hence, lower performance

4 Advantages Better MIPS and smaller clock period (higher clock frequency) Hence, better performance than Single Cycle processor Disadvantages Higher CPI than single cycle processor Pipelining: Want better Performance want small CPI (close to 1) with high MIPS and short clock period (high clock frequency)

5 Parallelism Pipelining Both!

6 Alice Bob They don t always get along

7

8 Saw Drill Glue Paint

9 N pieces, each built following same sequence: Saw Drill Glue Paint

10 Alice owns the room Bob can enter when Alice is finished Repeat for remaining tasks No possibility for conflicts

11 time Latency: Elapsed Time for 4 Alice: hours/task 4 Throughput: Elapsed Time for 1 Bob: task/4 4 hrs Concurrency: Total elapsed time: 1 4*N Can we do better? CPI = 4

12 Partition room into stages of a pipeline Dave Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep

13 Partition room into stages of a pipeline Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

14 Partition room into stages of a pipeline Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

15 Partition room into stages of a pipeline Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

16 Partition room into stages of a pipeline Dave Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

17 Partition room into stages of a pipeline Alice Alice Alice Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

18 time Latency: 4 hrs/task Throughput: 1 task/hr Concurrency: 4 CPI = 1

19 Time What if drilling takes twice as long, but gluing and paint take ½ as long? Latency: Throughput: CPI =

20 Time Done: 4 cycles Done: 6 cycles Done: 8 cycles What if drilling takes twice as long, but gluing and paint take ½ as long? Latency: 4 cycles/task Throughput: 1 task/2 cycles CPI = CPI = 2

21 Principle: Throughput increased by parallel execution Balanced pipeline very important Else slowest stage dominates performance Pipelining: Identify pipeline stages Isolate stages from each other Resolve pipeline hazards (next lecture)

22 Single Cycle vs Pipelined Processor

23 Single-cycle insn0.fetch, dec, exec insn1.fetch, dec, exec Pipelined insn0.fetch insn0.dec insn1.fetch insn0.exec insn1.dec insn1.exec 23

24 5-stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 24

25 Review: Single cycle processor memory inst register file alu PC new pc offset target imm control extend =? cmp addr d in d out memory

26 memory inst register file alu +4 addr PC control d in d out memory new pc imm extend compute jump/branch targets Instruction Fetch Instruction Decode Execute Memory Write- Back

27 memory register file alu +4 addr PC new pc control extend compute jump/branch targets d in d out memory Fetch Decode Execute Memory WB

28 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write- Back IF/ID ID/EX EX/MEM MEM/WB

29 Cycle add nand lw add sw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Latency: 5 cycles Throughput: 1 insn/cycle Concurrency: 5 CPI = 1 29

30 Break datapath into multiple cycles (here 5) Parallel execution increases throughput Balanced pipeline very important Slowest stage determines clock rate Imbalance kills performance Add pipeline registers (flip-flops) for isolation Each stage begins by reading values from latch Each stage ends by writing values to latch Resolve hazards 30

31 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write- Back IF/ID ID/EX EX/MEM MEM/WB

32 Stage Fetch Decode Execute Memory Perform Functionality Use PC to index Program Memory, increment PC Decode instruction, generate control signals, read register file Perform ALU operation Compute targets (PC+4+offset, etc.) in case this is a branch, decide if branch taken Perform load/store if needed, address is ALU result Latch values of interest Instruction bits (to be decoded) PC + 4 (to compute branch targets) Control information, Rd index, immediates, offsets, register values (Ra, Rb), PC+4 (to compute branch targets) Control information, Rd index, etc. Result of ALU operation, value in case this is a store instruction Control information, Rd index, etc. Result of load, pass result from execute Writeback Select value, write to register file 32

33 instruction memory addr mc +4 PC new pc - PC+4 - pc-rel (PC-relative); e.g. BEQ, BNE - pc-abs (PC absolute); e.g. J and JAL. (PC+4) target 00 - pc-reg (PC registers); e.g. JR

34 instruction memory addr mc PC = read word inst PC+4 Rest of pipeline new pc pc-reg pc-relpc-abs pc-sel IF/ID

35 instruction memory addr mc PC = read word inst PC+4 Rest of pipeline pc-reg pc-relpc-abs PC+4 pc-reg (PC registers: JR) pc-rel (PC-relative: BEQ, BNE) pc-abs (PC absolute: J and JAL) pc-sel IF/ID 36

36 Stage 1: Instruction Fetch inst ctrl PC+4 WE Rd D register file A B Ra Rb B A imm PC+4 Rest of pipeline IF/ID ID/EX

37 result Stage 1: Instruction Fetch inst ctrl PC+4 decode WE Rd D register file A B Ra Rb dest extend B A imm PC+4 Rest of pipeline IF/ID ID/EX

38 ID/EX EX/MEM ctrl Stage 2: Instruction Decode ctrl PC+4 imm B A B D Rest of pipeline alu target

39 ID/EX EX/MEM pcreg pcsel branch? Stage 2: Instruction Decode PC+4 imm B A pcrel pcabs B D ctrl Rest of pipeline ctrl + alu target

40 Stage 3: Execute B D target d in addr memory d out mc ctrl M D Rest of pipeline ctrl EX/MEM MEM/WB

41 pcsel pcreg branch? Stage 3: Execute B D target pcrel pcabs d in addr ctrl M D Rest of pipeline ctrl memory d out mc EX/MEM MEM/WB

42 ctrl Stage 4: Memory M D MEM/WB

43 result Stage 4: Memory ctrl M D dest MEM/WB

44 inst mem PC +4 inst PC+4 Rd A D B Ra Rb B A Rt Rd PC+4 imm OP B D OP Rd addr d in d out mem M D OP Rd IF/ID ID/EX EX/MEM MEM/WB 49

45 Consider a non-pipelined processor with clock period C (e.g., 50 ns). If you divide the processor into N stages (e.g., 5), your new clock period will be: A. C B. N C. less than C/N D. C/N E. greater than C/N 50

46 Consider a non-pipelined processor with clock period C (e.g., 50 ns). If you divide the processor into N stages (e.g., 5), your new clock period will be: A. C B. N C. less than C/N D. C/N E. greater than C/N 51

47 Pipelining is a powerful technique to mask latencies and increase throughput Logically, instructions execute one at a time Physically, instructions execute in parallel Instruction level parallelism Abstraction promotes decoupling Interface (ISA) vs. implementation (Pipeline)

48 Instructions same length 32 bits, easy to fetch and then decode 3 types of instruction formats Easy to route bits between stages Can read a register source before even knowing what the instruction is Memory access through lw and sw only Access memory after ALU 53

49 5-stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 54

50 add nand lw add sw r3 r1, r2 r6 r4, r5 r4 20(r2) r5 r2, r5 r7 12(r3) Assume 8-register machine 55

51 M U X PC 4 + PC+4 instruction rega regb Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R7 0 extend PC+4 vala valb imm Rd Rt op M U X M U X A L U target ALU result valb dest op Data mem ALU result mdata IF/ID ID/EX EX/MEM MEM/WB dest op M U X data dest

52 At time 1, Fetch add r3 r1 r2 4 0 extend data dest 0 0 M U X Time: 0 IF/ID ID/EX EX/MEM MEM/WB

53 add M U X 8 PC 4 4 Fetch: add Time: 1/ add Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend / / 36 / / 03 / 02 / add nop M U X M U X A L U nop Data mem IF/ID ID/EX EX/MEM MEM/WB nop M U X data dest

54 nand add M U X 12 PC 8 4 Fetch: nand Time: 2/ nand Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend / / 18 / add 36 9 M U X A L U / 36 M 3 U / 25 / 0 3 / nand X 0 / 0 45 / add nop Data mem IF/ID ID/EX EX/MEM MEM/WB / 0 4 / nop M U X data dest

55 lw 4 20(2) nand add M U X PC 12 4 Fetch: lw 4 20(2) Time: 3/ lw 4 20(2) 4 5 Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend nand nand (18 7) 18 = = = / 18 / 9 7 M U X M U X A L U add Data mem / 0 45 IF/ID ID/EX EX/MEM MEM/WB / 4 8 / -3 / 9 7 / 3 6 / nand 0 / 0 3 / add nop M U X data dest

56 add lw 4 20(2) nand add M U X 20 PC 16 4 Fetch: add Time: add Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend lw M U X M U X A L U nand 45 3 Data mem IF/ID ID/EX EX/MEM MEM/WB add M U X data dest

57 sw 7 12(3) add lw 4 20 (2) nand add M U X 24 PC 20 4 Fetch: sw 7 12(3) Time: sw 7 12(3) 2 5 Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend add M U X M U X A L U lw -3 6 Data mem nand IF/ID ID/EX EX/MEM MEM/WB 45 3 M U X data dest

58 sw 7 12(3) add lw 4 20(2) nand M U X 28 PC 24 4 No more instructions Time: Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend sw M U X M U X A L U add 29 4 Data mem IF/ID ID/EX EX/MEM MEM/WB lw -3 6 M U X data dest

59 nop nop sw 7 12(3) add lw 4 20(2) M U X 32 PC 28 4 No more instructions Time: 7 + Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend M U X M U X A L U sw Data mem IF/ID ID/EX EX/MEM MEM/WB add 99 4 M U X data dest

60 nop nop nop sw 7 12(3) add M U X 36 PC Register file R0 R1 R2 R3 R4 R5 R6 R extend M U X A L U Data mem M U X data dest No more instructions Time: 8 Bits Bits Bits M U X IF/ID ID/EX EX/MEM MEM/WB 7 sw 5 Slides thanks to Sally McKee

61 nop nop nop nop sw 7 12(3) M U X 40 PC Register file R0 R1 R2 R3 R4 R5 R6 R extend M U X A L U Data mem M U X data dest No more instructions Bits Bits Bits M U X Time: 9 IF/ID ID/EX EX/MEM MEM/WB

62 Pipelining is great because: A. You can fetch and decode the same instruction at the same time. B. You can fetch two instructions at the same time. C. You can fetch one instruction while decoding another. D. Instructions only need to visit the pipeline stages that they require. E. C and D 67

63 Pipelining is great because: A. You can fetch and decode the same instruction at the same time. B. You can fetch two instructions at the same time. C. You can fetch one instruction while decoding another. D. Instructions only need to visit the pipeline stages that they require. E. C and D 68

64 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write- Back IF/ID ID/EX EX/MEM MEM/WB

65 5-stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 70

66 Correctness problems associated w/processor design 1. Structural hazards Same resource needed for different purposes at the same time (Possible: ALU, Register File, Memory) 2. Data hazards Instruction output needed before it s available 3. Control hazards Next instruction PC unknown at time of Fetch 71

67 Dependence: relationship between two insns Data: two insns use same storage location Control: 1 insn affects whether another executes at all Not a bad thing, programs would be boring otherwise Enforced by making older insn go before younger one Happens naturally in single-/multi-cycle designs But not in a pipeline Hazard: dependence & possibility of wrong insn order Effects of wrong insn order cannot be externally visible Hazards are a bad thing: most solutions either complicate the hardware or reduce performance 72

68 Data Hazards register file (RF) reads occur in stage 2 (ID) RF writes occur in stage 5 (WB) RF written in ½ half, read in second ½ half of cycle x10: add r3 r1, r2 x14: sub r5 r3, r4 1. Is there a dependence? 2. Is there a hazard? A) Yes B) No C) Cannot tell with the information given. 73

69 Data Hazards register file (RF) reads occur in stage 2 (ID) RF writes occur in stage 5 (WB) RF written in ½ half, read in second ½ half of cycle x10: add r3 r1, r2 x14: sub r5 r3, r4 1. Is there a dependence? 2. Is there a hazard? A) Yes for both B) No C) Cannot tell with the information given. 74

70 Which of the following statements is true? A. Whether there is a data dependence between two instructions depends on the machine the program is running on. B. Whether there is a data hazard between two instructions depends on the machine the program is running on. C. Both A & B D. Neither A nor B 75

71 Which of the following statements is true? A. Whether there is a data dependence between two instructions depends on the machine the program is running on. B. Whether there is a data hazard between two instructions depends on the machine the program is running on. C. Both A & B D. Neither A nor B 76

72 time Clock cycle add r3, r1, r2 IF ID MEM WB sub r5, r3, r4 IF ID MEM WB lw r6, 4(r3) IF ID MEM WB or r5, r3, r5 IF ID MEM WB sw r6, 12(r3) IF ID MEM WB

73 add r3, r1, r2 sub r5, r3, r4 lw r6, 4(r3) or r5, r3, r5 How many data hazards due to r3 only A) 1 B) 2 C) 3 D) 4 E) 5 sw r6, 12(r3)

74 time Clock cycle backwards arrows require time travel add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 79

75 time Clock cycle add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 80

76 time Clock cycle add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 81

77 Data Hazards register file reads occur in stage 2 (ID) register file writes occur in stage 5 (WB) next instructions may read values about to be written i.e. add r3, r1, r2 sub r5, r3, r4 How to detect?

78 Detecting Data Hazards inst mem PC +4 inst PC+4 IF/ID Rd D A B Ra Rb IF/ID.Ra 0 && (IF/ID.Ra==ID/Ex.Rd IF/ID.Ra==Ex/M.Rd IF/ID.Ra==M/W.Rd) sub r5,r3,r4 OP Rt Rd PC+4 imm B A add r3, r1, r2 B D OP Rd d in addr d out mem M D OP Rd ID/EX EX/MEM MEM/WB

79 Detecting Data Hazards inst mem PC +4 inst PC+4 Rd D A B Ra Rb detect hazard OP Rt Rd PC+4 imm B A B D OP Rd addr d in d out mem M D OP Rd IF/ID ID/EX EX/MEM MEM/WB

80 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards.

81 What to do if data hazard detected?

82 What to do if data hazard detected? A) Wait/Stall B) Reorder in Software (SW) C) Forward/Bypass D) All the above E) None. We will use some other method

83 1. Do Nothing Change the ISA to match implementation Hey compiler: don t create code w/data hazards! (We can do better than this) 2. Stall Pause current and subsequent instructions till safe 3. Forward/bypass Forward data value to where it is needed (Only works if value actually exists already) 89

84 How to stall an instruction in ID stage prevent IF/ID pipeline register update stalls the ID stage instruction convert ID stage instr into nop for later stages innocuous bubble passes through pipeline prevent PC update stalls the next (IF stage) instruction

85 add r3, r1, r2 sub inst r5, r3, r5 or r6, r3, r4 mem add r6, r3, r8 PC +4 WE=0 inst PC+4 IF/ID Rd D A B Ra Rb detect hazard If detect hazard MemWr=0 RegWr=0 OP Rt Rd PC+4 imm B A B D OP Rd d in addr d out mem M D OP Rd ID/EX EX/MEM MEM/WB

86 time Clock cycle add r3, r1, r2 sub r5, r3, r5 or r6, r3, r4 add r6, r3, r8

87 time r3 = 10 add r3, r1, r2 r3 = 20 sub r5, r3, r5 Clock cycle IF ID Ex M W 3 Stalls IF ID ID ID ID Ex M W or r6, r3, r4 IF IF IF IF ID Ex M add r6, r3, r8 IF ID Ex

88 inst mem +4 inst D rd ra rb A B A B D B data mem D M PC (MemWr=0 RegWr=0) nop Op WE Rd Op WE Rd Op WE Rd sub r5,r3,r5 add r3,r1,r2 or r6,r3,r4 (WE=0) /stall NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) STALL CONDITION MET

89 inst mem PC +4 or r6,r3,r4 inst D rd ra rb A B (MemWr=0 RegWr=0) nop sub r5,r3,r5 (WE=0) /stall A B Rd WE Op NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) (MemWr=0 RegWr=0) nop D B Rd WE Op data mem add r3,r1,r2 STALL CONDITION MET D M Rd WE Op

90 inst mem +4 inst D rd ra rb A B A B D B data mem D M PC (MemWr=0 RegWr=0) nop Rd WE Op (MemWr=0 RegWr=0) Rd WE Op (MemWr=0 RegWr=0) sub r5,r3,r5 nop nop Rd WE Op add r3,r1,r2 or r6,r3,r4 (WE=0) /stall NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) STALL CONDITION MET

91 time r3 = 10 add r3, r1, r2 r3 = 20 sub r5, r3, r5 Clock cycle IF ID Ex M W 3 Stalls IF ID ID ID ID Ex M W or r6, r3, r4 IF IF IF IF ID Ex M add r6, r3, r8 IF ID Ex

92 How to stall an instruction in ID stage prevent IF/ID pipeline register update stalls the ID stage instruction convert ID stage instr into nop for later stages innocuous bubble passes through pipeline prevent PC update stalls the next (IF stage) instruction

93 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. *Bubbles in pipeline significantly decrease performance.

94 1. Do Nothing Change the ISA to match implementation Compiler: don t create code with data hazards! (Nice try, we can do better than this) 2. Stall Pause current and subsequent instructions till safe 3. Forward/bypass Forward data value to where it is needed (Only works if value actually exists already) 100

95 Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Three types of forwarding/bypass Forwarding from Ex/Mem registers to Ex stage (M Ex) Forwarding from Mem/WB register to Ex stage (W Ex) RegisterFile Bypass

96 inst mem D A B A B imm D B data mem D M detect hazard Rb Ra forward unit Rd WE MC Rd WE MC IF/ID ID/Ex Ex/Mem Mem/WB 102

97 inst mem D A B A B imm D B data mem D M detect hazard Rb Ra forward unit Rd WE MC Rd WE MC IF/ID ID/Ex Ex/Mem Mem/WB Three types of forwarding/bypass Forwarding from Ex/Mem registers to Ex stage (M Ex) Forwarding from Mem/WB register to Ex stage (W Ex) RegisterFile Bypass 103

98 Ex/Mem inst mem D A B data mem sub r5, r3, r1 add r3, r1, r2 add r3, r1, r2 sub r5, r3, r1 IF ID Ex M W IF ID Ex M W Problem: EX needs ALU result that is in MEM stage Solution: add a bypass from EX/MEM.D to start of EX 104

99 Ex/Mem A inst mem D B data mem sub r5, r3, r1 add r3, r1, r2 Detection Logic in Ex Stage: forward = (Ex/M.WE && EX/M.Rd!= 0 && ID/Ex.Ra == Ex/M.Rd) (same for Rb) 105

100 Mem/WB A inst mem D B data mem add r3, r1, r2 sub r5, r3, r1 or r6, r3, r4 IF ID Ex M W IF or r6, r3, r4 ID IF Ex M W ID Ex M sub r5, r3, r1 W add r3, r1,r2 Problem: EX needs value being written by WB Solution: Add bypass from WB final value to start of EX 106

101 Mem/WB inst mem D A B data mem or r6, r3, r4 sub r5, r3, r1 Detection Logic: forward = (M/WB.WE && M/WB.Rd!= 0 && ID/Ex.Ra == M/WB.Rd && not (ID/Ex.WE && Ex/M.Rd!= 0 && ID/Ex.Ra == Ex/M.Rd) (same for Rb) add r3, r1,r2 107

102 A inst mem D B data mem add r6, r3, r8 or r6, r3, r4 sub r5, r3, r1 add r3, r1,r2 Problem: Reading a value that is currently being written Solution: just negate register file clock writes happen at end of first half of each clock cycle reads happen during second half of each clock cycle

103 inst mem D A B data mem add r6, r3, r8 or r6, r3, r4 sub r5, r3, r1 add r3, r1,r2 add r3, r1, r2 IF ID Ex M W sub r5, r3, r1 or r6, r3, r4 add r6, r3, r8 IF ID IF Ex M W ID Ex M W IF ID Ex M W

104 5-stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 110

105 time Clock cycle add r3, r1, r2 sub r5, r3, r4 lw r6, 4(r3) or r5, r3, r5 sw r6, 12(r3) 111

106 time add r3, r1, r2 Clock cycle IF ID Ex M W sub r5, r3, r4 IF ID Ex M W lw r6, 4(r3) IF ID Ex M W or r5, r3, r6 IF ID Ex M W sw r6, 12(r3) IF ID Ex M W

107 time add r3, r1, r2 Clock cycle IF ID Ex M W backwards arrows require time travel sub r5, r3, r4 IF ID Ex M W lw r6, 4(r3) IF ID Ex M W or r5, r3, r5 IF ID Ex M W sw r6, 12(r3) IF ID Ex M W

108 A inst mem D B data mem or r6, r3, r4 lw r4, 20(r8) Data dependency after a load instruction: Value not available until after the M stage Next instruction cannot proceed if dependent THE KILLER HAZARD 114

109 inst mem A D B or r6,r4,r1 lw r4, 20(r8) data mem lw r4, 20(r8) or r6, r3, r4 115

110 inst mem A D B or r6,r4,r1 lw r4, 20(r8) data mem lw r4, 20(r8) IF ID Ex or r6, r3, r4 IF ID 116

111 inst mem A D B or r6,r4,r1 NOP data mem lw r4, 20(r8) lw r4, 20(r8) or r6, r3, r4 IF ID Ex M W Stall IF ID* ID Ex M W 117

112 inst mem D A B data mem or r6,r4,r1 NOP lw r4, 20( lw r4, 20(r8) or r6, r3, r4 IF ID Ex M W Stall IF ID* ID Ex Ex M W 118

113 inst mem D A B A B imm D B data mem D M detect hazard Rd Rb Ra MC forward unit IF/ID ID/Ex Ex/Mem Mem/WB Rd WE MC Rd WE MC Stall = If(ID/Ex.MemRead && IF/ID.Ra == ID/Ex.Rd 119

114 inst mem D A B A B imm D B data mem D M detect hazard Rd Rb Ra MC forward unit IF/ID ID/Ex Ex/Mem Mem/WB Rd WE MC Rd WE MC Most frequent 3410 non-solution to load-use hazards Why is this solution so so so so so so awful? 120

115 Forwarding values directly from Memory to the Execute stage without storing them in a register first: A. Does not remove the need to stall. B. Adds one too many possible inputs to the ALU. C. Will cause the pipeline register to have the wrong value. D. Halves the frequency of the processor. E. Both A & D 121

116 Forwarding values directly from Memory to the Execute stage without storing them in a register first: A. Does not remove the need to stall. B. Adds one too many possible inputs to the ALU. C. Will cause the pipeline register to have the wrong value. D. Halves the frequency of the processor. E. Both A & D 122

117 Two MIPS Solutions: MIPS 2000/3000: delay slot ISA says results of loads are not available until one cycle later Assembler inserts nop, or reorders to fill delay slot MIPS 4000 onwards: stall But really, programmer/compiler reorders to avoid stalling in the load delay slot 123

118 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. Bubbles (nops) in pipeline significantly decrease performance. Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Better performance than stalling.

119 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2)

120 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2) 5 Hazards

121 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2) Forwarding from Ex/M ID/Ex (M Ex) Forwarding from M/W ID/Ex (W Ex) RegisterFile (RF) Bypass Forwarding from M/W ID/Ex (W Ex) Stall + Forwarding from M/W ID/Ex (W Ex) 5 Hazards

122 Find all hazards, and say how they are resolved: add r3, r1, r2 sub r3, r2, r1 nand r4, r3, r1 or r0, r3, r4 xor r1, r4, r3 sb r4, 1(r0)

123 Find all hazards, and say how they are resolved: add r3, r1, r2 sub r3, r2, r1 nand r4, r3, r1 or r0, r3, r4 xor r1, r4, r3 sb r4, 1(r0) Hours and hours of debugging!

124 Delay Slot(s) Modify ISA to match implementation Stall Pause current and all subsequent instructions Forward/Bypass Try to steal correct value from elsewhere in pipeline Otherwise, fall back to stalling or require a delay slot Tradeoffs?

125 5-stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 131

126 i = 0; do { n += 2; i++; } while(i < max) i = 7; n--; i r1 Assume: n r2 max r3 x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 addiu r1, r0, 7 # i = 7 x24 subi r2, r2, 1 # n-- 132

127 Control Hazards instructions are fetched in stage 1 (IF) branch and jump decisions occur in stage 3 (EX) next PC not known until 2 cycles after branch/jump x1c blt r1, r3, Loop x20 addiu r1, r0, 7 x24 subi r2, r2, 1 Branch not taken? No Problem! Branch taken? Just fetched 2 addi s Zap & Flush 133

128 prevent PC update clear IF/ID latch branch continues inst mem PC +4 D A B New PC = 14 1C blt r1,r3,l 20 addiu r1,r0,7 24 subi r2,r2,1 14 L:addi r2,r2,2 branch calc decide branch data mem If branch Taken Zap IF ID Ex M W IF ID NOP NOP NOP IF NOP NOP NOP NOP IF ID Ex M W 134

129 prevent PC update clear IF/ID latch branch continues inst mem PC +4 D A B New PC = 1C 1C blt r1,r3,l 20 addiu r1,r0,7 24 subi r2,r2,1 14 L:addi r2,r2,2 branch calc decide branch data mem If branch Taken Zap IF ID Ex M W IF ID NOP NOP NOP IF NOP NOP NOP NOP For every taken branch? OUCH!!! IF ID Ex M W 135

130 1. Delay Slot You MUST do this MIPS ISA: 1 insn after ctrl insn always executed Whether branch taken or not 2. Resolve Branch at Decode Some groups do this for Project 3, your choice Move branch calc from EX to ID Alternative: just zap 2 nd instruction when branch taken 3. Branch Prediction Not in 3410, but every processor worth anything does this (no offense!) 136

131 inst mem PC +4 D A B New PC = 1C branch calc decide branch 1C blt r1, r3, Loop F D X 20 addiu r1, r0, 7 F D 24 subi r2, r2, 1 F data mem If branch Taken Zap

132 i = 0; do { n += 2; i++; } while(i < max) i = 7; n--; x10 addiu r1, r0, 0 # i=0 i r1 Assume: n r2 max r3 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 nop x24 addiu r1, r0, 7 # i = 7 x28 subi r2, r2, 1 # n++ 138

133 inst mem PC +4 D A B New PC = 1C branch calc decide branch 1C blt r1, r3, Loop F D X 20 nop F D 24 addiu r1, r0, 7 F data mem If branch Taken Zap

134 inst mem PC +4 D A B branch calc decide data mem New PC = 1C branch 1C blt r1, r3, Loop F D X 20 nop F D 14 Loop:addiu r2,r2,2 F If branch Taken No Zapping 140

135 x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 nop Compiler transforms code x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r1, r1, 1 # i++ x18 blt r1, r3, Loop # i<max? x1c addiu r2, r2, 2 # n += 2 141

136 inst mem PC +4 D A B branch calc decide data mem New PC = 1C branch 1C blt r1, r3, Loop F D X 20 addi r2,r2,2 F D 14 Loop:addi r1,r1,1 F Note: Insn in delay slot will always be executed whether branch take or not 142

137 Most processor support Speculative Execution Guess direction of the branch Allow instructions to move through pipeline Zap them later if guess turns out to be wrong A must for long pipelines 143

138 Pipeline so far Guess (predict) that the branch will not be taken We can do better! Make prediction based on last branch Predict take branch if last branch taken Or Predict do not take branch if last branch not taken Need one bit to keep track of last branch

139 What is accuracy of branch predictor? Wrong twice per loop! Once on loop enter and exit We can do better with 2 bits While (r3 0) {. r3--;} Top: BEQZ r3, End J Top End: While (r3 0) {. r3--;} Top2: BEQZ r3, End2 End2: J Top

140 Branch Not Taken (NT) Predict Taken 2 (PT2) Predict Taken 1 (PT1) Branch Taken (T) Branch Taken (T) Branch Not Taken (NT) Branch Taken (T) Predict Not Taken 2 (PT2) Predict Not Taken 1 (PT1) Branch Not Taken (NT)

141 Control hazards Is branch taken or not? Performance penalty: stall and flush Reduce cost of control hazards Move branch decision from Ex to ID 2 nops to 1 nop Delay slot Compiler puts useful work in delay slot. ISA level. Branch prediction Correct. Great! Wrong. Flush pipeline. Performance penalty

142 Data hazards Control hazards Structural hazards resource contention so far: impossible because of ISA and pipeline design

143 Data hazards register file reads occur in stage 2 (IF) register file writes occur in stage 5 (WB) next instructions may read values soon to be written Control hazards branch instruction may change the PC in stage 3 (EX) next instructions have already started executing Structural hazards resource contention so far: impossible because of ISA and pipeline design

144 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. Pipelined processors need to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. Nops significantly decrease performance. Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Better performance than stalling. 150

145 Control hazards occur because the PC following a control instruction is not known until control instruction is executed. If branch is taken need to zap instructions. 1 cycle performance penalty. Delay Slots can potentially increase performance due to control hazards. The instruction in the delay slot will always be executed. Requires software (compiler) to make use of delay slot. Put nop in delay slot if not able to put useful instruction in delay slot. We can reduce cost of a control hazard by moving branch decision and calculation from Ex stage to ID stage. With a delay slot, this removes the need to flush instructions on taken branches. 151