Techniques, October , Boston, USA. Personal use of this material is permitted. However, permission to

Transcription

1 Copyright 1996 IEEE. Published in the Proceedings of the 1996 Conference on Parallel Architectures and Compilation Techniques, October , Boston, USA. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O. Box 1331 / Piscataway, NJ , USA. Telephone: + Intl Improving Branch Prediction Accuracy by Reducing Pattern History Table Interference Po-Yung Chang Marius Evers Yale N. Patt Department of Electrical Engineering and Computer Science The University of Michigan Ann Arbor, Michigan Abstract Today's deeply pipelined, superscalar processors rely on accurate branch prediction in order to approach their performance potential. Branch mispredictions result in a ushing of the speculative information in the pipeline, thus limiting the amount of useful work that can be done. The 2-level branch predictors have been shown to achieve high prediction accuracy. However, it has also been shown that there isahighdegree ofpattern history table interference in 2-level branch predictors and that the interference generally has a negative eect on the prediction accuracy. This paper introduces a method for reducing the pattern history table interferencebydynamically identifying some easily predictable branches and inhibiting the pattern history table update for these branches. We show how this technique reduces pattern history table interference for two versions of the 2-level branch predictor and that this signicantly improves branch prediction accuracy for the SPEC 95 benchmarks. In particular, we eliminate up to 3% of the mispredictions for the gcc benchmark. 1 Introduction Branches can signicantly reduce the performance of high-performance processors. Speculative execution is a common solution to the branch problem. However, all speculative work beyond abranchmust be discarded if that branch is mispredicted. Therefore, a very accurate branch predictor is essential for achieving high performance on deeply pipelined, superscalar processors. Various prediction strategies, including 2-level branch predictors [5, 6, 7, 2], have been developed to improve branch prediction. Recent studies [4, 8] have shown that interference in the pattern history tables of these 2-level branch predictors can signicantly reduce their ability to make accurate predictions. Furthermore, it has been shown [1] that a signicant amount of branches in the dynamic execution stream tend to be mostly taken or mostly not-taken. The same study also showed the benets of separating these strongly biased branches from weakly biased branches based on proling information. Using static prediction on the strongly biased branches and dynamic prediction on the remaining branches resulted in lower contention in the pattern history table and thus more accurate predictions. Sechrest et al. [3] further classied the branches and showed that a signicant number of the biased branches are always taken and always nottaken. In this paper, we propose a new branch predictor that dynamically classies branches based on their history patterns. In order to reduce contention, this predictor does not update the pattern history table for easily predictable branches. A simple predictor is used to

2 handle these branches. To determine an eective classication of branches, we rst examine the frequency of regular recurring history patterns in branches because these branches are more likely to be easily predictable by simple predictors. Based on this information, we propose an implementation of the new predictor which dynamically detects easily predictable branches. It then predicts the easily predictable branches with a simple predictor while using a 2-level predictor for the other branches. This paper is organized into ve sections. Section 2 presents previous work. Section 3 determines a set of branches that can be easily identied and accurately predicted by a simple predictor. Section 4 introduces the new branch predictor and compares its performance to that of some previously proposed schemes. Section 5 provides some concluding remarks. 2 Previous Work To improve prediction accuracy, various branch prediction strategies have been studied. Yeh and Patt proposed the 2-level branch predictor which uses two levels of history to make branch predictions. The rst-level history records the outcomes of the most recently executed branches and the second-level history keeps track of the more likely direction of a branch when a particular pattern is encountered in the rst level history. The 2-level branch predictor uses one or more k-bit shift registers, called branch history registers, to record the branch outcomes of the most recent k branches. It uses one or more arrays of 2-bit saturating up-down counters, called Pattern History Tables, to keep track ofthe more-likely direction for branches. The lower bits of the branch address select the appropriate Pattern History Table(PHT) and the value in the branch history register (BHR) selects the appropriate 2-bit counter to use within that PHT. McFarling [2] introduced gshare, a variation of the global-history 2-level branch predictor whichxors the global branch history with the branch address to index into the PHT. Since the same global history patterns can occur for dierent branches during program execution, the global history pattern can be less ecient at identifying the current branch than the branch address itself. The gshare scheme tries to better identify the machine execution states by using both the branch address and the branch history. Recent studies [4, 8]have shown that PHT interference negatively aects the performance of 2-level branch predictors. Pattern history table interference occurs when a conditional branch references a PHT entry that was last referenced by another conditional branch. Talcott et al. [4] classied the interference as positive if the counter in the PHT entry correctly predicts the branch outcome. Otherwise, they classied the interference as negative. They showed that destructive interference causes branch prediction schemes to operate much below their potential performance level. Young et al. [8] classied interference as constructive if the counter correctly predicts the branch outcome and a predictor with an innite number of PHTs, thus having no PHT interference, mispredicts the outcome. They classied interference as destructive if the counter mispredicts while the predictor with an innite number of PHTs predicts correctly. Otherwise, they classied the interference as neutral. Young et al. also showed that although the branch interference can have a constructive, destructive, or neutral eect, destructive interference occurs much more frequently than constructive interference. Chang et al.[1] have proposed classifying branches to improve branch prediction. Branch classication partitions a program's branches into branch classes. Prediction accuracy is increased by associating each branch class with the most suitable predictor for that class. By classifying branches based on their dynamic taken rates collected by proling, a predictor which uses static prediction on the strongly biased branches and a dynamic prediction on weakly biased branches achieved improved performance. In this paper, we alsoimprove the performance of branch predictors by reducing the amount of destructive interference. The new classication method we propose dynamically separates the branches. 3 Characterization of Branch Behavior Dynamic predictors use past branch history to identify recurring execution patterns in order to make accurate prediction. In order to capture the behaviors of branches with complex recurring patterns, a lot of branch history information may have to be examined. For this reason, a sophisticated predictor, such as the 2-level branch predictor, which examines more branch history information, is able to detect more branch execution patterns and thus outperforms simpler predictors. On the other hand, even a less sophisticated predictor should be able to capture the dynamic behaviors of branches with simple recurring patterns. Thus, we propose to reduce the amount of PHT interference by using a simple predictor for predicting these easily predictable branches and inhibiting the pattern history table update for these branches. In this section, we will try to determine a set of

3 #ofcond Benchmark Input # of Instrs Branches compress test.in 1 13,15,25 1,661,855 gcc jump.i 154,45,36 25,93,86 go 2stone9.in 2 125,637,6 17,925,171 ijpeg specmun.ppm 3 26,82,135 2,441,37 perl scrabbl.pl 4 78,148,849 1,57,887 xlisp train.lsp 187,727,922 26,422,387 Table 1. Summary of six SPECint95 Benchmarks along with the input data sets. Frequency of Occurence (%) branches that can be easily identied and accurately predicted by simple predictors. 3.1 Experimental Model Pattern Length The experimental results presented in this paper were measured by trace-driven simulations using an instruction level simulator. These results are for six SPECint95 benchmarks. Each benchmark was simulated until completion. For each benchmark, Table 1 lists the input data set that was used and the exact number of instructions and conditional branches simulated. 3.2 Frequency of Recurring Patterns A branch may be easily predictable if it follows a regular repeating pattern of execution. In this section, we will measure how often these repeating execution patterns occur. A branch is said to have a repeating pattern of length k if the branch's outcome on the ith time, d i, is identical to the outcome on the (i + k)th time, d i+k. Furthermore, <d i d i+1 ::: d i+k > must be the shortest repeating pattern i.e. there can not be any repeating pattern within < d i :::d i+k >. For example, a branch that is always taken or always not-taken will have a repeating pattern of length 1. A branch that is taken every other time has a repeating pattern of length 2. To measure how often repeating execution patterns occur, we record the history of each branch in the 1 Abbreviated version of the SPECint reference input set bigtest.in. The initial list consists of 3 elements. 2 Abbreviated version of the SPECint training input set 2stone9.in using 19 levels instead of 5 levels. 3 Abbreviated version of the SPECint test input set specmun.ppm where the compression quality is 5 instead of 9. 4 Abbreviated version of the SPECint reference input set. Figure 1. Frequency of Pattern (1K entry BTB) branch target buer(btb). In this experiment, we use a 16-bit branch history register to holds the outcomes of the 16 most recent occurrences of the branch. A pattern is detected only if it repeats for the entire length of the history register. In addition, the pattern must repeat at least once in the history register i.e. a pattern in the 16-bit branch history register can be of at most length 8. Figure 1 and 2 show what fraction of the time patterns of dierent lengths were detected when using a 124 and 8192 entry BTB respectively. The 8192 entry BTB was used to approximate an innite size BTB 5. With the 8192 entry BTB, a repeating pattern of length one was detected for approximately 55 percent of the branches in the instruction stream, indicating that over half of the PHT accesses can be avoided if we inhibit updates to the PHT for these branches. With a smaller 124 entry BTB, fewer repeating patterns were detected because of more BTB misses. That is, when an entry is displaced from the BTB, the associated branch history is lost. This branch must occur 16 more times before its history register is lled with enough information for detecting repeating patterns. However, even with the smaller 1K entry BTB, which we will use for the remaining experiments in the paper, a repeating pattern length of one was still detected over 5 percent of the time. 5 For the SPEC95 integer benchmarks with the exception of gcc, only an insignicant amount of BTB misses occurred when the 8192 entry BTB was used.

4 Fraction of Occurences (%) Pattern Length Figure 2. Frequency of Pattern (8K entry BTB) Pattern Length 3.3 Accuracy of Static PHT Inhibiting PHT updates for certain branches can improve performance by reducing the negative eects of PHT interference. However, predicting these branches using simpler predictors may result in lower accuracy. Thus, to achieve higher performance, the disadvantage of having less accurate predictions for these branches must not outweigh the benets of reducing the PHT interference. For branches that have a recurring pattern, we expect the next outcome of the branch to be an extension of this pattern. Since the PSg(algo) [3] works on this premise, we expect it to perform well for branches with repeating patterns and apply it to these branches. Thus, we evaluate its performance. Figure 3 and 4 show the performance of PSg(algo) with a 124 and a 8192 entry BTB respectively. Branches with short repeating patterns were accurately predicted, indicating it could be advantageous to predict them with a simple predictor and inhibit the PHT update. 4 Predictor Model and Results Researchers have shown that interference in the pattern history tables can signicantly degrade the performance of 2-level branch predictors. In this section, we will rst present a ltering mechanism which will allow us to reduce the amount ofinterference in 2-level branch predictors. We will then present the perfor- Figure 3. PSg Performance with 1K entry BTB Pattern Length Figure 4. PSg Performance with 8K entry BTB

5 Global History (gshare) Pattern History Table Per-Address History Pattern History Tables Branch Target Buffer Branch History Register Count Direction Branch History Register Program Counter XOR Program Counter Prediction Prediction Figure 5. Structure of a Global History Branch Predictor. Although this gure only shows the gshare implementation, a similar structure can be implemented for all global history variations of the 2-Level Branch Predictor. mance benets of the ltering mechanism when applied to existing branch predictors. 4.1 Predictor Model As shown in section 3.2, branches with a repeating history pattern of length one occur frequently in the dynamic instruction stream. Section 3.3 also showed that these branches can be accurately predicted bya PSg predictor. In addition, a signicant amount of branch interference may be removed if the PHTs are not updated for these branches, improving the predictor's performance on the remaining branches. Therefore, to improve predictor performance, we propose the addition of a ltering mechanism which will dynamically separate always taken and always not-taken branches from the other branches the appropriate operations can then be applied to each branch based on its classication. The ltering mechanism is implemented in slightly dierent ways for per-address and global variations of the 2-level branch predictor. We needtobeableto dynamically identify branches with repeating patterns of length one. To do this for the global variations of the 2-level branch predictor, we propose the addition of a direction bit and a counter to each branch target buer entry. Figure 5 shows the structure of the gshare scheme with the ltering mechanism. The ltering mechanism works in the following way: If a BTB miss occurs, the default predictor is used Figure 6. Structure of a Per-Address History Branch Predictor for predicting this branch. For example, for the global history scheme, the PHT entry indexed by the global history register is used for predicting this branch. A new entry in the BTB is then allocated for this branch. The direction bit is initialized to taken and the counter is initialized to. If a BTB hit occurs, the value of the counter is examined. If the counter is not at its maximum value, the default predictor is used for predicting this branch and the PHT is updated as appropriate. If a BTB hit occurs and the value of the counter is at its maximum, the direction bit is then used for predicting this branch's direction and the PHT is not updated for this branch. When the branch is resolved, if the branch outcome matches the value in the direction bit, the counter is incremented. Otherwise, the direction bit is toggled to match the branch's new outcome and the counter is reset to. For the per-address variation of the 2-Level Branch Predictor, we already maintain the branch history of each branch for selection of the PHT entry, so the counters mentioned above are not needed. The branch history can be used to determine whether the outcomes of a branch are either always taken or always not-taken (as shown in Figure 6). 4.2 Experimental Results The performance of 2-level branch predictors is affected by the amount of pattern history table interfer-

6 ence. In this section, we will rst examine the PHT interference in two 2-level branch predictors. We will then study how dierent congurations of the ltering mechanism described in section 4.1 aect the amount of interference in 2-level branch predictors. Finally, we will present the performance impact of the ltering mechanism on 2-level branch predictors Pattern History Table Interference The amount of PHT interference in dierent 2-level branch predictors varies because of their dierent hashing schemes. Pattern history table interference occurs when a conditional branch references a PHT entry that was last referenced by another conditional branch. Tables 2 and 3 show the amount ofinterference for a per-address history variation (PAs) and a global history variation (gshare) of the 2-level branch predictor respectively. The tables also list the amount ofcon- structive and destructive interference [8] for each of the twoschemes. Interference is dened as constructive if the counter correctly predicts the branch outcome when a predictor with an innite number of PHTs, thus no PHT interference, mispredicts the outcome. If the counter mispredicts while the predictor with an innite number of PHTs predicts correctly, this interference is dened as destructive. Otherwise, the interference is neutral. Tables 2 and 3 show that the total amount of PHT interference for PAs is comparable to that for gshare. However, the amount of destructive interference for PAs is signicantly less than that for gshare. This is because the PAs scheme indexes into the PHT using the past branch history of the the same static branch. Thus, two static branches that hash into the same PHT have had the same past branch behavior and they are likely to have similar future behavior. Furthermore, most of the interference under PAs is due to strongly biased branches (e.g. always taken branches) because a large number of branches in the instruction stream are strongly biased. These branches tend to access the same PHT entries under the PAs scheme thus, most of the interference occurs at only a few PHT entries (as shown in Figure 7). Since the interference is due to strongly biased branches, most of the interference under PAs is neutral. On the other hand, there is a signicant amount of destructive interference for the gshare scheme. Branches whose past behavior are either all 's or all 1's may use various entries in the PHT because the gshare scheme selects the PHT entry by XORing the global branch history with the branch address. Due to this hashing scheme, branches that share the same PHT Figure 7. This gure shows the amount of interference ineach entry of a 8192 entry PHT for the PAs scheme when running gcc. The black squares represent counters with 66 cases of interference ormore (66 is the average amount of interference per entry). The white squares represent counters with 6 cases of interference or fewer. entries may not have similar past behavior, resulting in signicant amount of destructive interference. Unlike the PAs scheme, the interference for gshare is distributed across all PHT entries as shown in Figure 8. Since most of the interference due to strongly biased branches in a 2-level per-address predictor is neutral, removing these interference with the ltering mechanism has little eect on this predictor as shown in Figure 9. On the other hand, our experiments on the global history variations of the 2-level branch predictor showed that the ltering mechanism is eective for the GAs scheme, as shown in Figure 1. Futhermore, the performance benet of the ltering mechanism is greater for ghsare than GAs because the amount ofin- terference under the GAs scheme is less than that of the gshare scheme. For example, the ltering mechanism reduces the misprediction rate of a 2Kbyte GAs and a 2Kbyte gshare by 13% and 14.4% respectively. For the following studies, we will examine the performance impact of the ltering mechanism on the gshare scheme only Filtering Mechanism Congurations The performance of the ltering mechanism depends on the number of past history bits used for detecting regularly repeating patterns. Figure 11 shows the performance of the ltering mechanism using dierent number of branch history

7 predictor size 2KB 4KB 8KB Amount of PHT interference benchmark total constr. destr. total constr. destr. total constr. destr. compress gcc go ijpeg li perl Table 2. PHT interference for PAs predictor size 2KB 4KB 8KB Amount of PHT interference benchmark total const. destr. total const. destr. total const. destr. compress gcc go ijpeg li perl Table 3. PHT interference for gshare Figure 8. This gure shows the amount of interference ineach entry of a 8192 entry PHT for the gshare scheme when running gcc. The black squares represent counters with 325 cases of interference ormore (325 is the average amount of interference per entry). The white squares represent counters with 32 cases of interference or fewer..2 PAs PAs w/ filter. 2K 4K 8K 16K Branch Predictor Size (bytes) Figure 9. Performance of Per-address 2-Level Branch Predictors

8 bits 8 bits 16 bits com gcc go ijpeg li perl Figure 11. Performance vs Branch History Length GAs GAs w/ filter. 2K 4K 8K 16K Branch Predictor Size (bytes) Figure 1. Performance of GAs 2-Level Branch Predictors bits for detecting repeating patterns of 1's and 's. The predictor used in this experiment is a 2 KByte gshare with a 1K entry BTB. Figure 11 shows that for the gcc and go benchmarks, the performance of the ltering mechanism improves when using fewer branch history bits. Basing its decision on fewer history bits, the ltering mechanism will detect regularly repeating patterns for more branches. For example, with a loop that always executes for 1 iterations, the history pattern of the loop-ending branch would be a repeating pattern of Detecting patterns using 4 history bits, the ltering mechanism will use the BTB direction bit to predict this branch 6 out of 1 occurrences. Detecting patterns using 8 bits, the ltering mechanism will use the BTB direction bit only 2% of the time. Thus, basing its decision on fewer past history bits, the ltering mechanism will classify more branches as easily predictable and use the BTB direction bit for their predictions. Not updating the PHT for these occurrences reduces the amount of PHT interference in gshare. On the other hand, for the other benchmarks, the performance of the predictor improves as the number of history bits used by the ltering mechanism increases. Using more history bits will prevent the ltering mechanism from prematurely predicting a branch using the BTB direction bit instead of the more accurate dynamic predictor. Since there is only a small amount of PHT interference for these benchmarks, the benet of choosing the more accurate predictor outweighs the benet of reducing PHT interference Predictor Performance Branches with a repeating pattern length of one occur frequently as shown in Section 3.2, so ltering away these branches from the PHT can signicantly reduce

9 bC PAs Gshare GAs Gshare w/ filter. 2K 4K 8K 16K Branch Predictor Size (bytes) Figure 12. Performance of Branch Predictors the amount ofinterference. Table 4 shows the amount of PHT interference after ltering out these branches. The reduction in the amount ofphtinterference after ltering depends on the particular benchmark, from a maximum of 93.7% for the li benchmark to a minimum of 22.7% for the go benchmark (cf. Table 3). For this experiment, we use the outcomes of the last 8 occurrences of a branch for our ltering mechanism. The branch target buer has 1K entries and is 4-way set associative. Figure 12 shows the performance of gshare with and without the ltering mechanism. For illustration purposes, we also show the performance of three other dynamic branch predictors: GAs, PAs, and 2bC. Figure 13 shows that for the gcc benchmark, where we saw a large amount of destructive interference, the ltering mechanism is able to signicantly improve the performance of gshare. At lower implementation costs where a larger amount of PHT interference occurs, our scheme isable to remove more interference and thus achieve higher performance. For larger predictors where the eect of PHT interference is less signicant, the results of the two predictors are closer. Figure 14 shows the performance of the 2 KByte gshare schemes on the individual benchmarks. We present the results for the 2 KByte instead of the larger predictors because this predictor with its smaller implementation cost is more likely to be included in hybrid branch predictors. The ltering mechanism is able to improve the performance of these benchmarks, with the exception of compress. For the compress benchmark, since there is only a small amount of PHT in Gshare Gshare w/ filter. 2K 4K 8K 16K Branch Predictor Size (bytes) Figure 13. Performance of the gshare on gcc.3 gshare gshare w/ filter.2. com gcc go ijpeg li perl Figure 14. Performance of the 2 KByte gshare

10 predictor size 2KB 4KB 8KB Amount of PHT interference benchmarks total constr. destr. total constr. destr. total constr. destr. compress gcc go ijpeg li perl Table 4. PHT interference for gshare when PHT is not updated for branches whose per-address branch histories are either always taken or always not-taken terference, the benet of choosing the more accurate predictor outweighs the benet of reducing PHT interference. Thus, using the ltering mechanism which uses 8 history bits for detection slightly reduces prediction accuarcy. However, as shown in section 4.2.2, the performance of gshare can still be improved with a ltering mechanism by using more history bits, such as 16. For benchmarks, such as gcc, that have a large number of static branches in the dynamic instruction stream, a large amount of PHT interference occurs (see Table 3). The ltering mechanism was able to remove a signicant amount of the destructive interference (see Table 4), reducing the number of mispredictions by 3% for the gcc benchmark when using gshare. 5 Conclusions In this paper, we have introduced a method of reducing pattern history table interference by dynamically classifying branches as strongly biased or mixeddirectional. Our experiments showed that about half of the branches in the dynamic instruction stream are strongly biased, and can therefore be handled with a simple predictor. Inhibiting the update to the pattern history table for the strongly biased branches, we were able to get a considerable decrease in the amount of PHT interference with a resulting improvement in the accuracy of the gshare two-level branch predictor. For six of the SPECint95 benchmarks, we achieved an average 14% reduction of mispredictions for a 2 KByte gshare predictor. For the gcc benchmark which executes a large number of static branches, the ltering mechanism was able to remove a signicant amount of the destructive interference. We therefore reduce the number of mispredictions by 23% to 3% for the gshare predictor. 6 Acknowledgments We gratefully acknowledge the support of our industrial partners, without which it would not have been possible to undertake this work, in particular Intel Corporation and NCR Corporation. References [1] P.-Y. Chang, E. Hao, T.-Y. Yeh, and Y. N. Patt. Branch classication: A new mechanism for improving branch predictor performance. In Proceedings of the 27th Annual ACM/IEEE International Symposium on Microarchitecture, pages 22{31, a version of this paper has been accepted for publication in a special issue of International Journal on Parallel Programming. [2] S. McFarling. Combining branch predictors. Technical Report TN-36, Digital Western Research Laboratory, June [3] S. Sechrest, C.-C. Lee, and T. Mudge. The role of adaptivity intwo-level adaptive branch prediction. In Proceedings of the 28th Annual ACM/IEEE International Symposium on Microarchitecture, [4] A. R. Talcott, M. Nemirovsky, and R. C. Wood. The inuence of branch prediction table interference on branch prediction scheme performance. In International Conference onparallel Architectures and Compilation Techniques, [5] T.-Y. Yeh and Y. N. Patt. Two-level adaptive branch prediction. In Proceedings of the 24th Annual ACM/IEEE International Symposium on Microarchitecture, pages 51{61, [6] T.-Y. Yeh and Y. N. Patt. Alternative implementations of two-level adaptive branch prediction. In Proceedings of the 19th Annual International Symposium on Computer Architecture, pages 124{134, [7] T.-Y. Yeh and Y. N. Patt. A comparison of dynamic branch predictors that use twolevels of branch history. In Proceedings of the 2th Annual International Symposium on Computer Architecture, pages 257{266, 1993.

11 [8] C. Young, N. Gloy, and M. D. Smith. A comparative analysis of schemes for correlated branch prediction. In Proceedings of the 22st Annual International Symposium on Computer Architecture, pages 276{286, 1995.