Data Structures Week #9. Sorting

Transcription

1 Data Structures Week #9 Sortig

2 Outlie Motivatio Types of Sortig Elemetary O 2 )) Sortig Techiques Other O*log))) Sortig Techiques October 5, 2015 Boraha Tümer, Ph.D. 2

3 Sortig October 5, 2015 Boraha Tümer, Ph.D. 3

4 Motivatio Sortig is a fudametal task i may computer sciece problems. To sort a set of data is to put the data poits or records) i some order with regard to some feature of data. I a studet registratio system, a example to sortig would be to put the studet records i ascedig order with respect to studets IDs. October 5, 2015 Boraha Tümer, Ph.D. 4

5 Types of Sortig Sortig techiques may be classified based o whether the etirety of data fits i the mai memory. Sortig techiques for data that etirely fit i the mai memory are called the iteral sortig techiques. Those for data that do ot are called the exteral sortig techiques. We will discuss iteral sortig techiques. October 5, 2015 Boraha Tümer, Ph.D. 5

6 Elemetary Sortig O 2 ))Techiques We discuss three iteral) sortig techiques Selectio Sort Isertio Sort Bubble Sort Others: Comb sort improvemet o bubble), shell sort improvemet o bubble ad isertio), radix sort, coutig sort, bucket sort October 5, 2015 Boraha Tümer, Ph.D. 6

7 Selectio Sort void selectiosortusiged it *a, usiged it ) //sorts itegers i ascedig order i.e., a[]<a[+1]) { it i, j, mi; for i=1; i<; i++) { // i th smallest umber gets placed i its correct positio. mi=i; for j=i+1; j<=; j++) if a[j] < a[mi]) mi=j; swapa[mi],a[i]); } } October 5, 2015 Boraha Tümer, Ph.D. 7

8 Selectio Sort Example i mi j a[j] < a[mi] for i=1; i<; i++) { mi:=i; for j=i+1; j<=; j++) if a[j] < a[mi]) mi:=j; t:=a[mi]; a[mi]=a[i]; a[i]=t; } October 5, 2015 Boraha Tümer, Ph.D. 8

9 Selectio Sort Example i mi j a[j] < a[mi] for i=1; i<; i++) { mi:=i; for j=i+1; j<=; j++) if a[j] < a[mi]) mi:=j; t:=a[mi]; a[mi]=a[i]; a[i]=t; } etc. October 5, 2015 Boraha Tümer, Ph.D. 9

10 October 5, 2015 Boraha Tümer, Ph.D. 10 Algorithm Aalysis of Selectio Sort Barometer statemet or piece of code): the coditio of if a[j] < a[mi]) We fid how may times it is executed The outer loop turs -1 times. Ier loop turs -i times at the i th tur of the outer loop. ) ) ) O i i i i j i i i

11 Isertio Sort Isertio sort keeps the left portio of the array sorted. The isertio sort algorithm places the curret i.e., i th ) elemet at its correct positio at the i th tur of the outer loop. October 5, 2015 Boraha Tümer, Ph.D. 11

12 Isertio Sort Algorithm void isertiosort) { it i, j, v; it smallest; // boolea variable for i=2; i<=; i++) { // a sigle elemet array is sorted as is. v=a[i]; j=i; smallest=0; while a[j-1] > v &&!smallest) { a[j] =a[j-1]; j=j-1; if j <= 1) smallest=1; } a[j]=v; } } October 5, 2015 Boraha Tümer, Ph.D. 12

13 Isertio Sort Example Iitial sequece After fourth tur of for loop v=16<a[j-1]=48 After first tur of for loop v=24<a[j-1]=48 After secod tur of for loop v=20<a[j-1]={24,48} After third tur of for loop v=8<a[j-1]={16,...,48} v=12<a[j-1]={16,...,48} After fifth tur of for loop v=32<a[j-1]={16,...,48} After sixth tur of for loop v=54<a[j-1]=48 false! After seveth tur of for loop v=72<a[j-1]=54 false! After eighth tur of for loop October 5, 2015 Boraha Tümer, Ph.D. 13

14 Algorithm Aalysis of Isertio Sort Here, the ier loop has a stochastic coditio. Hece, we either have to work usig expected executio times or make a worst case aalysis. Barometer statemet or piece of code): j=j-1 We fid how may times it is executed The outer loop agai turs -1 times. I the worst case, ier loop turs i-1 times at the i th tur of the outer loop. October 5, 2015 Boraha Tümer, Ph.D. 14

15 October 5, 2015 Boraha Tümer, Ph.D. 15 Algorithm Aalysis of Isertio Sort cot d ) 2 2 1) 1 2 1) 1) O i i i j i

16 Bubble Sort Pass through the array of elemets Exchage adjacet elemets, if ecessary Whe o exchages are required, the array is sorted. Make as may passes as the umber of elemets of the array October 5, 2015 Boraha Tümer, Ph.D. 16

17 Algorithm of Bubble Sort void bubblesort) { it i, j; for i=; i>=1; i--) for j=2; j<=i; j++) if a[j-1] > a[j]) swapa[j-1],a[j]); } October 5, 2015 Boraha Tümer, Ph.D. 17

18 Bubble Sort Example Iitial sequece a[j-1]>a[j] swap After first tur of outer for loop a[j-1]>a[j] swap After secod tur of outer for loop ier loop swaps outer loop swaps After third tur of outer for loop a[j-1]>a[j] swap After fourth tur of outer for loop No more swaps ecessary! Fial sequece a[j-1]>a[j] swap Keys i purple spots are those that have bee swapped to move towards their correct positio October 5, 2015 Boraha Tümer, Ph.D. 18

19 Algorithm Aalysis of Bubble Sort Barometer statemet or piece of code): the coditio of if a[j-1] > a[j]) We fid how may times it is executed The outer loop turs times. Ier loop turs i-1 times at the i th tur of the outer loop. i 1 i 1) i 1 i1 j2 i1 i1 i1 1) O 2 2 ) October 5, 2015 Boraha Tümer, Ph.D. 19

20 Other O*lg))) Sortig Techiques We discuss three iteral) sortig techiques Heapsort Mergesort Quicksort October 5, 2015 Boraha Tümer, Ph.D. 20

21 Heapsort Idea: Delete_Mi operatio i a miimum heap removes the key i root. The hole at the root percolates dow where, i the array we keep the miimum heap, proper keys are moved left accordigly. The last cell i the array becomes available for storig aother key. If we perform the delete_mi operatio times i a -elemet mi heap ad place the i th key removed at the available cell A[-i], we will obtai a sorted sequece of the keys i the heap i descedig order. October 5, 2015 Boraha Tümer, Ph.D. 21

22 Algorithm of Heapsort it heapsortelmt_type *A) { // sorts keys i miheap A i descedig order... Elmt_Type x; for i=1;i<=;i++) { x=deletemia); // Olg) A[-i]=x; // O1) } } October 5, 2015 Boraha Tümer, Ph.D. 22

23 Heapsort Example Empty Empty Empty October 5, 2015 Boraha Tümer, Ph.D. 23

24 Heapsort Example Empty Empty Empty October 5, 2015 Boraha Tümer, Ph.D. 24

25 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 25

26 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 26

27 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 27

28 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 28

29 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 29

30 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. October 5, 2015 Boraha Tümer, Ph.D. 30

31 Heapsort Example Empty Empty Sequece i the array is sorted i descedig order! Empty October 5, 2015 Boraha Tümer, Ph.D. 31

32 Algorithm Aalysis of Heap Sort Every tur of the for loop: Oe delete_mi is performed, Olg ); Key i the root is placed i last elemet of the curret queue, O1). For loop is executed times for a heap with elemets Hece, ruig time of heap sort is O lg). October 5, 2015 Boraha Tümer, Ph.D. 32

33 Mergig two sorted lists Cosider two sorted lists L1, L2 ad a empty list R. The followig algorithm merges L1 ad L2 i R: a ad b poit to first elemet of L1 ad L2, respectively, c poits to the empty resultig list; while both lists have elemets left select as the ext elemet of resultig list mil1[a],l2[b]); advace the poiters or idices of the result list ad the iput list with the smaller elemet Apped rest of loger list amog L1 ad L2) to R October 5, 2015 Boraha Tümer, Ph.D. 33

34 Mergig two sorted lists: Algorithm struct odetype { it k; struct ode * ext; } typedef struct odetype odetype; typedef odetype * odeptrtype; it N,M; odeptrtype z;// z is a special header odeptrtype merge odeptrtype a, odeptrtype b) { odeptrtype c; c=z; do if a->k <=b->k) {c->ext=a; c=a; a=a->ext;} else {c->ext=b; c=b; b=b->ext;} while c->k < maxit); merge=z->ext; z->ext=z; } mai ); { scaf %d %d, &M, &N); z=mallocodetype); z->k=maxit; z->ext=z; /* Lists a ad b are costructed here */ merge a,b); } October 5, 2015 Boraha Tümer, Ph.D. 34

35 Mergig two sorted lists: Example L L c R z October 5, 2015 Boraha Tümer, Ph.D. 35

36 Mergig two sorted lists: Example L1 a b L a<b c R 14 z October 5, 2015 Boraha Tümer, Ph.D. 36

37 Mergig two sorted lists: Example L1 a b L a<b R c z October 5, 2015 Boraha Tümer, Ph.D. 37

38 Mergig two sorted lists: Example L1 a b L a<b R c z October 5, 2015 Boraha Tümer, Ph.D. 38

39 Mergig two sorted lists: Example L1 a b L a<b c R z October 5, 2015 Boraha Tümer, Ph.D. 39

40 Mergig two sorted lists: Example L1 a b L a<b c R 14 z October 5, 2015 Boraha Tümer, Ph.D. 40

41 Mergig two sorted lists: Example L1 a b L2 c a<b ull R z October 5, 2015 Boraha Tümer, Ph.D. 41

42 Mergig two sorted lists: Example L1 a b L a<b c R z October 5, 2015 Boraha Tümer, Ph.D. 42

43 Mergesort: Idea This is a divide & coquer algorithm. Give the merge algorithm, we may sort a sequece of keys by dividig the sequece ito divide sectio) 2 subsequeces each of /2 elemets, 4 subsequeces each of /4 elemets, 8 subsequeces each of /8 elemets,... subsequeces each of 1 elemet, recursively sort usig mergesort) each cosecutive pair of sequeces, coquer sectio) merge the two sorted subsequeces to obtai the sorted sequece combie sectio) October 5, 2015 Boraha Tümer, Ph.D. 43

44 Mergesort: Algorithm odeptrtype sort odeptrtype c; it N) { odeptrtype a,b; it i; if c->ext == z) retur c; else { a=c; for i=1;i<n/2;i++;c=c->ext); b=c->ext; c->ext=z; retur mergesorta,n/2), sortb,n-n/2)) } } October 5, 2015 Boraha Tümer, Ph.D. 44

45 Mergesort: Example z a sort z z b October 5, 2015 Boraha Tümer, Ph.D. 45

46 Mergesort: Example a z a sort b z 37 z October 5, 2015 Boraha Tümer, Ph.D. 46

47 Mergesort: Example b z a sort b z 40 merge z 43 a z October 5, 2015 Boraha Tümer, Ph.D. 47

48 Mergesort: Example a b z 37 z merge a z October 5, 2015 Boraha Tümer, Ph.D. 48

49 Mergesort: Example z z b sort z a b October 5, 2015 Boraha Tümer, Ph.D. 49

50 Mergesort: Example z z a b sort sort z z z z a b a b October 5, 2015 Boraha Tümer, Ph.D. 50

51 Mergesort: Example z z z z a merge b a merge b z z a b October 5, 2015 Boraha Tümer, Ph.D. 51

52 Mergesort: Example z z a merge b z a October 5, 2015 Boraha Tümer, Ph.D. 52

53 Mergesort: Example a z z a z b merge October 5, 2015 Boraha Tümer, Ph.D. 53

54 Algorithm Aalysis of Mergesort The maximum umber of comparisos per mergig of two sequeces for a total of elemets is -1= Θ). How may partitios of how may elemets are merged? Mergig oce /2 keys with /2 keys = /2+/2-1 Mergig twice /4 keys with /4 keys = 2/4+/4-1) Mergig 4 times /8 keys with /8 keys = 4/8+/8-1)... Mergig /2 times oe key with aother key = /2. October 5, 2015 Boraha Tümer, Ph.D. 54

55 October 5, 2015 Boraha Tümer, Ph.D. 55 Algorithm Aalysis of Mergesort t t t O O i i i i i lg ) 1) 2 lg ) 2 2 ) ) ) lg 1 lg lg 0 1 lg 0 lg lg lg Hece, ruig time of mergesort is Θ lg).

56 Quicksort: Idea Aother popular divide & coquer sort algorithm. The idea here is that, at every call of a specific partitioig algorithm, a selected key pivot) is placed at its correct positio, ad all keys less are moved to left while those greater tha that key are carried over to the right of the key. The origial partitioig algorithm used i quicksort is devised by C.A.R. Hoare. October 5, 2015 Boraha Tümer, Ph.D. 56

57 Quicksort: Idea After partitioig, the origial sequece is divided ito three subsequeces: 1. Numbers to the left of pivot still eeds sortig) 2. Pivot correctly placed) 3. Numbers to the right of pivot still eeds sortig) Recursively callig quicksort fuctio for the left ad right subsequeces, we sort the etire array. October 5, 2015 Boraha Tümer, Ph.D. 57

58 Quicksort: Algorithm QuicksortA,l,r) { //A is the array holdig the key sequece // l,r are the lowest ad highest idex values of // the key sequece i respective order. // m is the idex value of the pivot. if l<r) m=partitioa,l,r); QuicksortA,l,m-1); QuicksortA,m+1,r); } October 5, 2015 Boraha Tümer, Ph.D. 58

59 Quicksort: Coceptual Example pivot October 5, 2015 Boraha Tümer, Ph.D. 59

60 pivot Quicksort: Coceptual Example October 5, 2015 Boraha Tümer, Ph.D. 60

61 Quicksort: Coceptual Example pivot 11 6 Empty! pivot October 5, 2015 Boraha Tümer, Ph.D. 61

62 pivot Quicksort: Coceptual Example October 5, 2015 Boraha Tümer, Ph.D. 62

63 Quicksort: Coceptual Example pivot pivot October 5, 2015 Boraha Tümer, Ph.D. 63

64 Quicksort: Coceptual Example pivot The right subsequece of the origial sequece is sorted similarly. October 5, 2015 Boraha Tümer, Ph.D. 64

65 Partitioig*: Idea The idea here is to group keys so that all keys less tha pivot are at left of pivot, ad all greater keys are at the right of pivot, The pivot is the last key of the sequece. At ay time, the rest of the sequece cosists of three subsequeces: 1. the left subsequece holds keys less tha the pivot; 2. the middle subsequece holds keys greater tha the pivot; 3. the right subsequece holds keys ot yet sorted; Iitially, the left ad middle subsequeces are empty. Ay time a key i the right subsequece is processed i.e., compared with pivot), it is purged from the right ad placed to the left or middle sequece. *mai referece for this algorithm is [1] October 5, 2015 Boraha Tümer, Ph.D. 65

66 Partitioig: Algorithm PartitioA,l,r) { x=a[r]; i=l-1; for j=l;j<r; j++) { if A[j]<=x) {i++; swapa[i],a[j])} } swapa[i+1],a[r]); retur i+1; } *mai referece for this algorithm is [1] October 5, 2015 Boraha Tümer, Ph.D. 66

67 Partitioig: Example i l j Iitial Sequece r l i j After 1st tur of for loop r l i j After 2d tur of for loop r October 5, 2015 Boraha Tümer, Ph.D. 67

68 Partitioig: Example l i j After 3rd tur of for loop r l i j After 4th tur of for loop r l i After 5th tur of for loop j r October 5, 2015 Boraha Tümer, Ph.D. 68

69 l Partitioig: Example i After 6th tur of for loop j r l i After 7th tur of for loop j r l After 8th tur of for loop i j r October 5, 2015 Boraha Tümer, Ph.D. 69

70 l Partitioig: Example After 9th tur of for loop i j r l After 10th tur of for loop i j r l After 11th tur of for loop i j r October 5, 2015 Boraha Tümer, Ph.D. 70

71 l Partitioig: Example After 12th tur of for loop i j r l After 13th tur of for loop i j r l After 14th tur of for loop i j r October 5, 2015 Boraha Tümer, Ph.D. 71

72 Partitioig: Example l After 15th tur of for loop i j r l After 16th tur of for loop i j r l After 17th ad 18th tur of for loop i r j October 5, 2015 Boraha Tümer, Ph.D. 72

73 l Partitioig: Example After executio of retur statemet i r j October 5, 2015 Boraha Tümer, Ph.D. 73

74 Hoare s Partitioig: Idea The idea is to simply check the sequece startig at both eds to see that o keys less tha pivot should remai to right of pivot, ad o greater keys to left, If ay two such keys exist, they are swapped to have them at their correct sides. October 5, 2015 Boraha Tümer, Ph.D. 74

75 Hoare s Partitioig: Idea To accomplish this, two poiters e.g., i ad j) start at both eds of the sequece to sca through movig towards each other. Wheever a poiter locates a key i a icorrect positio i.e., a greater key to left of pivot or a key less tha the pivot at its right), it stops movig. Whe both poiters stop, the keys they poit to are swapped ad they restart movig. Scaig termiates whe poiters pass crossig each other, ad the key is placed at the positio poited to by j. October 5, 2015 Boraha Tümer, Ph.D. 75

76 Hoare s Partitioig: Algorithm Partitio_HoareA,l,r) { x=a[l]; i=l-1; j=r+1; while true do j-=1 while A[j]>x; do i+=1 while A[i]<x; if i<j) swap A[i] A[j]; else { swapa[j],a[l]) retur j; } } October 5, 2015 Boraha Tümer, Ph.D. 76

77 Hoare s Partitioig: Example i l r j Iitial Sequece l i After 1st tur of while loop j r l After 2d tur of while loop j i r i>j, ot exchaged October 5, 2015 Boraha Tümer, Ph.D. 77

78 Hoare s Partitioig: Example l At the ed of partitioig j i r keys < pivot Pivot correctly placed keys > pivot October 5, 2015 Boraha Tümer, Ph.D. 78

79 Algorithm Aalysis for Quicksort Average case: Each pivot at positio m has a left ad a right subsequece with m-1 ad -m keys, respectively, to sort. Hece, the ruig time f) of quicksort for elemets ca be expressed as follows f)=fm-1)+f-m)+θ). The term Θ) is the time that partitioig takes to place the key selected as pivot at its correct positio m. October 5, 2015 Boraha Tümer, Ph.D. 79

80 October 5, 2015 Boraha Tümer, Ph.D. 80 Algorithm Aalysis for Quicksort m ad -m ca be ay umber betwee betwee 1 ad. To come up with a geeral solutio, we ca average fm-1) ad f-m): )) lg )) lg ) ) ) 2 ) ) ) 2 ) ) ) 1) 1 ) O O f m f E f E m f E f E m f m f E f E m m m

81 October 5, 2015 Boraha Tümer, Ph.D. 81 Algorithm Aalysis for Quicksort Best Case: The pivot is placed always i the middle. f)=2f/2)+θ); )) lg )) lg ) lg ) lg 2 ; 2 2 ) 0 2) : ; 2 1) 2 ) 2 ); 2) / 2 ) O O f c c f k k c c k f x CE c k f k f f f k k k k k

82 October 5, 2015 Boraha Tümer, Ph.D. 82 Algorithm Aalysis for Quicksort Worst Case: The sequece is sorted i the opposite order.. f)=f-1)+θ); ) ) 1 ) ; ) 0 1) : ; 1) ) ); 1) ) O O f c c c f x CE c f f f f

83 Selectig the pivot... Pivot selectio method may essetially affect quicksort performace. Selectig the first or the last) would work alright if the sequece is purely radomly built. If ot, the worst case is ot totally ulikely to occur. Selectig the pivot radomly works well. However, the radom umber geerator should geerate umbers sufficietly radomly. Furthermore, radom umber geeratio is a expesive process. Media-of-3 partitioig media of a sequece is the /2 highest amog keys i a sequece) is to select as the pivot the secod largest amog the leftmost, rightmost ad middle keys i the sequece. October 5, 2015 Boraha Tümer, Ph.D. 83

84 A O)-Expected-Time Selectio Algorithm Quicksort ca be modified to select the k th largest key i the sequece. 1. Select a pivot 2. Have it placed at its correct positio m 3. If k<m) recursively call quicksort for the left subsequece oly 4. Else recursively call quicksort for the right subsequece oly. October 5, 2015 Boraha Tümer, Ph.D. 84

85 Pivot Coceptual Example: O)-Expected- Time Selectio Problem: Fid third smallest key i.e., key at cell 2)! Iitial Sequece placed at array cell 10 Sice 2 < 10, we cosider oly umbers less tha placed at array cell 8 Sice 2 < 8, we cosider oly umbers less tha placed at array cell 2 Sice 2=2, we stop. Third smallest key is 21. October 5, 2015 Boraha Tümer, Ph.D. 85

86 Referece... [1] T.H. Corme, C.E. Leiserso, R.L. Rivest, C. Stei, Itroductio to Algorithms, 2d editio, MIT Press, 2003 [2] M.A. Weiss, Data Structures ad Algorithm Aalysis i C, 2d editio, Addiso-Wesley, 1997 October 5, 2015 Boraha Tümer, Ph.D. 86