Suffix arrays, BWT and FM-index. Alan Medlar Wednesday 16 th March 2016

Size: px

Start display at page:

Download "Suffix arrays, BWT and FM-index. Alan Medlar Wednesday 16 th March 2016"

Shanna Sparks
6 years ago
Views:

1 Suffix arrays, BWT and FM-index Alan Medlar Wednesday 16 th March 2016

2 Outline Lecture: Technical background for read mapping tools used in this course Suffix array Burrows-Wheeler transform (BWT) FM-index Lab session: Using BWA to map paired-end data against the human genome, SAM/BAM files, etc

3 Read mapping Sequencers can generate up to 100 million reads per sample Human genome is ~3 billion basepairs Need to map reads to the genome to discover variants (SNVs, indels), counts (gene expression)

4 Preliminaries String sequence of characters, e.g. "banana", "ATGC", "MDLISTFS" Alphabet { A, C, G, T, $ }, { A-Z, a-z, $ } Lexicographical order $ < A < C < G < T

5 Preliminaries Prefix non-empty substring that is the beginning of another string (left-to-right) e.g. "banana", "ATGC", "MDLISTFS" Suffix non-empty substring that is the ending of another string (right-to-left) e.g. "banana", "ATGC", "MDLISTFS"

6 Naïve exact search Text = "banana" Query = "nana" Linear search

7 Naïve exact search B A N A N A Text = "banana" N A N A Query = "nana" Linear search

8 Naïve exact search B A N A N A Text = "banana" N A N A Query = "nana" Linear search

9 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A Linear search

10 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A Linear search

11 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A N A N A Linear search

12 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A N A N A Linear search

13 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A N A N A Linear search

14 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A N A N A Linear search

15 Naïve exact search B A N A N A Text = "banana" Query = "nana" N A N A N A N A N A N A Linear search

16 Naïve search is too slow Human genome ~3 billion basepairs Read 100 basepairs Complexity of search scales linearly with the length of the text!

17 Suffix array Introduced by Manber and Myers (1990) as a space efficient alternative to suffix tree (independently by Gonnet (1987)) Sorted array of all suffixes of a given text Allows fast search of very large texts (e.g. genomes)

18 SA: building B A N A N A $ $ is lexicographically lower than all other characters in the alphabet and cannot appear in the text otherwise

19 SA: building B A N A N A $ A N A N A $

20 SA: building B A N A N A $ A N A N A $ N A N A $ A N A $ N A $ A $ $

21 SA: building B A N A N A $ 0 A N A N A $ 1 N A N A $ 2 A N A $ 3 N A $ 4 A $ 5 $ 6

22 SA: building B A N A N A $ 0 A N A N A $ 1 N A N A $ 2 A N A $ 3 N A $ 4 A $ 5 $ 6 $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

23 SA: building B A N A N A $ 0 A N A N A $ 1 N A N A $ 2 A N A $ 3 N A $ 4 A $ 5 $ 6 $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

24 SA: querying Search for prefixes in the suffix array that match our query string SA is sorted, so we can use binary search!

25 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

26 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

27 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

28 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

29 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

30 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

31 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

32 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

33 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

34 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

35 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

36 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

37 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

38 N A N A $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2

39 SA vs. naïve search Searching the human genome (~3 billion basepairs, n) for a single-end read (100 basepairs, m) Naïve search O(mn) Suffix array search O(m log(n))

40 SA vs. naïve search Searching the human genome (~3 billion basepairs, n) for a single-end read (100 basepairs, m) Naïve search O(mn) Suffix array search O(m log(n)) n O(n) O(log(n))

41 Good enough for read mapping? Human genome is ~3 billion basepairs Assume 5 bytes per basepair (1 byte characters, 4 byte integers) = ~14 GB NGS data really hit in 2009 (16 GB RAM at the time was a luxury!)

42 Burrows-Wheeler transform Invented by Burrows and Wheeler (1994) while working at DEC Used in compression (.bz2 files) Interested in three things: how to perform BWT why BWT is useful for compression how to reverse BWT

43 B A N A N A $ BWT

44 BWT B A N A N A $ A N A N A $

45 BWT B A N A N A $ A N A N A $ B

46 BWT B A N A N A $ A N A N A $ B N A N A $

47 BWT B A N A N A $ A N A N A $ B N A N A $ B A

48 BWT B A N A N A $ A N A N A $ B N A N A $ B A A N A $ B A N N A $ B A N A A $ B A N A N $ B A N A N A

49 BWT B A N A N A $ A N A N A $ B N A N A $ B A A N A $ B A N N A $ B A N A A $ B A N A N $ B A N A N A $ B A N A N A A $ B A N A N A N A $ B A N A N A N A $ B B A N A N A $ N A $ B A N A N A N A $ B A

50 BWT $ B A N A N A A $ B A N A N A N A $ B A N A N A N A $ B B A N A N A $ N A $ B A N A N A N A $ B A

51 BWT compression T = "banana$" BWT(T) = "annb$aa"

52 BWT compression T = "peter_piper_picked_a_peck_of_pickled_peppers_a_peck_of _pickled_peppers_peter_piper_picked_if_peter_piper_picked _a_peck_of_pickled_peppers_wheres_the_peck_of_pickled_ peppers_peter_piper_picked" BWT(T) = "ddsddkkkkaeaaddddsfsrrrrffffrrrrss eeeeiiiiiiiieeeeeeeehpp ppkkkkllllpppppppptttthpppprppppiooootwpppppppp_pppp cccccccccccckkkk iiiipppp eee eeeeeeeeeeeeeerrrereeee "

53 Relation to suffix array BWT matrix truncated at "$" in each row is the suffix array of the same text BWT can be computed directly from the suffix array $ B A N A N A A $ B A N A N A N A $ B A N A N A N A $ B B A N A N A $ N A $ B A N A N A N A $ B A

54 Reverse BWT It not very useful to compress something if we cannot get the original text back! BWT'(BWT(T)) = T

55 LF mapping (T-rank) B A N A N A $

56 LF mapping (T-rank) B A N A N A $ T-rank B0 A0 N0 A1 N1 A2 $

57 LF mapping (T-rank) B A N A N A $ F L T-rank B0 A0 N0 A1 N1 A2 $ $ B0 A0 N0 A1 N1 A2 A2 $ B0 A0 N0 A1 N1 A1 N1 A2 $ B0 A0 N0 A0 N0 A1 N1 A2 $ B0 B0 A0 N0 A1 N1 A2 $ N1 A2 $ B0 A0 N0 A1 N0 A1 N1 A2 $ B0 A0

58 LF mapping (T-rank) B A N A N A $ F L T-rank B0 A0 N0 A1 N1 A2 $ $ B0 A0 N0 A1 N1 A2 A2 $ B0 A0 N0 A1 N1 A1 N1 A2 $ B0 A0 N0 A0 N0 A1 N1 A2 $ B0 B0 A0 N0 A1 N1 A2 $ N1 A2 $ B0 A0 N0 A1 N0 A1 N1 A2 $ B0 A0

59 LF mapping (T-rank) B A N A N A $ F L T-rank B0 A0 N0 A1 N1 A2 $ $ B0 A0 N0 A1 N1 A2 A2 $ B0 A0 N0 A1 N1 A1 N1 A2 $ B0 A0 N0 A0 N0 A1 N1 A2 $ B0 B0 A0 N0 A1 N1 A2 $ N1 A2 $ B0 A0 N0 A1 N0 A1 N1 A2 $ B0 A0

60 LF mapping (T-rank) B A N A N A $ F L T-rank B0 A0 N0 A1 N1 A2 $ $ B0 A0 N0 A1 N1 A2 A2 $ B0 A0 N0 A1 N1 A1 N1 A2 $ B0 A0 N0 A0 N0 A1 N1 A2 $ B0 B0 A0 N0 A1 N1 A2 $ N1 A2 $ B0 A0 N0 A1 N0 A1 N1 A2 $ B0 A0

61 LF mapping (T-rank) B A N A N A $ F L T-rank B0 A0 N0 A1 N1 A2 $ $ B0 A0 N0 A1 N1 A2 A2 $ B0 A0 N0 A1 N1 A1 N1 A2 $ B0 A0 N0 Ns in the L column are sorted by their "right context", same as Ns in F column! A0 N0 A1 N1 A2 $ B0 B0 A0 N0 A1 N1 A2 $ N1 A2 $ B0 A0 N0 A1 N0 A1 N1 A2 $ B0 A0

62 LF mapping (B-rank) B A N A N A $ F L T-rank B-rank B0 A0 N0 A1 N1 A2 $ B0 A2 N1 A1 N0 A0 $ $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

63 LF mapping (B-rank) B A N A N A $ F L T-rank B-rank B0 A0 N0 A1 N1 A2 $ B0 A2 N1 A1 N0 A0 $ $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

64 LF mapping (B-rank) F L $ B0 A2 N1 A1 N0 A0 F column contains very little information, just counts of each character A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

65 LF mapping (B-rank) L A0 N0 N1 { $:1, A:3, B:1, N:2 } Which row contains N1 in the F column? B0 $ A1 A2

66 LF mapping (B-rank) L A0 N0 N1 B0 $ A1 A2 { $:1, A:3, B:1, N:2 } Which row contains N1 in the F column? Skip $ (+1) Skip As (+3) Skip Bs (+1) Skip first N (+1) = 6

67 LF mapping (B-rank) F 0 $ 1 A0 2 A1 3 A2 4 B0 5 N0 6 N1 L A0 N0 N1 B0 $ A1 A2 { $:1, A:3, B:1, N:2 } Which row contains N1 in the F column? Skip $ (+1) Skip As (+3) Skip Bs (+1) Skip first N (+1) = 6

68 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 $

69 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 A0 $

70 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 N0 A0 $

71 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 A1 N0 A0 $

72 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 N1 A1 N0 A0 $

73 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 A2 N1 A1 N0 A0 $

74 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 B0 A2 N1 A1 N0 A0 $

75 Reverse BWT Use B-ranking to reverse BWT, recreating the text T from right-to-left 0 $ 1 A0 F L A0 N0 { $:1, A:3, B:1, N:2 } B0 A2 N1 A1 N0 A0 $ 2 A1 3 A2 4 B0 5 N0 6 N1 N1 B0 $ A1 A2 B0 A2 N1 A1 N0 A0 $

76 FM-index All BWT allows us to do is compress text Ferragina and Manzini (2000) "Full-text index in Minute space" Combine BWT with other auxiliary data structures to get an index Space savings: e.g. Human genome (3 billion bp) SA = ~14 GB (5 bytes/bp) FM = ~1.5 GB (2 bits/bp)

77 Cannot search BWT like SA Rotation matrix contains the suffix array But we only store F and L columns, so binary search of prefixes not possible $ B A N A N A A $ B A N A N A N A $ B A N A N A N A $ B B A N A N A $ N A $ B A N A N A N A $ B A

78 Cannot search BWT like SA Rotation matrix contains the suffix array But we only store F and L columns, so binary search of prefixes not possible $ B A N A N A A $ B A N A N A N A $ B A N A N A N A $ B B A N A N A $ N A $ B A N A N A N A $ B A

79 BWT search In SA, we matched successively longer prefixes (left-to-right) of query string (binary search) In BWT, we will match successively longer suffixes (right-to-left) of query string (reverse BWT transform)

80 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

81 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

82 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

83 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

84 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

85 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

86 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

87 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 We know BWT contains the query, but unlike SA, we do not know the location of the match in T! A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2

88 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2 $ 6 A $ 5 A N A $ 3 A N A N A $ 1 B A N A N A $ 0 N A $ 4 N A N A $ 2 Idea: just store SA as well?

89 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2 $ 6 A $ A N A $ 3 A N A N A $ B A N A N A $ 0 N A $ N A N A $ Idea 2: store part of SA?

90 BWT search N A N A +1 F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2 $ 6 A $ A N A $ 3 A N A N A $ B A N A N A $ 0 N A $ N A N A $... and walk backwards through the BWT!

91 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2 $ 6 A $ A N A $ 3 A N A N A $ B A N A N A $ 0 N A $ N A N A $... and walk backwards through the BWT!

92 BWT search N A N A F L $ B0 A2 N1 A1 N0 A0 A0 $ B0 A2 N1 A1 N0 A1 N0 A0 $ B0 A2 N1 A2 N1 A1 N0 A0 $ B0 B0 A2 N1 A1 N0 A0 $ N0 A0 $ B0 A2 N1 A1 N1 A1 N0 A0 $ B0 A2 $ 6 A $ A N A $ 3 A N A N A $ B A N A N A $ 0 N A $ N A N A $... and walk backwards through the BWT!

93 BWT search F L $ B0 A2 N1 A1 N0 A0 $ 6 6 A0 $ B0 A2 N1 A1 N0 A $ 5 A1 N0 A0 $ B0 A2 N1 A N A $ A2 N1 A1 N0 A0 $ B0 A N A N A $ 1 +1 B0 A2 N1 A1 N0 A0 $ B A N A N A $ 0 0 N0 A0 $ B0 A2 N1 A1 N A $ 4 N A N A N1 A1 N0 A0 $ B0 A2 N A N A $ 2... and walk backwards through the BWT!

94 BWT search Finding location takes constant time if the offsets into T are evenly spaced in T, not in the SA! Make tradeoff between space (RAM) and time (how long lookups take)

95 Things we left out Rank calculations in the BWT need to be fast! Needs another auxiliary data structure Only covered exact matching, read alignment requires mismatches (e.g. SNP in read, not in genome) Other details: store forwards and backwards indices of genome due to sequencing error profile

96 Lab exercises BWA, SAM/BAM format, samtools

97 Reference Data Reference genome wget chromosome22.fa.gz md5sum chromosome22.fa.gz (168c78298e731128ee622cf422e70f1el) gunzip chromosome22.fa.gz du -h chromosome22.fa (49 MB, genome is 2.9 GB) less chromosome22.fa (where's the DNA?) grep -nv "^N" chromosome22.fa head (line !)

98 bwa index Indexing options: bwa index Index human chromosome 22 (~1.5 mins, genome takes ~1.5 hours): bwa index chromosome22.fa

99 bwa index output bash-3.2$ bwa index chromosome22.fa [bwa_index] Pack FASTA sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textlength= , availableword= [BWTIncConstructFromPacked] 10 iterations done characters processed. [BWTIncConstructFromPacked] 20 iterations done characters processed. [BWTIncConstructFromPacked] 30 iterations done characters processed. [BWTIncConstructFromPacked] 40 iterations done characters processed. [bwt_gen] Finished constructing BWT in 40 iterations. [bwa_index] seconds elapse. [bwa_index] Update BWT sec [bwa_index] Pack forward-only FASTA sec [bwa_index] Construct SA from BWT and Occ sec [main] Version: r1039 [main] CMD: bwa index chromosome22.fa [main] Real time: sec; CPU: sec

100 Read Data Paired-end reads wget chromosome22.reads_1.fastq.gz wget chromosome22.reads_2.fastq.gz md5sum chromosome22.reads_1.fastq.gz chromosome22.reads_2.fastq.gz de1cd26056c61571de5cdf246ede60d3 chromosome22.reads_1.fastq.gz 2be64fb5848c2997af0ab8fab416d539 chromsome22.reads_2.fastq.gz gunzip chromosome22.reads_1.fastq.gz (and the other file) less chromosome22.reads_1.fastq

101 bwa mapping options Several alignment options: bwa mem (70bp+ Illumina, 454, IonTorrent, Sanger) bwa bwasw (Smith-Waterman, frequent gaps) bwa aln/samse/sampe (short reads, original)

102 bwa mem Mapping paired-end data bwa mem [options] <idxbase> <in1.fq> <in2.fq> bwa mem -t 4 chromosome22.fa chromosome22.reads_1.fastq chromosome22.reads_2.fastq > chromosome22.sam -t specifies the number of CPUs to use

103 Sequence Alignment/Map format (SAM) SAM format is a TAB-delimited text file, we can inspect with a pager: less -S chromosome22.sam Each row represents an alignment, at least 11 fields Specification:

104 SAM fields Column 1: read name Column 3: reference sequence name (in our case "22") Column 4: reference sequence position (reads were extracted from 2Mbase region)

105 SAM flags SAM flags in column 2 describe mapping result

106 SAM post-processing Convert SAM file to BAM format: samtools view -Sb -o chromosome22.unsorted.bam chromosome22.sam Sort BAM file: samtools sort -o chromosome22.bam chromosome22.unsorted.bam Index BAM file: samtools index chromosome22.bam

107 samtools tview View alignment in console (in pileup format en.wikipedia.org/wiki/pileup_format ): samtools tview chromosome22.bam chromosome22.fa Scroll with arrow keys (but remember beginning of chr22 is all Ns) Type "g" (without quotes) and type "22: " to get to a region where reads are mapped Get to help screen by typing "?" Exit with "q"

Supplementary Figure 1 Examples of detection of MDA products based on molecular markers. To assess quality of whole-genome amplification by MDA, we

Supplementary Figure 1 Examples of detection of MDA products based on molecular markers. To assess quality of whole-genome amplification by MDA, we selected 10 markers (one per chromosome), segregating