On the Applicability of Convex Relaxations for Matching Symmetric Shapes

Transcription

1 On the Applicability of Convex Relaxations for Matching Symmetric Shapes Nadav Dym Weizmann Institute of Science In collaboration with Haggai Maron and Yaron Lipman

2 Shape matching Finding isometries

3 Shape matching-finding isometries aa ii aa jj AA iiii = dd aa ii, aa jj BB kkkk = dd(bb kk, bb ll ) bb kk bb ll Goal: find mapping (permutation) σσ: aa 1,, aa nn bb 1,, bb nn such that AA iiii = BB σσ ii,σσ(jj)

4 Graph isomorphism Input: AA = AA TT, BB = BB TT Goal: find permutation (if exists) σσ such that AA iiii = BB σσ ii,σσ(jj) Goal: find permutation matrix PP Π nn such that AA = PP TT BBBB

5 Graph matching/quadratic assignment Input: AA = AA TT, BB = BB TT Output: PP Π nn such that AA PP TT BBBB: PP = aaaaaaaaaann PP Πnn AA PP TT BBBB FF PP = aaaaaaaaaann PP Πnn PPPP BBBB FF Graph matching

6 DS relaxation Π nn PP = aaaaaaaaaann PP Πnn PPPP BBBB FF NP Hard! conv Π nn SS = aaaaaaaaaann SS cccccccc Πnn SSSS BBBB FF

7 DS relaxation conv Π nn is the Birkhorff polytope: conv Π nn DDDD = SS SS 0, SS1 = 1, SS TT 1 = 1} DDDD relaxation: SS = aaaaaaaaaann SS DDDD SSSS BBBB FF GGMM DDDD (AA, BB)

8 DS relaxation-does it work? PP = aaaaaaaaaann PP Πnn PPPP BBBB FF Part I: exactness? SS PP SS = aaaaaaaaaann SS DDDD SSSS BBBB FF Part II: projection SS

9 Exactness affects projection efficiency Exact case Exact+small noise Very noisy SS SS S SS = aaaaaaaaaann SS DDDD SSSS BBBB FF

10 Π nn Part I: Exactness [Aflalo et al 2015], [Fiori and Sapiro 2015] Assume: (i) AA BB (ii) Unique isomorphism PP min AAAA SSSS FF = 0 SS DDDD PP Usually Usually not SS SS = aaaaaaaaaann SS DDDD SSSS BBBB FF

11 Problem: Unique solution assumption ψψ φφ multiple solutions ψψ, ψψ φφ symmetric

12 Symmetries of natural shapes Bilateral symmetry: [SCAPE, FAUST, TOSCA] [SHREC]

13 Exactness vs. convex exactness Exactness (asymmetric) PP SS Convex exactness (symmetric) PP 1 PP 2

14 Convex exactness-definition IIIIII AA, BB = PP Π nn AAAA = PPPP}, IIIIoo cccccccc AA, BB = SS DDDD AAAA = SSSS} GGMM DDDD (AA, BB) is convex exact if IIIIoo cccccccc (AA, BB) = cccccccc(iiiiii(aa, BB)) PP 1 PP 2

15 A convenient reduction (B=A) (easy) Lemma: GGMM DDDD (A, A) is convex exact For any BB s.t. BB AA, GGMM DDDD (A, B) is convex exact

16 Goal Almost surely Usually Usually not PP SS Almost surely Usually? Usually not? PP 1 PP 2

17 Measure for the space of asymmetric graphs Asymmetric graphs: {AA = AA TT AA has no non-trivial automorphisms} SS nn = AA AA = AA TT } Asymmetric graphs VV GG 1 μμ SS nn = LLLLLLLLLLLLLLLL VV GG 2 GG ii > {II nn } VV GG 3 VV GG = AA SS nn PPPP = AAAA, PP GG}

18 Measure for Graphs with prescribed symmetry group GG 0 Graphs with sym group GG 0 : Graphs whose automorphism group is GG 0 VV GG 0 = AA SS nn PPPP = AAAA, PP GG 0 }

19 Measure for Graphs with prescribed symmetry group GG 0 VV(GG 0 ) Graphs with sym group GG 0 VV GG 2 VV GG 1 μμ GG0 = LLLLLLesgue GG ii > GG 0 VV GG 3 VV GG = AA SS nn PPPP = AAAA, PP GG}

20 Convex exactness for reflective groups Theorem 1: If GG ZZ 2, then for μμ GG almost every AA, GGMM DDDD (AA, AA) is convex exact. Also true for GG = {II nn } PP 1 PP 2

21 In general Theorem 1 holds if GG is reflective (PP 2 = II nn, PP GG ). GG s action on the vertices has a full orbit. PP 0 = II 4 PP 1 PP 2 PP 4

22 ?

23 General groups: 0-1 probability Theorem 2: For any GG Π nn, either (i) For μμ GG a.e. AA, GGMM DDDD (AA, AA) is convex exact. Or (ii) For all AA with sym group GG, GGMM DDDD (AA, AA) is not convex exact. Proof is constructive...

24 2 AA iiii = pp ii pp jj 2 AA iiii = pp ii pp jj 2 AA iiii = pp ii pp jj 1 GGMM DDDD (AA, AA) GGMM DDDD (AA, AA) GGMM DDDD (AA, AA)

25 General groups: 0-1 probability Theorem 2: For any GG Π nn, either (i) For μμ GG a.e. AA, GGMM DDDD (AA, AA) is convex exact. Or (ii) For all AA with sym group GG, GGMM DDDD (AA, AA) is not convex exact.

26 Summary- Part I convex exactness? almost everywhere almost everywhere

27 Part II: Where s my permutation? symmetric seeee oooo mmmmmmmmmmmmmmmmmmmm = SS DDDD AAAA = SSSS} Simplex algorithm

28 Part II: Where s my permutation? symmetric noise SS

29 DS relaxation- LL 2 projection S = aaaaaaaaaann SS DDDD SSSS BBBB FF SS PP LL2 = aaaaaaaaaann PP Π SS PP FF PP LL2

30 DS++: convex2concave projection SS PP LL2 PP ++

31 convex2concave projection [Zaslavskiy, Bach and Vert 2009] Observation: Convex energy EE 0 is equivalent over Π nn to concave energy EE TT. Concave energy: (i) Local/global minima are permutations! (ii) intractable Convex energy: (i) minima may not be permutations (ii) tractable! SS llllll SS SS gggggggggggg = PP

32 convex2concave projection convex concave EE tt0 EE tt1 EE tt2 EE TT SS kk = "aaaaaaaaaannn SS DDDD EE ttkk (SS) Warm start optimization from SS kk 1. SS 0 SS TT

33 DS++: convex2concave projection EE tt SS = SSSS BBBB FF 2 + tt(nn SS FF 2 ) convex concave EE tt0 EE tt1 EE tt2 EE ttff EE 0 = EE EE tt = EE over Π nn EE ttff strictly concave for t F 0 nn SS FF 2 = 0

34 Choosing [tt 0, tt FF ] EE tt SS = SSSS BBBB FF 2 + tt(nn SS FF 2 ) convex concave EE tt0 EE tt1 EE tt2 EE ttff Best choice of tt FF : tt FF = λλ mmmmmm tt FF = λλ mmmmmm over VV DDDD = SS SS1 = 0, SS TT 1 = 0} Best choice of tt 0 : (DS) tt 0 = 0 [Aflalo et al. 15] (DS+) tt 0 = λλ mmmmmm [Fogel et al. 13,15] (DS++) tt 0 = λλ mmmmmm over VV DDDD

35 Relaxation comparison

36 DS++ vs local minimization DS++ vs local minimization with 1000 different initializations:

37 Projection comparison Retrieval ratio LL 2 DS++ Noise (10 xx )

38 Symmetric, no noise EE tt SS = SSSS BBBB FF 2 + tt(nn SS FF 2 ) For tt > 0, PP ii are the only global minima! PP 1 PP 2

39 Symmetric, no noise Theorem 3: If DS(A,B) is convex exact, then (under some conditions) SS 1, SS 2,, SS TT = PP ii SS 0 SS TT PP 1 PP 2

40 Thank you! For more details see: Exact Recovery with Symmetries for the Doubly-Stochastic Relaxation. DS++: A Flexible, Scalable and Provably Tight Relaxation for Matching Problems. Acknowledgments: - European Research Council (ERC Starting Grant) - Israel Science Foundation