ESE 415: Optimization Making best decisions in a high-dimensional universe

Transcription

1 ESE 415: Optimization Making best decisions in a high-dimensional universe Prof. Ulugbek Kamilov Computational Imaging Group (CIG) WashU (ESE 415) Spring cigroup.wustl.edu kamilov@wustl.edu

2 Today we will talk about Motivating example from CIG Optimization as a pillar of machine learning Brief overview of optimization What will we learn during this semester? Class structure Assignments, grades, assistant instructors, etc.

3 Today we will talk about Motivating example from CIG Optimization as a pillar of machine learning Brief overview of optimization What will we learn during this semester? Class structure Assignments, grades, assistant instructors, etc.

4 What are some examples of imaging systems? MRI Phone Optical microscope Depth sensor Electron microscope CT

5 Imaging systems are going through a paradigm shift with computation at its core Past: Solely rely on hardware for image formation input imaging system instrument output Present: Use digital processing for improved performance input instrument computation output Future: Advanced inference for retrieving hidden information input instrument cloud computation output

6 b<latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> A<latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> x<latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> Medical image quality directly depends on the amount of collected data MRI in an ideal world where patients do not move =<latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> full sampling = as many measurements as unknown image pixels

7 Medical image quality directly depends on the amount of collected data MRI in an ideal world where patients do not move high-quality image

8 b<latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="raey/bt4xedenab8y9ragc0ksl0=">aaab/hicbzdlsgmxfibp1futt6pln8eiucoziujcrcgnywr2au1qkjrty5pmkgsemtrncktrd+lwd3hpm5hpz2fbfwh8/occzslpysgn9f1vr7c2vrg5vdwu7ezu7r+ud4+ajko0zq0aiui3ctzmcmuallvb2rfmwblbwmr8m9vbt0wbhqkho4lzkpfq8qgn2dqr2suyjdneuejx/znqkgq5vcbxvvf+6fyjmkimlbxyme7gxzzmsbaccjytdrpdykzhemg6dhwwzitp7noponnohw0i7z6yaob+nuixngyiieuu2i7mci0z/6t1eju4dlou4sqyreelbolanklz11gfa0atmdjavhn3k6ijrdg1lqcflurmmqtlcaxc86iaol6/rnru8nskcaknca4bxeen7qaodadwcc/wcm/es/fufxif89acl88cw4k8r1+c9pxl</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> <latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> A<latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> <latexit sha1_base64="gdiufbk56qej3ezv6hxrar2bywo=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4oblxxsbdqhznjmg5tkhiqjlke+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvs3q7seqnivkg5ne1bd4kfnicdbwavuckd5m++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8f1mpx+fpfoeetuecpkhbhe6gau0g8agv8apvzrpz7nw4n/pwgppphmocnk9ftukvqg==</latexit> x<latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> <latexit sha1_base64="o/fgcfx5ut2bqymljp4egcdol+w=">aaab/hicbzdlsgmxfibp1futt6pln8eiucozitsfi4iblxxsbdqhznjmg5tkhiqjlqe+g1tduxo3votl38rmowvb+kpg4z/nce7+iozmg9f9dgpr6xubw8xt0s7u3v5b+fcopaneedokey9uj8cacizp0zddasdwfiua03ywvsnq7ueqnivkvzne1bd4kfnicdbwavuckt5n++wkw3vnqqvg5vcbxi1++ac3iegiqdsey627nhsbp8xkmmlptnrlni0xgemh7vquwfdtp7nrp+jmogmurso+addm/turyqh1ras2u2az0su1zpyv1k1meowntmajozlmf4ujryzc2dfrgclkdj9yweqxeysii6wwmtaghs2bydlxlhnyhdzf1bn8d1mpx+fpfoeetuecpkhbhw6hau0g8aav8apvzrpz7nw4n/pwgppphmocnk9fpa6v4q==</latexit> Medical image quality directly depends on the amount of collected data =<latexit sha1_base64="hn4q3tknhqwxwo7zopyyftde/9s=">aaab93icbza9swnbeibn4lemx1flm8ugwiu7ebrqcnhyjma+idnc3myuwbk3d+zucufil7dv2k5s/tmw/hm3yrum8ywfh3dmmnk3satxxnw/nclg5tb2tng3tld/chhupj5p6thvdjssfrhqbfsj4bkbhhubnuqhjqkb7wd8mku3n1fphssnkyxor3qoecgzndzq3pflfbfqzkxwwcuharnq/fjpbxcznejpmkbadz03mf6eksozwgmpl2pmkbvtixytshqh9ifzq6fkwjodesbkpmni3p07magr1lku2m6impferc3m/2rd1is3/otljduo2wjrmapiyjl7nrlwhcyizajlittbcrtrrzmx2sxtcakpzcrbtwadwldvz3ljulk7y9mpwhmcwyv4cam1eiq6niebwgu8wputoe/oh/o5ac04+cwplmn5+gusnjnr</latexit> undersampling = less measurements than unknown image pixels MRI in a real world where patients breath and move

9 Medical image quality directly depends on the amount of collected data MRI in an ideal world where patients do not move high-quality image MRI in a real world where patients breath and move noisy image

10 Medical image quality directly depends on the amount of collected data MRI in an ideal world where patients do not move MRI in a real world where patients breath and move Question: How to compensate for the missing data?

11 sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit> sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit Assume that we have a dataset consisting of noisy images with their clean counterparts <latexit Multiple Scattering Truth noisy x 64 2 x 2 max pooling {yi }i=1,...,n <latexit sha1_base64="vmjngclxwcrmghlngkjfmao1zcu=">aaacfnicbvdlsgmxfm3uv62vuzecbivgopqzexshuhdjsoj9qgcymmmmdu0yq5irhmf2fotf4fbx7sstw5f+ieljyvspba7n3mo9owhcqnko822vvlbx1jfkm5wt7z3dpxv/ok3ivglswjglztdeijaqsettzug3kqtxkjfoolod+51hihwnxypoeujznba0ohhpiwx2szd7ic+ziqbeeequ3kc3br1+rfuniikwq07dmqaue3dgqmcgzmd/mcxooream6ruz3us7ediaoozkspeqkic8agnsm9qgthrfj75rwfpjdkhusznexpo1l+jhhglmh6asy70uc16y/e/r5fq6mrpquhstqselopsbnumx6xappuea5yzgrck5laih0gire11c1tcpu7exwxgmbtp667h9xfvxvwsnti4aifgdljgejtahwicfsdgcbyav/bmpvvv1of1or0twbpmizid9ful1m2fbq==</latexit> x 2 up-conv. + BN + ReLU domain to the image domain. We define the backprojection of the measurements generated by Question: How to train the deep neural net? the kth transmitter as zk = Pk yk k=1 sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit> sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit Pk <latexit A modern machine learning solution: Train a deep neural net on a dataset Vol. 26, No May 2018 OPTICS EXPRESS M s Backpropagation Re( ) Im( ) U-Net Decoder yk ScaDec Recon Fig. 2. The overview of the proposed approach that first backpropagates the data into a Im( ) Re( ) complex valued image and then maps this image into the final image with a ConvNet. deep neural net x 3 conv. + BN + ReLU where vector yk 2 C M are the measurements consistent with the kth transmitter and collected by M receivers, and matrix Pk 2 C N M is the backprojection operator. Inside the operator Pk, matrix SH 2 C N M is the Hermitian transpose of the discretized Green s function S, and u in,k is the element-wise conjugate of the incident light field emitted by the kth transmitter. The output zk 2 C N is a complex vector with N elements, which matches the number of pixels in the original image. When the data is collected with multiple transmissions, we define the backprojection of K transmitters as K K w= zk = Pk yk, (9) with Pk, diag(u in,k )SH, skip connection + concatenation clean x 1 conv. Fig. 3. Visual illustration of the proposed learning architecture based on U-Net [45]. The input consists of two channels for the real and imaginary parts of the backpropagated vector w 2 C N. The output is a single image of the scattering potential x 2 R N. xi = f (yi ) <latexit sha1_base64="k2ptv2veyd5j9b4k+etx5wksdje=">aaachhicbvdlssnafj3uv62vqks3g1wom5kioaufghuxfwwttcvmppn26ewszm7eevidfotf4fbx7sst4ni/cdjmyvspdjx7zr3co8elbndg299wywl5zxwtuf7a2nza3inv7rv0gcvkmjquowp7rdpba9yedok1i8wi9as790bxmx//wjtmyxah44j1jbke3oeugjhc8lhxk8lj6nj8hx03yaoudbmqnk1mxdhyj265ytfscfaicxjsqtkabvmn2w9plfkavbcto44dqs8hcjgvlc11y80iqkdkwdqgbkqy3usmv0nxsvh62a+veqhgifp3iifs67h0tkckmntzxib+53vi8c96cq+igfhap4v8wgaicryn7npfkiixiyqqbm7fdeguowacnnniydrk4swnsehapzxh8nuzsv0yt6eidtahqiihnam6uken1equpaex9irergfr3fqwpqetbsuf2uczsl5+atcmooo=</latexit> {xi }i=1,...,n <latexit sha1_base64="2m1umcgrgepirxzstsum17tcmok=">aaacfnicbvdlsgmxfm34rpu16lkqybfcldijgi4ucm5cvrap6axdjk3b0cqzjbmxdlpzi/wgt7p2j27duvrpzlszsk0haodz7uhendbmvgnh+bawlldw19zlg+xnre2dxxtvv6wirglsxbglzcdeijaqsfntzugnlgtxkjf2olrj/fydkypg4l6py+jznbc0tzhsrgrsiy/1qp4+zgh1sicl8bq6vej1iq2qugsbxxfqzgrwkbgfqyacjcd+mvmccci0zkipruve2k+r1bqzkpw9rjey4reakk6hangi/htyjwyegkuh+5e0t2g4uf8mussvgvpqthkkh2rey8x/vg6i+5d+skwcaclwdfe/yvbhmc8f9qgkwloxiqhlam6feigkwtpun7ml5hkn7nwdi6r1vnmnvzuv1k+kdkrgebydu+ccc1aht6abmgcdj/acxsgb9wy9wx/w53r0ysoyb2ag1tcv1mofba==</latexit> (8) k=1 where vector w 2 C N is the linear combination of zk and K denotes the number of transmitters. Note that the backprojection (9) does not rely on the actual forward model H( ) in (5) which is both nonlinear and object dependent. Remarkably, as we shall see, our simple backprojection followed by a specific ConvNet architecture will be sufficient to recover a high-quality image

12 A modern machine learning solution: Train a deep neural net on a dataset Vol. 26, No May 2018 OPTICS EXPRESS Backpropagation M s Multiple Scattering sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit> sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit sha1_base64="wrvmvahdiv+rxnkqf2utmh6lm8k=">aaab/xicbvdjtgjbeo3bdxebl5uxiwccfzldry9elx7ryjiwhpq0ndchz0l3jrenxf/x4kfjvpof3vwbg5idgi+p5ow9qltv82lbfdr2t5fbwv1b38hvfra2d3b3zp2dpoosyadbihhjtkcvcb5cazkkamcsaoajahmjq6nfugepebte4tigbkahifc5o6ilnnnkijyg8tnbkjdc1o+wddbpmuw7ys9glrmni0wsod4zv9x+xjiaqmscktvx7bi7kzximybjwu0uxjsn6aa6moy0anvnz9dprfot9c0/krpctgbq74mubkqna093bhshatgbiv95nqt9i27kwzhbcnl8kz8icynrgoxv5xiyiremlemub7xykerkuadw0ce4iy8vk2a14tgv56zarf1mcetjmtkhzekqc1ij16rogosrr/jmxsmb8ws8go/gx7w1z2qzh+qpjm8fucmuuq==</latexit><latexit Re( ) <latexit Pk U-Net Decoder Assume that we have a dataset consisting of noisy images with their clean counterparts sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit> sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit sha1_base64="st2bqj20yeutesdi5o+jrmrg1m4=">aaab/xicbvc7tsnaedyhvwgv8+holbkk0er2gigjakalenlisrwdz+fkldvbulsjghxxkzquietlf9dxn1wsf5aw0kqjmv3t7vgjzwps+9sorkyurw8un0tb2zu7e+b+qvvfqss0rwiey66pfeusoi1gwgk3krqln9oop7qa+p17khwlozsyj9qtebcxkbemwuqbry7qb1bhdioqfzcemvtojn2zbnfsgaxl4uskjhi0++axg8qkftqcwrfspcdowmuwbey4nztcvneekxee0j6merzuedns+ol1qpxacmopkwjrpv6eylbqaix83skwdnwinxx/83ophbdexqikbrqr+aiw5rbe1jqkk2cseubjttcrtn9qksgwmiaorkrdcbzfxibtes2xa85tvdy4zomoomn0gqriqeeoga5re7uqqy/ogb2in+pjedhejy95a8hizw7rhxifp7gyllg=</latexit><latexit <latexit Im( ) Truth Recon yk ScaDec Fig. 2. The overview of the proposed approach that first backpropagates the data into a Im( ) Re( ) complex valued image and then maps this image into the final image with a ConvNet x x 2 max pooling {yi }i=1,...,n <latexit sha1_base64="vmjngclxwcrmghlngkjfmao1zcu=">aaacfnicbvdlsgmxfm3uv62vuzecbivgopqzexshuhdjsoj9qgcymmmmdu0yq5irhmf2fotf4fbx7sstw5f+ieljyvspba7n3mo9owhcqnko822vvlbx1jfkm5wt7z3dpxv/ok3ivglswjglztdeijaqsettzug3kqtxkjfoolod+51hihwnxypoeujznba0ohhpiwx2szd7ic+ziqbeeequ3kc3br1+rfuniikwq07dmqaue3dgqmcgzmd/mcxooream6ruz3us7ediaoozkspeqkic8agnsm9qgthrfj75rwfpjdkhusznexpo1l+jhhglmh6asy70uc16y/e/r5fq6mrpquhstqselopsbnumx6xappuea5yzgrck5laih0gire11c1tcpu7exwxgmbtp667h9xfvxvwsnti4aifgdljgejtahwicfsdgcbyav/bmpvvv1of1or0twbpmizid9ful1m2fbq==</latexit> x 2 up-conv. + BN + ReLU x 3 conv. + BN + ReLU skip connection + concatenation x 1 conv. xi = f (yi ) Fig. 3. Visual illustration of the proposed learning architecture based on U-Net [45]. The input consists of two channels for the real and imaginary parts of the backpropagated vector w 2 C N. The output is a single image of the scattering potential x 2 R N. <latexit sha1_base64="k2ptv2veyd5j9b4k+etx5wksdje=">aaachhicbvdlssnafj3uv62vqks3g1wom5kioaufghuxfwwttcvmppn26ewszm7eevidfotf4fbx7sst4ni/cdjmyvspdjx7zr3co8elbndg299wywl5zxwtuf7a2nza3inv7rv0gcvkmjquowp7rdpba9yedok1i8wi9as790bxmx//wjtmyxah44j1jbke3oeugjhc8lhxk8lj6nj8hx03yaoudbmqnk1mxdhyj265ytfscfaicxjsqtkabvmn2w9plfkavbcto44dqs8hcjgvlc11y80iqkdkwdqgbkqy3usmv0nxsvh62a+veqhgifp3iifs67h0tkckmntzxib+53vi8c96cq+igfhap4v8wgaicryn7npfkiixiyqqbm7fdeguowacnnniydrk4swnsehapzxh8nuzsv0yt6eidtahqiihnam6uken1equpaex9irergfr3fqwpqetbsuf2uczsl5+atcmooo=</latexit> {xi }i=1,...,n <latexit sha1_base64="2m1umcgrgepirxzstsum17tcmok=">aaacfnicbvdlsgmxfm34rpu16lkqybfcldijgi4ucm5cvrap6axdjk3b0cqzjbmxdlpzi/wgt7p2j27duvrpzlszsk0haodz7uhendbmvgnh+bawlldw19zlg+xnre2dxxtvv6wirglsxbglzcdeijaqsfntzugnlgtxkjf2olrj/fydkypg4l6py+jznbc0tzhsrgrsiy/1qp4+zgh1sicl8bq6vej1iq2qugsbxxfqzgrwkbgfqyacjcd+mvmccci0zkipruve2k+r1bqzkpw9rjey4reakk6hangi/htyjwyegkuh+5e0t2g4uf8mussvgvpqthkkh2rey8x/vg6i+5d+skwcaclwdfe/yvbhmc8f9qgkwloxiqhlam6feigkwtpun7ml5hkn7nwdi6r1vnmnvzuv1k+kdkrgebydu+ccc1aht6abmgcdj/acxsgb9wy9wx/w53r0ysoyb2ag1tcv1mofba==</latexit> to the image domain. Weof definethe the backprojection of the measurements Training is andomain optimization weights in thegenerated by the kth transmitter as z = Pand y withreliable P, diag(u algorithm )S, (8) neural net using some fast k k in,k H where vector yk 2 C M are the measurements consistent with the kth transmitter and collected by M receivers, and matrix Pk 2 C N M is the backprojection operator. Inside the operator Pk, matrix SH 2 C N M is the Hermitian transpose of the discretized Green s function S, and u in,k is n of the incident light field emitted by the kth transmitter. The output the element-wise conjugate zk 2 C N is a complex vector with N elements, which matches the2number of pixels in the original image. When the data is collected withi multiple transmissions, i we define the backprojection of K transmitters as K K i=1 w= zk = Pk yk, (9) = arg min <latexit sha1_base64="llk2ffw8dcgmxqlggkqvw3pgcv8=">aaachxicbvfnb9qwehvsokv8leweufhdirwjvklvaqcqlbhwlblbvlrvro53smvvdij7unxl+ifwaznyg/gdookeui0jwx6e98yzelm2sjrmst9juvhg4apnrcfbt54+e/5i8hln1nwtftastarteckdkglghbivndcwuc4vnjux3zr+7bksk7x5icsgjprpjayk4bhtxeaxk7vnuadkycq4q/qfms1xute+v13luz2z91qaeap/sx4ou1ah86yyxpg8eboya3xhzfwkd1nd2xwnvwqf3k/wa/e6xzis79j19jbzov8gc8vgmb1kfdd7if+bivnfsth4w2a1aduyfio7n86zbidxyprcqdhmryogiws+h3gehmtwe9/bfujbmjnrqrbxgkr99naf59q5ps6jsnfiltcl/8enw6w+tbw0tytgxe2jqluua9rtgm6kbyfqgqexvszzqvjwaclgta11kxxnsx7xgfvg9pagj/jh0fd488qdlfkg7ji9kpop5jh8jydkrat5m7xodpnhupnup0fphxtpmqxqxpg1sl/+a0wdydk=</latexit> k k ( 1X kx n f (y )k k=1 ) find weights that produce the smallest average prediction error k=1 where vector w 2 C N is the linear combination of zk and K denotes the number of transmitters. Note that the backprojection (9) does not rely on the actual forward model H( ) in (5) which is both nonlinear and object dependent. Remarkably, as we shall see, our simple backprojection followed by a specific ConvNet architecture will be sufficient to recover a high-quality image

13 A modern machine learning solution: Train a deep neural net on a dataset full data 5x faster 5x faster + ML Example 1 Example 2

14 A modern machine learning solution: Train a deep neural net on a dataset evolution of the cost across iterations

15 Today we will talk about Motivating example from CIG Optimization as a pillar of machine learning Brief overview of optimization What will we learn during this semester? Class structure Assignments, grades, assistant instructors, etc.

16 The goal of optimization is to find the smallest value of a function under constraints Mathematical optimization problem minimize subject to f(x) x 2 X Optimization variable: x =(x 1,...,x n ) Objective function: f : R n! R Contraint set: X = {x h(x) =0} \ {x g(x) apple 0} Optimal solution x has smallest value of f among all vectors that satisfy the constraints

17 Sample applications of optimization: finance, engineering, and computer science portfolio optimization variables: amounts invested in different assets constraints: budget, minimum return objective: overall risk or return variance device sizing in electronic circuits variables: device width and lengths constraints: manufacturing limits, timing requirenemnts objective: power consumption machine learning variables: model parameters constraints: prior information, parameter limits objective: prediction error

18 Brief history of optimization Antiquity: Greek mathematicians solve some optimization problems that are related to their geometrical studies Euclid (300bc): minimal distance between a point and a line Heron (100 bc): light travels between two points through the path with shortest length when reflecting from a mirror 17th-18th: before the invention of calculus of variations only some separate optimization problems are being investigated Kepler (1615): Optimal dimensions of wine barrel. Formulated the secretary problem when searching for a new wife Fermat: derivative of a function vanishes at the extreme point the (1636). Shows that light travels between two points in minimal time (1657) Newton (1660s) and Leibniz (1670s) create mathematical analysis that forms the basis of calculus of variations

19 Brief history of optimization 19th century: the first optimization algorithms are presented Legendre (1806): least square method, which also Gauss claims to have invented Cauchy (1847): presents the gradient method Gibbs shows that chemical equilibrium is an energy minimum 20th century: the field of algorithmic research expands as electronic calculation develops von Neuman and Morgenstern (1944): dynamic programming for solving sequential decision problems Dantzig (1947): simplex method for solving LP-problems Kuhn and Tucker (1951): reinvent optimality conditions for nonlinear problems. Similar conditions in 1939 by Karush. Nesterov (1983): accelerated gradient method Karmarkar (1984): polynomial time algorithm for LP-problems begins a boom of interior point methods. Modern era: nonsmooth analysis, stochastic optimization

20 Today we will talk about Motivating example from CIG Optimization as a pillar of machine learning Brief overview of optimization What will we learn during this semester? Class structure Assignments, grades, assistant instructors, etc.

21 Practical information Course website: cigroup.wustl.edu/teaching/ese / Schedule: Lectures: Tue and Thu at 1:00-2:30 pm Additional tutorials: TBD Optional reading to complement lecture notes:

22 Practical information By the end of the semester, hopefully, you will be able to: recognize and formulate problems as optimization characterize and understand optimal solutions develop code for optimization algorithms Topics: optimality conditions convex sets and functions constrained and unconstrained optimization optimization algorithms analysis of optimization algorithms examples and applications

23 Practical information Grading policy: homework (40%) consists of 6 assignments midterm (30%) is on Thursday, 7 March 2019 final (30%): is on Tuesday, 7 May 2019 Grading convention: A A B B B C D below 60 F

24 Practical information Homework assignments will be collected only via Gradescope (use code: M2P3R8) Assignment #0 is out and due this Thursday! It is mandatory, but you will not get graded on it Academic integrity policy: You are encouraged to discuss with colleagues, but you must submit your own individual homework. No copying from colleagues will be tolerated in this class.

25 Practical information Head assistant instructors: Xiaojian Xu Hesam Mazidi Yu Sun The rest of the team: Yiran Sun Weiran Wang Jiaming Liu Joseph Han Yifan Wang Fa Long Guancheng Jiang

26 To conclude Optimization is extensively used in almost all engineering applications The goal of ESE 415 is to build solid basics that you can rely on for the rest of your career Optimization is still an active research area with many open questions

27 Computational Imaging Group (CIG) at Washington University in St. Louis (WashU) Prof. Ulugbek Kamilov Personal CONTACT INFO Web: Group If you are interested in doing research at the intersection of Computational Imaging, Machine Learning, and Optimization contact me or my PhD students