ѧ
JOURNAL OF SOFTWARE
2000Vol.11No.4P.532-539




Сռͼκϳ
쵤衡ʯӢ
ժҪһµĻ2DɢС任ͼμͼϳɷ.÷ȫ֪һͼϳڼı任ͼ,궨Ϣ.˼ǣ,ӾҳͼԼĻ,ͼ任һ淶ģʽ,ʹͼƽƽ,ҶӦλͬһɨ.Ȼ,һСռͼη,Կٵڼı任ͼ,һڽһСϵ.ʱ,ҪĽ,ͨС任ϳɵСϵָͼռ.,һñ任,ڼͼӹ淶ģʽصԭģʽ.÷صǣС任ͬʱƵͿ,һṩһֱֶʵıʾģʽ,㷨.,»ԲͬС͵Ӱ.
ؼʡͼϳ,С任,ͼ,,IBMR(image-based modeling and rendering),.
ͼšTP391
View Synthesis via Wavelet-Based Morphing
XU Dan
State Key Laboratory of CAD & CG Zhejiang UniversityHangzhou310027
BAO Paul
Department of ComputingThe Polytechnic University Hong Kong
SHI Jiao-ying
State Key Laboratory of CAD & CG Zhejiang UniversityHangzhou310027
AbstractThis paper presents a new view synthesis technique using image morphing and 2-D discrete wavelet transform. The paper aims at pairwise images that are known without camera calibration and any other geometric information. Firstly, estimate the fundamental matrix related with any pair of images by techniques from stereovision. Using fundamental matrix, any pair of image planes can be rectified to be parallel and their corresponding points lie on the same scanline. Secondly, the paper describes an image morphing algorithm running in the wavelet domain. The proposed algorithm can synthesize new views more quickly. Any quantization techniques can be embedded here to compress the coefficients in depth. When displaying, compressed images are decoded and an inverse wavelet transform is achieved. Finally, a postwarping procedure is used to transform the interpolated views at desired position. Nice features of using wavelet transformation are its good locality in both space and frequency and its multiresolution representation mode, which makes the generated views be refined progressively and hence is suitable for communication. Also, the paper discusses the influences of different kinds of wavelet on the results.
Key wordsView synthesis, wavelet transform, image morphing, fundamental matrix, IBMR (image-based modeling and rendering), epipolar line.
IBMR(image-based modeling and rendering),ͼĽģͻ,ǽͼѧͼӾۡоĽ.ڼģ,һԤȻõͼ볡,ͨʵĲֵϳλڲͬӵͼ.Ŀǰ˺ܶIBMR1,ȫӺķӾķͼֵͼεķͼƴϼֲƵ.лͼ(view morphing)ͼϳɷSeitzDyerSIGGRAPH'96һֲͼμķ2.֪,ͼ(image morphing)״ɫȫͬͼ֮һϵڼͼ,ЩڼͼĿʼսͼȫͬ.෴,ͼǿĺ,ɵڼͼӦñʼսͼ״.,ͼεӦòڲͬӵӦ͸ͼ.
Seitz˵ͼηܽΪ3.,ʼսͼ任һ淶Ľṹģʽ,ʹǵͼƽƽ,ҶӦλͬһɨ,һΪɨ.Ȼ,ڼͼͨһͼμ.ɨԱ֤ͼεĽ״.,ͨһñ任мͼԭسʼĽṹģʽ.IBMR,ͼηŵ,򵥡ҪϢ,ڵ(monotonicity)Լ¿Բǿɼ3.ȻŴͳͼηٵ,⽫ļ.
ͼμһּΪձͼμ,һϵ仯ڼͼ,ʹʼͼ𽥵ر仯սͼ.еĹؼδ֪ͼӦ()ݳһӦı任(warp functions)4,ҲΪݺ(transition function).ʵ,ݺһ2Dα任,һ棬ʹʼͼսͼķŤ,һʹսͼ෴,γԵʱ.,ͬһʱ̵ͼϳɸʱ̵ͼ.
Mesh-Base5ݺField-Based6еͼη.ǰһַ,ָΪǾȵĿ,任ֵͨ.ȱÿһʱ̶ҪϢ,ĿƲݽΪ.һṩõĽӿָʼĿͼе߶.任ͼʱ,ÿһ߶ζһӰ,ͼ߶ӰļȨƽȷ.ЩһͬȱǷǳʱ,ʱȡͼֱʺĿ.,ͼĸƵӰ.
֪,ĳֶǽռתƵ,˵Ƶ֣һ,η㷨Чʵļ;.㿼,˻Сķ.ΪСֽⲻṩһֱֶʵĲαʾṹ,ͬʱֱ˿Ƶþֲ.ʵ,Ѿ߷Ƶռάμ.,ף7һֻСķʵݼıΣ˻Fourier任άη8.
һIBMRӦõĿͼη.÷Ƕף2㷨һָĽ,صڽͳͼμֲСռ,㷨Чʣ.,ǵ㷨ϵ,ǶֻСռͼ,˲ȵ.ǳͼƵݵͨԼWebIBMRӦ.
ǵķ4ɣI1I2֪ͼ.
1. .û(fundamental matrix)ҳ任H1H2,οͼ任淶ģʽ,ʹH1I1H2I2ͼƽƽ,ɨ.һԱ֤ͼĲֵԵΪ״ֵ.
2. Сռͼ.2DС任ͼֽɽṹ,ȻField-BasedͼμزֵмͼСϵ.
3. .С任ϵԽһڴ洢ͨ.,ԲVQ.ʾʱ,ݿԺܿؽ.
4. ع.С任ֵڼϵ任ͼռµͼ.Ȼ,һñ任ڼͼɹ淶ģʽ廹ԭģʽ.
1ͼУ
ǵǰڸһͼϳ,ֻ֪οͼ,ûκ.һʺIBMRӦ.ΪˣҪһЩӾ֪ʶ.Ӿ,ͶӰռ(Ǵŷ¿ռ)֮ļιϵ,Ϊ(epipolar geometry)ϵ.֮ҪϢһ33Ϊ2ľ,ƸþΪ(fundamental matrix),ΪFM9.
ͼ1ʾ,ͶӰΪC1C2,ͼڵƽֱΪI1I2,ƹͶӰĵֱΪ(baseline),ΪC1C2.C1C2I1I2ϵͶӰֱΪe1e2,Ϊ(epipole).I1ͨe1ֱߺI2ͨe2ֱ߳Ϊ(epipolar line).֪I1I2Ӧmm,I1Ϲmļ߳Ϊmļ,Ϊl䣻ƹmļΪmļ,Ϊl.

Fig.1 Epipolar geometry between two cameras
ͼ1ļԼ
FMһͼȫļϢ,ǵҪҪƳFM.Ϊ,Ҫ֪οͼӦϵ.ѾһСռԶѰҾֲӦƥķ10,ȱǷǳʱ.һּ򵥶ֱӵķǽָ8ԶӦ,ܱ֤.ף9ݷòͬFMȶԺͲȷ.
ͼI1еһm,ӦͼI2еļlʾΪ
l=Fm(1)
ӾļԼ(epipolar constrain),I1һmI2еĶӦmһڼl,
mFm=0(2)
֪8Ӧ,FĲС˷ԵķƳ.ԷͨԷȷȶ.ǵķ,ԷһʼֵF0,Ȼ÷ԵĵҵȶϾȷĽ.ϸۺͷף9.
Ϊ˽ͼһ淶ģʽ,ҵ任H1H2,ʹH1I1H2I2ɨ.ף2֤ɨԵĳֱҪǣFMʽ
(3)
ʽ(3)һ.Ҳ˵H1H2
(4)
ף2ͬʱҲF任H1H2ķ.
2άС任
СźŴļ11,12,Ŀǰѹ㷺ͼѧ,磬͡ȡ߸١ӳ䡢Ƽͼѹ.ԭһ棬С任ڿƵкܺõľֲԣһ棬СֱʷΪźűʾṩһֲνṹ.С任һֽΪֱͬʵĲ,ÿһ4,һǶԭƽ,Ϊ(approximated band)3ԭĲ,Ϊϸ(detail band).ֱʾΪĳЩͼ㷨ṩ˿.Ҫһ2DСֱʷֽ.
ʽ,һfL2(R)ڷֱ2iϵƽһͶӰ,ΪA2jL2RV2j,V2jL2Rϸ߷ֱ2jϵϸfL2RӿռO2jϵͶӰ,ΪP2jL2RO2jji.ǿʵѡȡͶӰ,ʹǱ˴,ҶV2i.ɢ,V2iO2j,ֱΪСͳ߶Ⱥ,Ϊ
jn2j2ף2jtnnZ(5)
jn2j2գ2jtnnZ(6)
,ֱ2jϵϸֱΪ
(D2jf)=2-j/2fujn(7)
(Ad2jf)n=2-j/2fujn(8)
ʽ(7)ʽ(8)ڻúĽֽ.ڶάɢͼ,㷨ÿһάֽ.ͨһͨ˲һͨ˲ʵ,ͼ2ʾ.HGֱ˲Ӧ,Ϊ
H(n)=2-1(2-1u),(u-n)(9)
G(n)=2-1(2-1u),(u-n)(10)

Fig.2 Wavelet decomposition for image
ͼ2ͼĶάСֽ
ͼ3һС任.ע⣬ÿһֱʲϵϸ򶼱˿ռֲ,ͬʱС任ͨͨ͵ͨ˲ѸƵ͵Ƶָ.½ǵƵ,ϽˮƽƵֱƵ,½ǴֱƵˮƽƵ,½ǸƵ.Ϊȡṩ˷.


Fig.3 Two-Dimensional wavelet transform
ͼ32DС任
3Сռͼμ
С任Ŀռֲ,˿ǰѴͳ㷨ֲС任ÿһͨͼ.ΨһҪСռжλӦ߶.
3.1ֲ
ڸСеλõһּ򵥷ǽռÿһάԳ2j-n,2nԭʼͼֱ,2jǵǰķֱ.
ȻʵеСһ򵥵Ŀռ,ΪͼɢС任ǻʸǻڵ.Ҳ˵,Сռÿһ㶼ͼռжصӰ,ظȡڴͨ˲ĳ.,ͼռÿһضСռĶϵй.֮,Ǳ֪С任һֲ.Ϊ,ǽһͼ(ֻһرͼ)任Сռ,۲ֲ.ͼ4һͼڲͬС任µֲ,ϵԽ,Խ.


Fig.4 Energy distribution of various kinds of wavelet
ͼ4ͬС任ĵֲ
С任븺ֵϵ,ͼ4ʾʱȡ˾ֵ.,ЩӰϵΪ̫СȻ.ͼ4Կ,˲Խ,ķֲԽУԽӽλ,Գ2-1С.෴,СԽƽ,ֲԽ.ע,ͼ4ʾֲһͨ.
3.2λõȷ
Ѿ˲ͬСѡȡӰ첻ͬСߵȷ.Ϊ˾ȷءӦػڸ,һ򵥵㷨ٵؾԭ߶һµλ.һζ߶οƵѡȡĺ.
㷨. forÿһֱʲl
forÿһ߶μÿһ˵
(1) ͼ任ĳѡСռ,ͼĴСʼͼսͼͬ
(2) ϸѡȡǰ˵λã
(3) ÿнһ뵱ǰ߶ζӦµ߶Σ
end for line
end for level
ϸѡȡǰ˵λÿвͬĹ,ѡСֵԼֵϵ.ǵʵһϺ,ѡĵΪµƵ.
3.3Сռеı
һȷĳһϼĳһе߶μ,ͿԽһͼռеı㷨ֱӦõСռ.ȱʼսͼСϵ,ǲֵڼͼСϵ,С任ͼռͼ.ͼ5ʾһͼıϵ,һת.е1һȡSeitz'sҳ,Ƿֱʼսͼ.


Fig.5Sequence of morphing image
ͼ5ͼϵ
3.4ͬСıȽ
ڲͬСвͬ,õ3.2нڽܵķõ߶ܱ˴˲ͬ.ζű㷨ÿһ,ǳַΪ㷨.,Сķƺûдﵽٱ㷨Ŀ.ͨͬͨߵ,Ƿ֣СĵѡС,ʹǵȫͬ.
3.4.1 Сı
HaarСDaubechies-4С򵥵С.ߵÿһ.ˣHaarDaubechies-4,һνĿٱ㷨,ֻÿϵıλ,Ȼֱһ3ıϵ.ֻѴͳ㷨ķ֮һʱ.ͼ6HaarDaubechies-4Сı,ǱȽģͼ7˿㷨͸㷨ıȽ.ͼеĽʾʼֹͼı仯Умһͼ.СԽƽ,˲Խ,4Խ,ʱٱξͻʧ(ͼ7(a)ʾ,ӱǳģ),ֻ㷨ؽм.


Fig.6 Fast view morphing of simple wavelet
ͼ6СĿٱ


Fig.7 Comparison between the fast algorithm and the complex algorithm
ͼ7㷨͸㷨ıȽ
3.4.2 ˫Сı
ǰѾ˿㷨ֻʺϼ򵥵С,㷨ͨΪƽСṩһϺĽ.ʱȴڴͳı㷨,һ(Ƚͼ7(b)).Ϊٻøıͼ,ǿһν˫ԳС.
ͬǰС,˫Сŵڣ˲Ǵʸһеģߵۺʱȵ.ˣǿֱȷڶֱСеλ,߿ƵسԳ2,ͨ㷨.Ȼ˿ֱ̿Ӳÿ㷨.ͼ4(b)(d),˲ж೤,ķֲ,λΪ.ʧһ,ֻСʵ,ͼ8ʾ.


Fig.8 View morphing via biorthogonal wavelet
ͼ8˫Сıν
ͼ˵,С,㷨ܻúĽ.
4ƺϳ
ǰѾСռĶֱʱʾɲֵСϵ.ڽع,ָǵ㷨ϵǳʺƵͼͨ.
4.1ñ任(postwarp)
Ϊ˲ͼռڼͼ,ֻ轫СռıνС任صͼռ,ֻҪ12ʱ.ʣµǰЩͼӱ׼ģʽָģʽ.
һ򵥵ЧķԴοͼзֱѡһıε4Ƶ,ȻʱԲֵڼı.ıμ䶼Բ任,ʵͼ״ı任.,ʼοͼͬһڼ״,Ȼٶǵɫֵϲɫ.
4.2ɼԷ
ӵı仯໥ڵ,ʹЩԭɼĳĳЩͼвɼ,һЩԭɼĳͼֱÿɼ.⽫ºϳͼгһЩնص.ͼ任ĽǶ,صΪرӳ䵽һ.նΪر任,.ʹһЩɼǰ㱻ñɫ.IBMRӦ,նǺ,Ϊ޷ԭͼлÿնλõϢ.,ֻ򵥵øֵ֪,ͼЧ.оһµĽ,ͼָܹÿһϢ.
4.3ͨŵص
ܵͼηһŵǱͨ.,С任Ķֱ,ϲǳصõһԵĽ,Ȼý.һصǳʺͼƵĴ,ǷֲʽIBMRӦ.,ΪСͼѹǷǳļ,һЩͿֱǶ뵽Сռ,VQ.Դٴ.ȻҪڿͻ˽нͻ.
5ʵ
Ѿ֪ͳı㷨ִЧȫȡߵĸͼķֱ.Сķܼ֮ٱ㷨Ч,ԭ֮һСֽԿǶԭͼӲ.ǵĿ㷨,С任ֻֽһ,ϳٶ3.ʾͼ58.,С任ͼתƵռ,ǶһЩƵռķ,ȥ롢˵.һͼռеı㷨ܵõЧǿʹͬƵвͬĴ.,õƵֿٹ,Ƶֻ,֮Ҳ.֪ƵֶӦͼϸ,ƵַӳͼȫϢ.ˣı䲻ͬƵɷֵĴʿԵõһЩȤڼͼ.
СηȱǽȴͳһЩ,ⲢСռı㷨ɵ,ΪС任Ӳ.ǱȽǵĽ봫ͳһͼֱµĽ֮,ȫһ.Ϊ˸Ľͼ,԰СνĸƵִ֮ԴͳĸƵ.һǣڷŴԭͼѡ,ٰ߼ԭͼԭĴСʵʩ㷨.ĿҪָߵľ.
6 
,һIBMRӦкϳͼЧͼη.÷ĵͼβַŵСռȥʵ,Ӷ㷨ִеЧ,ͬʱ.˲ͬСѡԽӰ,ͬʱҲӦĽ.Ҫָ,÷Ӿ֪ʶйͼļԼϵ,ûж,ҪУ.,Сķܹ󾫵Ľ,ʺWebIBMRӦ.
쵤㽭ѧCADCGصʵҡݡ310027
裨ѧӼϵۣ
ʯӢ㽭ѧCADCGصʵҡݡ310027
ο
1Xu Dan, Pan Zhi-geng, Shi Jiao-ying. Techniques of image-based rendering in virtual reality. Journal of Image and Graphics of China, 1998,3(12):10051010
(쵤,־,ʯӢ.ʵлͼĻƼ.йͼͼѧ,1998,3(12):10051010)
2Seitz S M, Dyer C R. View morphing. In: Proceedings of the ACM Special Interest Group on Computer Graphics (SIGGRAPH'96). New York: ACM Press, 1996. 2130
3Seitz S M, Dyer C R. Photorealistic scene reconstruction by voxel coloring. In: Proceedings of the Image Understanding Workshop. 1997. 10671073
4Wolberg G. Digital Image Warping. Los Alamitos, CA: IEEE Computer Society Press, 1990
5Nishita T, Fujii T, Nakamae E. Metamorphosis using Bezier clipping. In: Proceedings of the 1st Pacific Conference on Computer Graphics and Applications. Seoul: World Scientific Publishing Co., 1993. 162173
6Beier T, Neely S. Feature-Based image metamorphosis. Computer Graphics, 1992,26(2):3542
7He T S, Wang S, Arie K. Wavelet-Based volume morphing. In: Proceedings of the Visualization'94. Washington, DC: IEEE Computer Society Press, 1994. 8592
8Hughes J F. Schedule Fourier volume morphing. In: Proceedings of the ACM Special Interest Group on Computer Graphics (SIGGRAPH'92). Computer Graphics. New York: ACM Press, 1992. 4346
9Luong Q T, Deriche R, Faugeras O et al. On determining the fundamental matrix: analysis of different methods and experimental results. Technical Report, No.RR-1894, NRIA (Institut National de Recherche en Informatique et en Automatique), 1993
10Xu Dan, Bao Paul, Shi Jiaoying. Image mosaics based on complex wavelet decomposition. Journal of Softwae, 1998,9(9):656661
(쵤,,ʯӢ.ڸֵСֽͼƴ.ѧ,1998,9(9):656661)
11Stollnitz E J, Derose T D, Salesin D H. Wavelets for computer graphics: theory and applications. San Francisco, CA: Morgan Kaufmann Publishers, Inc., 1996
12Mallat S. Multiresolution approximation and wavelet orthonormal bases of L2(R). IEEE Transactions on Pattern Analysis and Machine Intelligence American Mathematics Society, 1989,11(7):698
