о뷢չ
JOURNAL OF COMPUTER RESEARCH AND DEVELOPMENT
1999ꡡ3611ڡVol.36No.111999



VRͼмؼо
ں鴨⸣ڲšΤ
ժҪ˻ͼVRͼУδ궨תͼеĲ岹ϼȫͼ3ؼ⣬岹ϼȫͼɷ.ŵ㣺岹ͼԴͼӦӵµͼⱣ֤˲岹ͼʵԣƲ仯ɵõӵ㷽Ĳ岹ͼ񣬼Կӵ㷽ڴ˻Ͽеۻȫͼɼ˷˴ͳмȱݣʵǿ⣬Ҫȱ궨.ʵ÷򵥣Ч.
ؼʣƥͼ岹ͼϣȫͼ
ͼţTP391
THE STUDY OF SEVERAL KEY TECHNIQUES IN
BUILDING VR VISUAL FIELD
YU Hong-Chuan, WU Fu-Chao, RUAN Zong-Cai, and WEI Sui
(Institute of Artificial Intelligence, Anhui University, Hefei 230039)
AbstractIn this paper, three key techniques are discussed in building VR visual field based on real images. They are interpolation, registration, and panorama of uncalibrated rotating image sequence. The merits of their implemented methods include the following, (1)The interpolated (registration) image accords with perspective geometry. This one ensures that the interpolated (registration) image is real; (2)When the coefficient() is changed, the inter polated (registration) images corresponding to different views can be obtained. It means that the view direction can be controlled;(3)The tac-point cumulation method of panoramic images overcomes some drawbacks in the previous methods; (4) The camera needn't be calibrated. The experiment indicates that the effect is very ideal. 
Key wordsmatched matrix, image interpolation, image registrati on, panoramic images
1
ʵͼVRɼоĿǰVRоΪԾо֮һ. ͳͼѧعĸڣڼģͣȻȡһͼУԻб룬ͨʵر任Щͼλڲͬӵͼʵֻȫ.WŵڣɵĻͼӳĿ͹ʵ.漰3⣺
(1) ĳһʵͬӵµĳͼӵµĳͼ񣬼ͼ岹⣻
(2) ĳʵһֲͼɰֲͼͼͼ ⣻
(3) νʵͼƴӳȫͼ.
ڵ1⣬ĿǰʹImage MorphingеķǻԴͼ λúɫڲ壬ͼʵûжϱ׼.һȻʵԱ׼Ӧǣò岹ͼǷͬһĳӵµ.оеImage Morphingܱ֤һ㣬ʹ2DҲ.Seitzˣ1View Morphingܱ֤岹ͼʵԣ֪֪Ϣ.δδ궨ͼв岹ͼоص.ף2һӵӦδ궨ͼͼ岹ּܹ֤岹ͼĳһĳһӵµͼʵ֣˷View Morphingȱ.λ뵱ǰӦĲ岹ͼε֪岹ͼӦӵķλ̬()Щдһо.Ľδ궨תͼһ뵱ǰĳһȷ̬¶Ӧͼ岹.
ڵ2⣬Ѱһ任ʹͼصĲֶ׼ϡ һµĸӳΧͼ.漰ͼƥ䲿ֵĶӦϵʵֶӦֶ׼ı任.ĿǰʹõļõͼԱ֤͸ӹϵ.ת˶ͼе⣬ŵ㣺ͼ͸ӼιϵⱣ֤ͼʵԣϲı仯Ӧڲͬӵµϣɵõͬӵµͼ񣻴⣬÷Ҫȱ궨.
ڵ3⣬ĿǰȫͼɷΪ桢桢ʽȫͼ ʵֶձã4,5.ͳĹȫͼǣҪȷͼصΧȥصӷ촦ϴƴӳ360ȫͼ4.صΧȷѣӷ촦ϴʵвƴȫͼʱҪԤȱ궨ȼԭһȫͼʮ.ⷽо϶࣬McMillan6 Szeliski7ݵ˹Ϊͻ.Ƕݼ򻯵ģͣͨͼ йƳ༰̬ɶı궨Դȫͼ.ֱӺ ǴӡۻϴĽһ.Ϊ˷ȱݣе ۻȫͼɼ.
2תͼƥ
2.1ƥ
ģͲģ.ȡͼΪI1ת˶ȡͼΪI2ɷΪIi1Ii2I11(I21)ΪI2(I1)ص֣Ii2Ϊ˶ǰ󹫹ӳ (ͼ1ʾ).


ͼ1תͼͶӰϵ(ͷʾ᷽)
ϵ£תΪRP1=(u1,v1,w1)I1,P2=(u2,v2,w2)I2ΪռXƽI1,I2ͶӰ㣬
p1=KX, p2=KRX 
УKΪڲ(ͨ5)
p2=KRK-1p1(1)
M=KRK-1=(mij)33MΪI1I2ƥ.ֱϵ£ƥMһʱΨһȷ.ѿֻԤ֪ڲKſɲ㷨7M.δ궨ʱͨð˲㷨8,9.
2.2ƥ
ͨͼI1,I2֮N(4)ƥ㣬Ӧð˲㷨ȷƥMֵ.
(1) ֵm
(xi1,yi1)I1(xi2,yi2)I2Ϊһƥ㣬i=1,2,,Nʽ (1)ÿһƥԵõԷ̣
(xi1,yi1,1,0,0,0,-xi1xi2,-yi1xi2)m=xi2
(0,0,0,xi1,yi1,1,-xi1yi2,-yi1xi2)m=yi2
Уm=(m11,m12,m13,m21,m22,m23,m31,m32)T
ǿɵõ2Nڲ(m11,m12,m13,m21,m22,m23,m31,m32)ķ.дɾʽ
Am=b
ATAСֵӦΪmС˽10.
(2) m
С˷m
minAm-b
µĽ⣬˽(xi1,yi1)ʽ(1)õƥ (xi2,yi2)һ.
Ǹ㷨ʹﵽС.

Ȼmһm*ʹ
G(m*)=minG(m)
ǷС⣬Levenberg-Marquardt㷨10.㷨ѳΪı׼. Szeliski㷨УHessian󷽷3.Levenberg-Marquardt㷨Hessian󷽷ļ޺ ½֮ƽͣһԶСֵʱãӽСֵʱл ǰһ.㷨Ƕ޷˷ԳʼֵѡȡУܱ֤ȫŽȱݣٶ.ʼֵѡȡǷϸƹתƥѡǷȷ.ͨʼֵһΧڣ㷨ƥ0.1طΧ֮.ʼֵϴʱУƥ. 㷨ϰ˲㷨ͻصǣ٣ڶڲĹƾ.
McMillan6ͼͨ漫ԼʵƥԶƥ.ԼĻ (ʾ)Уƥ޷ʵ.Ŀǰⷽоƥ ƥʱУƥƥ侫ȿоո𲽣ûһۿ .
3תͼͼ岹
3.1ͨתͼI1,I2岹ͼI
ͼ1ʾƹתǰͼΪI1,I2߼ƥ M=KRK-1MRֵͬ1ei, Rת󣬦Ϊת.Ϊ˶Mһֽ⣬ף11ѵõϸ֤ .
. ti(i=1,2,3)ΪTĵi TΪMT=TRz()ķf=t1+it2,=t1-it2,ֱΪMֵe-i, ei,1.У

ݸڹתͼƥ󣬾ΨһֽΪ M=TRz()T-1ɴ˲ѹ תǱΪʱӦƥ
M()=TRz()T-1(2)
M()ʣ
 M(0)=IIΪλ
 M(1)=M
 M()M()=M(+).
ˣԴͼڲͬһתתӦĲ岹ͼI.岹ͼIƥPΪ
(3)
У0ܦ1Ϊ岹.
Ȼp1I1p2I2Ϊһƥʱ
M()(u1,v1,w1)T=M-1(1-)(u2,v2,w2)T
ѿIӦͶӰΪ=KR() ͬһתȽͼǲ岹ͼI.
Ϊʽ(3)ʵûΪֱʽ

У

(mij())33=M(),(mij(1-))33=M-1(1-).
δ궨ʱͨƥֽȷͼתǣ޷ȷת. ͼתӵת᲻ʱתĳһתӦͼ.궨ڲK֪ʽ(2)ɽһֽΪ
R=K-1MK=K-1TRzT-1K
ֵ1Ӧʸn=K-1t3 Ϊת᷽ʸͨڲԻõ.Ҫδ궨 . Szeliski㷨3Ϊת᷽ʸڲ ʽԱԤȶ.
Szeliski㷨McMillanͶӰУڽԤȹƺ󣬵У˽ֱֵӰƥM.ͨԴͼУ Իýྫȷֵϴ󣬴УȻۻŴõ ƥ󾫶Կ.ƥֽⷨԭʼƥMκνƺ͹ƣת仯ԼΪһɱӲԿƣ ǼᱻۻŴõƥ󾫶ȿЧƣ³ǿ.
3.2ν
岹=0ʱI1䣬I2I21ĲֱͶӰI1ʱ岹ͼ I=0=I1I1ͶӰΪ=K I1ͬͼ2.


ͼ2岹=0ʱ
=1ʱI2䣬I1I11ĲֱͶӰI2ʱ岹ͼI=1=I2I2ͶӰΪ=KR()I2ͬ ͼ3.


ͼ3岹=1ʱ
01ʱI1I2ĹӳI12I22ͶӰI(1,2)I11ͶӰI(1)I21ͶӰI(2)ʱ岹ͼI=I(1)I(1,2)I(2)ͶӰΪ=KR()ͼ4.


ͼ4岹01ʱ
岹ȡֵͬͼIӦͬһͬתµͼ.
SeitzView MorphingΪʵƽ˶ͼ岹ҪԷƽͼԤУ.ķתͼв岹ԤУͼ.⣬궨.
3.3ͼɫ
(x,y)Iʽȷɫ
(4)
Уc(x, y)ʾ(x, y)ɫ.ȷƥʱ岹Ӧɫʽ(4)δƹתı£ıᵼ²岹ͼɫ仯Ϊȷɫ仯ɲõ뵭岹ͼδƹתͨᵼ²岹ͼӰ(ģ)ֻҪƲɲ岹ͼݵԴͼ񷽷Ч˷ͼӰȷ岹ͼصΧΪɫֵ뵭ṩ.
4ͼе
4.1תͼΪͼI
תͼϣ䴦3ͼ岹ͬ.תͼ±任.
תͼƥMi=KRiK-1, RiΪͼIiIi-1ת
p1=M1p0
p2=M2p1

pi=Mipi-1
ɴ˲ƳתͼIiI0ƥӦתתǦ(i)=ik=0kʽ(2)ƥҲƷֽ
M(i)()=TR(i)z((i))T-1
Ϊϲ0ܦ1.ͬƵϹʽ

Ϲʽβͼͬһӵµı任ϵмͼ֮ƣƥϵɵƵ.
ӵڳʼͼI0תǶΪi=(i)βͼϺתͼIi-1I1ʱi(i)Ӧʵi-1=i-i(i)=(i)-1-iϲΪ

ƥҲӦΪɴ˲ѵnתͼϹʽ
(5)
ͼIӦͶӰ=KR((n)).
4.2ν
ͼϵļϽͼ岹ļν.ͼ5ʾ3ͼϹϵ.ԿͼI0,I1,InзصֱֱϵI(0),I(1),I(n),صI02I11ϵI(0,1)I12In1ϵI(1,n)ʱͼI=I(0)I(0,1)I(1)I(1,n)I(n). ϲȡֵͬͼIͬһӦڲͬתӵµͼ.


ͼ5ͼϵļ
Դͼ޷ƲͬתĲ岹ͼ񣬱һƲͬתԴͼ񣬷ʽ(5)ͼһ(ʹ)ɻƲͬתĲ岹ͼ.
5ȫͼ
ȫͼ(panoramic images)һʵͼȫռļȫͼʵֶձ. Apple Inc˾ƳQuicktime VRϵͳ4ݣͨȫͼϿʵʱûҪ۲⻷ͬȫͼлʵ⻷еΣʹûشӲͬӵͷ˽⻷.ȫͼɷܶ࣬ձͼصΧȷѣӷ촦ϴʵвƴȫͼʱҪԤȱ궨ȱݣMcMillanSzeliskiδܱЩ.еۻȫͼɼЧؿ˷ȱݣЧ.
ͼ6ʾͶӰ温ͼͼIͶӰʱеCı⣬߶ͶӰACCB䣬ΣAC,CBȻͼصΧȷӷ촦ϴ.360תӵµ񣬲еۻɱⲢȫͼ.3תͼͼ岹ɲ岹ͼתǶȶӦͼһתͼÿͼԼתͼ岹ۻе㣬ɻȫͼ.Ҫעǲ岹(仯ˡе㡱ֿխӦͼֱʶ.)


ͼ6ͶӰ温ͼ
6ʵ
ñͼ岹ʵ飬ص岹ͼģɫʹɲ.ͨСʱͨͼݺɫֵ뵭ɻýΪЧ.
ͼ7(a),(b),(c)ֱΪڹͬһתͼУͼ7(d)3ͼͬһӵµͼ.ϲ =0ͼ7(a)Ӧӵµ.ͼ7(d)ԿϵĽǷǳ.



ͼ7ͬһתͼе.(a)(b)(c)Դͼ(d)ͼϲ=0
ͼ8(a),(b),(c),(d)ֱΪڹƲͬתͼ(e),(f)4ͼͬһӵµͼ.ͼӦԴͼ8(a),(d)=0.5ӵ.



ͼ8Ʋͬתͼе.(a),(b),(c),(d)Դͼ(e),(f)ͼϲ=0.5
ͼ9ͨ10תͼɵȫͼ.ԿûнӷۼЧ.ȫͼϸΪʾݺЩʧ.


ͼ9еۻɵȫͼ
7
ĽԲؽͼVRУͼĲ岹ϼȫͼ3ؼ.ŵ㣺岹ͼԴͼӦӵµͼ岹ͼ͸ӼιϵⱣ֤˲岹ͼʵԣƲı仯ɵõӵԷȷĲ岹ͼڴ˻Ͽеۻȫͼɼ˷˴ͳмȱݣʵǿ⣬Ҫ֪ڡ.ʵҲ򵥣Ч.˱ķʵɡͼбؽзǳ㷺Ӧüֵ.ʵȫͼԼĲ岹ʵʵεĹؼҲһоص.
*õȻѧ (Ŀ69875001).ں鴨У19696ʿоҪоΪӾʵ.
߼飺⸣У195710 ڣҪоΪӾ.
ڲţУ1970 5ʿоҪоΪӾʵ.
Τ룬Ů19462ڣʿʦҪоΪӾʵͼ.
ߵλմѧ˹оϷʡ230039
ο
1Seitz S M, Dyer R. View morphing. In: Proc of ACM SIGGRAPH96, New Orleans, 1996. 2130
2Yuan Bo, Wu Fuchao, Yu Hongchuan, Wei Sui. Image interpolation correspondence with viewpoint. In: Proc of Int Symposium on Multispectral Image Processing, SPIE Vol 3545, Wuhan, 304308
3Werner T, Hersch R D, Hlavac V. Rendering real-world objects using view interpolation. In: Proc of ACM 15th Int Conf on Computer Vision, 1995(ICCV). 957962
4Chen S E. Quicktime VR: An image-based approach to virtual environment navigation. In: Proc of ACM SIGGRAPH95, Los Angeles, 1995. 2938
5http://www.bdiamon.com
6McMillan L,Bishop G.Plenoptic modeling:An image-based rendering system.In:Proc of ACM SIGGRAPH95,1995,Los Angeles.3946
7Szeliski R, Shum H Y. Creating full view panoramic image mosaics and environment maps. In: Proc of ACM SIGGRAPH97, Los Angeles, 1997
8Irani M, Anandan P, Hsu S.Mosaic based representations of video sequences and their applications. In:Proc 5th ICCV95. 1995.605611
9Szeliski R. Video mosaics for virtual environments. IEEE Trans Computer Graphics and Applications, 1996, 16(2): 2230
10Press W H, Teukolsky S A et al. Numerical Recipes in C: The Art of Scientific Computing(2nd Edition). Cambridge University Press, 1992
11⸣,ں鴨. Ա궨㷨. Զѧ, 1999, 25(6)
(Wu Fuchao,Yu Hongchuan et al.Camera Self-CalibrationTheory and Algorithms.Acta Automatica Sinica(in Chinese),1999,25(6)
ԭյڣ1999-03-17޸ĸյڣ1999-06-21.
