∵sina=√2cosb,∴(sina)^2=2(cosb)^2,∴1-(cosa)^2=2-2(sinb)^2,
∴(cosa)^2=2(sinb)^2-1.
又tana=√3cotb,∴(sina)^2/(cosa)^2=3(cosb)^2/(sinb)^2,
∴2(cosb)^2/[2(sinb)^2-1]=3(cosb)^2/(sinb)^2,
∴2(sinb)^2(cosb)^2=6(sinb)^2(cosb)^2-3(cosb)^2,
∴4(cosb)^2[4(sinb)^2-3]=0.
∵0<b<π, ∴cosb=0, 或sinb=±√3/2,
∴b=π/2,或b=π/3,或b=2π/3.
由b=π/2,得:sina=0; 由b=π/3,得:sina=√2/2; 由b=2π/3,得:sina=√2/2.
∵-π/2<a<π/2,∴sina=0不合理,即b=π/2不合理,应舍去. ∴a=π/4.
综上所述,得:满足条件的a值为π/4; b值为π/3或2π/3.