(1)c/a=√3,
右焦点(c,0)到一条渐进线:bx-ay=0的距离=bc/c=b=√2,
∴c^2=a^2+2=3a^2,∴a^2=1,
∴双曲线方程是x^2-y^2/2=1.①
(2)把y=x+m代入①,2x^2-(x^2+2mx+m^2)=2,
整理得x^2-2mx-m^2-2=0,
设交点A(x1,y1),B(x2,y2),则x1+x2=2m,x1x2=-m^2-2,
y1y2=(x1+m)(x2+m)=x1x2+m(x1+x2)+m^2,
由向量OA*向量OB=0得
0=x1x2+y1y2=2x1x2+m(x1+x2)+m^2=2(-m^2-2)+2m^2+m^2=m^2-4,
∴m^2=4,m>0,
∴m=2.
O到l:x-y+2=0的距离=2/√2=√2=圆O的半径,
∴直线l是圆O的切线.