(1)、ab=0,2cosx-sinx=0,4cos²x=sin²x,cos²x=1/5,f(x)=(1+sin2x+cos2x)/(1+tanx.)=(2sinxcosx+2cos²x)/(1+tanx.) =2cos²x(sinx+cosx)/(sinx+cosx)=2cos²x=2/5；
(2)、f(x-π/8)=2cos²(x-π/8)=1+cos(2x-π/4),g(x)=1+cos(2x-π/4)+sin(2x-π/4)=1+√2[√2cos(2x-π/4)/2+√2sin(2x-π/4)/2]=1+√2[sinπ/4cos(2x-π/4)+cosπ/4sin(2x-π/4)]=1+√2sin2x,x∈(π/6,π/2),2x∈(π/3,π),当2x=π/2时,g(x)有最大值1+√2.