(1)f(x)=cosx(asinx-cosx)+cos2([π/2]+x)
=[a/2]sin2x-cos2x,
由f(?
π
3)=f(0),解得a=2
3
故f(x)=[a/2]sin2x-cos2x=
3sin2x-cos2x=2sin(2x-[π/6]),
故单调递减区间为[kπ+
π
3,kπ+
5π
6],k∈Z
(2)由
a2+c2?b2
a2+b2?c2=
c
2a?c,可解得2sinAcosB=sinA,
∵sinA≠0,∴2cosB=1,即cosB=[1/2],又0<B<π
∴B=[π/3],又A+[π/3]>
π
2,
∴[π/6<A<
π
2],
则f(A)=2sin(2A-[π/6]),[π/6]<2A?
π
6<
5π
6,
∴2×
1
2<f(A)<1×2,即f(A)∈(1,2].