f(x)=(sinx)^2-√3sinxcosx
=(1-cos2x)/2-√3/2sin2x
=-(√3/2sin2x+1/2cos2x)+1
=-sin(2x+π/6)+1
(1)当0≤x≤3π/2时,π/6≤2x+π/6≤19π/6
而y=-sinx在[2kπ-π/2,2kπ+π/2]上单调递减,在[2kπ+π/2,2kπ+3π/2]上单调递增
所以当2x+π/6∈[π/2,3π/2]∪[5π/2,19π/6]时,单调递增,此时对应x∈[π/6,2π/3]∪[7π/6,3π/2],
所以函数f(x)在[0,3π/2]上的单调递增区间为:[π/6,2π/3]∪[7π/6,3π/2].