设P的坐标为(m,6/m)(m>0).则A的横坐标为m,纵坐标为k/m;B的纵坐标为6/m,横坐标为km/6.
S△PAB=PA*PB/2=(6/m-k/m)(m-km/6)/2=(6-k)^2/12.
S△OAB=梯形OCPB的面积- S△PAB-S△OAC
=(2m-km/6)*6/m*(1/2)- (6-k)^2/12.-k/2
=(12-k)/2- (6-k)^2/12.-k/2=6-k-(6-k)^2/12,
∴S=S△OAB-S△PAB=6-k-(6-k)^2/6=-(x-3)^2/6+3/2,
∴当k=3时S取最大值3/2,这时S△OAB=9/4.