(1)
OP=(2,1),OA=(1,7),OB=(5,1),
let M be(x,y)
M is on OP
=> x/y = 2/1
x=2y
MA.MB
=(OA-OM).(OB-OM)
=(1-x,7-y).(5-x,1-y)
=(1-x)(5-x)+(7-y)(1-y)
=(1-2y)(5-2y)+(7-y)(1-y)
=12-20y+5y^2
=5(y-2)^2-8
min MA.MB = -8
(2)
y= 2,x = 4
OM = (4,2),MA=(-3,5),MB=(1,-1)
MA.MB=|MA||MB|cos∠AMB
-8=√34√2 cos∠AMB
cos∠AMB = -4√17/17