That man is a friend of my brother‘s是对的
a friend of my brother‘s=one of my brother's friends 要用所有格表示所属关系