This is not an easy question to answer.Let's discuss some examples:
Guangzhou is in the south of China.--- Duangzhou is in China,but in the south of China.
India is on the south of China.--- India is not in China,but on the south of China.
To say "at the south of..." is not common.It appears in some phrases such as:
He arrived at the South of Human.