The key point of the two is the feeling of "in" and "into".
Look up in the sky:it is just a body movement which means there is something need your attention on the direction of the sky.
Look up into the sky: this one is something with feeling. When you feel happy, anger or anything, someone may smile or cry. But someone may just look up into the sky. This is a body language.