Do you have a fire place?
The person could have been a burglar, peeping tom, or rapist, but it's also possible the person was looking for his missing cat. The fact that he never actually tried to break in is a positive sign. I don't think you need to go to a shelter. If you can, and this might be regulated in your area, buy some pepper spray. Try to locate something you can sound that's very loud, like a fog horn or a whistle, so that if he tries breaking in, the loud noise will scare him off. Most criminals are scared when they're committing crimes, so chances are he'll flee when confronted. And, if he doesn't, that's what the pepper spray is for (assuming you can get it). Check that your windows and door(s) are secure. If not, talk to your landlord about what happened and ask if he can address that ASAP. Best of luck. I really think you're okay.