Any intelligence capable of interstellar travel at practical timescales would not bother with our planet. There are much easier planets to go to for resource harvesting, even in this system. If you want water, you've got tons of comets that are already in space that you can harvest from, let alone smaller planets with less gravity to deal with.
No, a Von Neumann self-replicating probe swarm is probably the greatest extraterrestrial threat we might face.