How do I know my tools aren't compromised?

After reading different articles, most notably/concerningly:

• If the NSA has been hacking everything, how has nobody seen them coming?


• Ken Thompson's Hack (more the concept than specifics)

I am wondering: How can I be sure that my toolchains and application software are not compromised?

The obvious but elongated answer is to write a compiler in machine language for a specific processor, then an OS, and so on up from there. (Though this does not discount the possibility of hardware compromise.) Open source software is fantastic, but that does not remove the possibility of a Kevin Thompson-esque hack such as not allowing a compiled application to open a connection on TCP Port 12345 (for example), and seamlessly bridging it to (say) 12344 unless certain conditions are met. (Critical for an application like nmap).

How likely are these essentially "invisible" exploits that are baked into our software without our knowledge?