remote desktop connection to a potentially compromised machine

I am using rdesktop client on Linux to connect to windows machines. If some of these machines were compromised by some evil malware/virus/worm/trojan, is there any way for the malware to "jump" to my computer over the rdesktop connection?

I have always thought this is impossible, but recently I have read that similar technology, x2go is inherently insecure.

EDIT

To clarify my question, I am mostly interested whether the rdp protocol (or session) can be potentially dangerous for the client.

I don't understand the internals of the rdp protocol. But it seems to me, if the rdesktop client is just rendering images/bitmap then there is not much a potentially compromised server can do. If, on the other hand, the rdesktop client is interpreting some commands sent from the server, it could potentially be exploited.

For analogy: Modern browsers do not just display static HTML webpages. Instead, they interpret Javascript, Flash, ... Is a rdesktop client similar in that it interprets potentially dangerous commands from the server?

If the only think the client does is rendering images, than apart from some bug in rendering library, it cannot be exploited. That would be comparable to a image viewer viewing jpg images.