On my MacBook, is my user password now the Achilles heel of disk encryption?

From a cold boot, the only password needed to unlock the primary encrypted volume on my MacBook's SSD is just that of my main admin user. This was not always the case; it used to be that I could specify a fairly long and thorough passphrase to be used in key derivation whenever I had to access the volume in its locked state, then a separate passphrase for the user account after unlocking the partition.

It is very inconvenient for me to type out such a lengthy passphrase for every admin-level action I might need to perform during casual use. It seems unlikely that Apple would roll out such a bold detriment to one of either data security or user experience, so I'm confident that I've missed something. What (if any) safeguards exist to protect against brute force now that the only user input required to decrypt my whole system is a frankly modest password at the boot screen? I understand that it may wipe after N attempts, but of course I am speaking more about the risk of brute force given physical access to the drive/access through an agnostic interface.

Another question: is there any way for me to revert back to the old experience without emulating it with a new user and promoting it to the primary administrator of the system? For what it's worth, I am using OS X 10.10 (Yosemite). I believe that change of experience/apparent ability came during my use of the prior major release of the operating system, but it may have occurred earlier.