Is it possible to use an open collection of default SSL certificates for my browser?

Browsers and OSes come with a set of default SSL certificates for well known organizations, such as root certificates from CAs.

I understand that the trust model is as follows: I pick (actually my software's developers pick for me) a few major certificate authorities and decide I trust them. While I hardly ever go to the sites of these CAs, the sites I do go to use certificates that are dependent on the CAs, such that I automatically trust any certificate descended from a root certificate that I trust.

However, I actually trust the sites I visit more than the root CAs. So, I don't want to use this tree-based model. Instead, I would like it to be my personal responsibility to individually verify and store the certificate of every site I visit. I understand that this may introduce risks since my own resources for detecting compromised certificates are limited.

It seems like I could do this today: I can delete all my root certificates, and just start adding exceptions for every site I visit (assuming that I decide to trust them). However, this would introduce a very frustrating adjustment period where my browsing is frequently interrupted by having to constantly add exception for sites I browse frequently.

I further imagine that this impracticality can be easily solved: Many sites I visit are uncontroversial, trustworthy, straightforward cases like Google, StackExchange, news sites and so forth. If a few random strangers on the internet all agreed that a certificate is the legitimate SSL certificate for www.cnn.com, I could believe them and just use the certificate they gave (what are the odds that these people have compromised the certificate of CNN and at the same time conspired to fabricate an apparent consensus in favor of the forged certificate?). So, there could be a crowd-sourced online repository of common, uncontroversial sites so that I can simply download their certificates in bulk every year or so and massively reduce the number of exceptions I need to add. Obviously, sensitive cases like my bank would not be included in this repository - I would manually vet that certificate myself after careful consideration.

This would eliminate my need to trust root certificates, and eliminate the risk of compromised or malicious CAs. In exchange, the new risk is that there happens to be a very concerted operation to poison the repository with forged cetrificates of an inconsequential site that has been compromised just as I download my certificates for the year (I can further compare those to the ones I already have and scrutinize unexpected changes). The impact on usability is negligible: After installing my browser, I just need to download a zip full of certificates and dump them into my certificate cache. I would now need to personally decide whether to trust obscure or sensitive sites instead of automatically trusting them through the root certificate, but perhaps that is not such a bad thing.

Is this sort of model possible? Does such a repository already exist? Are there important flaws that I have failed to consider?

Note that I am interested also in implications of long-standing security problems. For instance, I think it is interesting to consider the implication of adopting my above proposed scheme, given the possibility that my root CA and my bank have already been compromised and have been so for several years.