IMHO, the problem with Homebrew is it tries to use /usr/local in a way it was never meant to be used: owned by a user other than root. While I understand the homebrew developers take care not to munge with anything else in /usr/local, nothing else that installs to /usr/local will do the same for Homebrew. This can cause problems, and has for me... usually permissions issues that result from installing other software that sets permissions on /usr/local/ based on "how they should be". You'll never see another software package expecting /usr/local/ to be owned by a single user other than root, so why does Homebrew? Why not just use
Also, a little known fact about why Fink & MacPorts compile their own libraries:
There are several reasons why MacPorts uses its own libraries. It makes ports more consistent across different versions of Mac OS X. For example, if we can rely on openssl 1.0.0 from MacPorts, we don't have to test every port that needs ssl for every available openssl installation. Apple's software tends to break from time to time (e.g. openssl refuses to build with an old zlib, but for awhile Apple shipped the old headers of the vulnerable zlib version). Even if Apple's versions aren't broken, they're rarely up-to-date. Apple has a habit of not updating the libraries in Mac OS X until absolutely necessitated by a security vulnerability.
The drawbacks of this policy are minimal: Wasting a few megabytes for e.g. a Python installation is next to nothing if you have a multi-gigabyte hard disk, and the time required to build the additional ports decreases as computers get faster.
So while Homebrew is faster to install what you want, it can have other bad side effects from using pre-built Apple system libraries.
Again, I hate to dig against Homebrew. I like the software & I think it's great for some things, but it has it's downfalls as is currently.