Is there a Firefox Task Manager (like in Google Chrome) that would allow me to see how much RAM my plugins/extensions/tabs are using?
From Firefox 6+ there is
For example, this is what Firefox
That's indeed a tab. Extensions are not shown as far as I know; plugins are available as a process:
I don't know a way to do this in Firefox 5. In Firefox 7, about:memory got a lot of upgrades, including at least part of what you want.
Edit: Another answer has a nice screenshot, but I believe it will show the memory usage for each tab.