The majority of resource usage by Firefox isn't used by the core Firefox functions but by the sites opened in Firefox. So whether the resource usage you list is reasonable or not depends entirely on you "very specific task" and without knowing what it is it is impossible to judge.
That said there are a few things that can cause Firefox to use excessive resources because of a damaged profile or a buggy add-on. You can try launching firefox with the -P switch to launch the profile manger and create a new profile. Then try doing your "very specific task" in the clean profile and see if the resource usage of Firefox is lower.
Another thing to look at is newer versions of Firefox. Mozilla has a MemShrink project going on to reduce the resource usage of Firefox. The first improvements are available Firefox 7 Beta, and further improvements are available in Firefox 8 Aurora and Firefox 9 Nightlies. They will be buggy, though, but they probably could do your "very specific task" with lower resource usage.