I am currently using a Zotac GeForce GTX 970 Amp! Extreme Core Edition, but ever since I did a Windows Reset, my 970's fans are always on starting from the log in screen and continue until I shut down my computer. I am not doing intensive tasks all the time but the fans still remain on.
I use a MSI Gtx 970. Using the afterburner software the lowest I can set my fan speed is 25%.
As Matthew has mentioned the graphics card will still generate heat, and will keep the fans ticking over to keep it at an acceptable level. Of course your milage may vary between manufacturer & Software!
Most GPUs have fans that are always on, as no matter how little work they do, it will still produce more heat then can be passively dissipated by the heat sink. The software in your drivers will automatically manage the speed of the fan, but it should never turn it off entirely. How can you turn it off? You can use third party software such as MSI afterburner, which can override the fan speed set by the drivers, and sometimes turn it off.