I am running a computer with 4 Nvidia Titan X for training deep neural networks. I have problems with my power consumption. The consumption of Titan X is not supposed to be above 250W but as you can see on the plot, it can be above 350W. When I use nvidia-smi to monitor the power consumption, the parameter enforced power limit is set to 250W as it should be.
At first I thought it was not a problem except I cannot run all 4 GPUs simultaneously on my 1200W PSU Corsair Axi. However when I tried a different neural network more computationally expensive on a single GPU I crash the computer. In fact I have burnt the PSU and I have sent it back. I have now bought a bigger PSU (1600W) but I do not want to have the same problem again.
I don't remember how the PSU was plugged but maybe the GPU I used was plugged with something similar to this since Titan x needs 2 pci-e connector and my PSU had only 8 ( 4*2 for GPUs and 1 for CPU it is not enough). If yes the GPU was plugged on a single pci-e output instead of two.
Is it possible that the GPU consumed more than possible on this entry (40A, 480W) and burned the whole PSU ? Can I really enforce the 250W power limit ?