GPGPU linux setup with multiple GPU's and Multiple Monitors

Question

I'm trying to get a GPGPU workstation set up. I've got 4 Tesla C2075's and dual monitors. Getting the dual monitors working on 2 separate cards works fine but I can't get nvidia-settings to recognize all 4 GPU's, currently I can only get it to find a max of 3 at a time.

lspci | grep VGA

02:00.0 VGA compatible controller: nVidia Corporation Device 1096 (rev a1)
04:00.0 VGA compatible controller: nVidia Corporation Device 1096 (rev a1)
83:00.0 VGA compatible controller: nVidia Corporation Device 1096 (rev a1)
85:00.0 VGA compatible controller: nVidia Corporation Device 1096 (rev a1)

nvidia-settings -q gpus

 2 GPUs on RAMA:0
[0] RAMA:0[gpu:0] (Tesla C2075)
[1] RAMA:0[gpu:1] (Tesla C2075)

If I play around with Xorg.conf I can get 3 GPU's to show but then the second monitor stops displaying.

xorg.conf

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    Screen      1  "Screen1" 1920 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "1"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "DELL 2408WFP"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    Identifier     "Monitor1"
    VendorName     "Unknown"
    ModelName      "DELL 2408WFP"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 76.0
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Tesla C2075"
    BusID          "PCI:2:0:0"
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Tesla C2075"
    BusID          "PCI:133:0:0"
EndSection

Section "Device"
    Identifier     "Device2"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Tesla C2075"
    BusID          "PCI:131:0:0"
EndSection

Section "Device"
    Identifier     "Device3"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Tesla C2075"
    BusID          "PCI:4:0:0"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "TwinView" "0"
    Option         "metamodes" "1920x1080 +0+0"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Screen"
    Identifier     "Screen1"
    Device         "Device1"
    Monitor        "Monitor1"
    DefaultDepth    24
    Option         "TwinView" "0"
    Option         "metamodes" "nvidia-auto-select +0+0"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Section "Extensions"
    Option         "Composite" "Disable"
EndSection

Messing around with Xorg.conf can get up to 3 so that makes me think the issue is most likely with that, but I really don't know.