There was a period in the Software development process when we had 7-8 hours long nightly builds. This was an year ago. I started making changes and today we have builds happening in 2-2.5 hours.
Intel Xeon E5-2650 2.0 GHz CPU
Intel 500GB SSD
64 GB 1666MHz RAM
The Software we compile:
Code Size 11-12 GB - Native C++, C#, JAVA
One VS 2010 Solution with 250 projects - Release and Debug
One VS 2010 solution with 80 projects - Release and Debug
Four VS 2010 Solutions with around 20 projects each - Release and Debug
A large JAVA codebase JAVADOC
The process we follow:
1) Get code from Clearcase - 15-20 minutes
2) Compile the source - 1 hour
3) Generate Installshield Media- 8 installers - 15-20 minutes
4) Archive all the media and source code -10 minutes
5) Some more activities
The problem is that the customer seems to be really depressed by the 2-2.5 hour build times. I have iteratively brought down the build times from 7-8 hours to 2-2.5 hours, parallelizing as much as possible. I see I can reduce another 10-15 minutes if i work really hard for around another 20-25 hours. But anything beyond that seems unrealistic.
The bottlenecks are none. That is, no hardware element seems to be under any pressure. We have more than 1,50,000 files involved in the build. This seems to be holding the SSD back. The Reads/Writes are slower than the specified value.
I feel any further reduction in build times can be achieved only if we restructure the code base. But the customer feels that I am incompetent and does not bother about my suggestions. He wants to hire some consultant to figure out the issue.
Please let me know the right way to solve this issue.