What are the technological reasons whereby GPUs are cheaper than CPUs?
GPUs are inherently massively parallel - you take one logic block, repeat it thousands of times and it can now process more pixels. CPUs need to deal with less parallelizable instructions and that's harder. So on a FLOPs basis a GPU can be cheaper to produce than a CPU.
I guess that one of the reasons is that GPU can allow to be modern. In x86 we deal with a CISC architecture which then have been modified into RISC/CISC hybrid. Modern PC supports 3 modes of operation at the same time (16, 32 and 64 bits) and support such instructions like decimal addition (who needs it anyway). On the other hand GPU deals with relatively HL domain-specific 'language' (OpenGL/DirectX) - supported on CPU side (drivers). It means each generation can (and often do like the transition from r100/r200 to r300/r400 or r300/r400 to r600/r700 [I don't know where r500 is]) have much of its internals removed.
Now imagine that we can simply replace X86-64 (which is in fact upgraded 25-years old architecture) by something like Itanium without rewriting everything. Or something like ARM. Something which could be updated to modern requirements (no need for complex slow instructions as today no one needs them as no one is writing in assembly - possibly more conditional instructions to avoid jumping like on ARM etc.).
Also - the GPU deals with much less complex problems - or rather much more parallelised. It does not need to do such things as supporting 25-years old instructions that someone might have used them. It can just have them emulated on CPU/by few GPU instructions. It does not have to predict branches inputted in a weird way just because someone thought it would save space or something.
As a side note - last time I build a computer there were about 2:2:1:1 share between the processor, graphic card, motherboard, and rest.
what do you mean cheaper? There are expensive GPUs, more expensive than certain CPU. how do you compare them?
They are usually created with 90 nm process, while CPUs are produced with 45 nm transistors nowadays. In fact the gpu unit then requires more energy, but the one time end-user price might be lower - older the technology the cheaper it is (in relatively close time scope 3-4 years).
Specialism vs generalism
Which way around is up to you... each works in different ways for different processing types, different data, different access patterns etc
Compare CPU and GPU architectures (picture from Nvidia).
A lot of ALUs (arithmetic-logic units) give you a lot of computational power (1000s of threads). Minimal control results in severe costs of branching and other operations that are not "mathematical" (and lowers the price tag in store). Cache is very small and managed mostly by the programmer (hardware does not have to predict what to store), so it's cheap. What is more, GPU has its own "RAM", therefore it can work more efficiently on for example 2GB of data (but still pays for it) than a processor which is limited by communication with external units.