You are not right, nor are you wrong.
Your computers needs 'something' to generate the image which is send to your monitor(s). That something is the GPU. Which means that you already have at least one GPU in your computer. It is either already on a dedicated card, or it is build into the chipset, or it is part of the CPU/APU.
If it is already on a dedicated card you can replace that with a faster model.
If it is in the chipset or in the APU/CPU then you can add more GPUs by adding more cards. You then can then disable or not use the old GPU part and use the faster card.
As a result 3D games will run faster. Programs which use the Open Computing Language (OpenCL) will run their openCL code on the GPU instead of the CPU and will run faster. (Or they will use both GPU and CPU and run even faster).
However not everything will be faster. If something is not limited by the graphical card then improving that will not help.
(In non computer term, think of it as writing a letter and getting it mailed once per day. Getting a faster pen will not help if the post if still delivered once per day).
In praxis however:
- Modern 3D games can update their screen faster, or update fast enough with extra feaures (fog, anti aliassing etc) enabled.
- A few programs will leverage the power of the GPU and run faster. There are not many program out there which do that yet, but their number is steadily increasing. Flash is once of these.
- 2D stuff (office, skype, surfing the net) usually does not have an advantage.