I frequently need to convert videos from one format to another.
So I'm wondering whether the conversion uses my graphics card's processor, so that I can accelerate the process by replacing my low-end GPU with a more powerful one?
It depends. Video encoders may use the graphics card's processing capabilities if they're programmed to do so. Others might only use the CPU without additional features. Some might use special capabilities of the CPU, even.
One technology that just relies on CPU features is Intel® Quick Sync Video. The website lists several tools that have Quick Sync support.
For GPUs, there's NVIDIA's CUDA technology as well as AMD's Stream and Intel's MediaSDK. A recent article that compares these technologies and lists common tools can be found here: H.264 encoding - CPU vs GPU: Nvidia CUDA, AMD Stream, Intel MediaSDK and x264 – I'd recommend you to read it, it's highly informative.
Whether your encoder supports any of the listed methods, you'll have to look that up. Generally, x264, the most popular h.264 encoder these days, does not have any special GPU or CPU capabilities. This is also the encoder FFmpeg or VLC use, for example.