when I am connected via VPN, can the client monitor what I am doing on my machine
It depends on what they actually installed, and on how the VPN client is configured.
A normal VPN client doesn't transfer information about what you're doing in general – the server doesn't know that you're editing a file, and it doesn't know which file you're editing.
But it does handle your network traffic (obviously) and a lot of information can be determined from that traffic. For example, the VPN server's administrator can know if you're using TeamViewer (but not the actual data – that's encrypted), or if you're watching YouTube (but not the actual video URL – that's encrypted), or if you're sending an email (but not the actual email contents). In other words they'll see everything that your ISP would see, but usually nothing more.
So first, a VPN client can be configured to either route all traffic through the tunnel, or just specific traffic. (It is very common to use VPNs which connect only to school/company networks while leaving everything else untouched, aka "split tunnel" VPNs.)
If the client is honest (and not lazy), they can configure the VPN to only catch traffic to that client's servers, and nothing else. However, they can also configure the VPN client to capture all your traffic (or just traffic to their competitor's website, etc). Of course, enabling the VPN for all traffic is not malicious in itself at all, but it does allow your client to monitor you.
And in your case, "My IP address is different when I am connected to their VPN" is a strong indication that everything goes through the VPN.
But second, you're not 100% sure whether they installed just a VPN. They could have installed other software, e.g. something that specifically logs all your browser visits or tracks which program is currently active.
Can I prevent the client from monitoring my other tasks?
You allowed the client to install software on your computer – you've already lost.
It is possible to use a VPN for connecting to the client's network while still remaining safe; however, exactly how to do this depends on what VPN client you're required to use.
For a start, you would need to download and configure the VPN client yourself from provided information (instead of letting the client do it), and you would need to make sure the VPN client doesn't have any "remote provisioning" features which would allow it to locally install more components.
If in doubt, only install any client-provided software to a separate machine (maybe a VM) – never to your main computer.