It depends. X.509v3 certificates usually come with an "Extended Key Usage" extension field, which contains a list of permitted usages (EKUs).
Regular web server certificates contain the "TLS Server Authentication" usage (sometimes shown as "TLS Web Server", but it really is not Web-specific at all).
To act as a client, you need a certificate with "TLS Client Authentication" (again often shown as "TLS Web Client", despite having nothing Web-specific in it).
It is quite common for regular "web server" SSL certificates to contain both usages – for example, I'm looking at certificates issued by Let's Encrypt and DigiCert, and all of them contain both usages, therefore can be used for client/mutual authentication.
However, it's possible that in some other CAs, especially private organization CAs, most certificates have one usage or the other, but rarely both.
For example, OpenVPN used to strictly require "only TLS Client" (rather than "at least TLS Client" as other programs do), therefore its
easy-rsa script issues server-only and client-only certificates but not the mixed kind.
So you should examine your certificates and make sure they contain the required EKU. Practically any certificate tool will do the job – for example, the Certificate properties dialog in Windows will show this under "V3 Extensions", as will web browsers,
Also worth remembering if you run your own internal CA: If you use an intermediate CA certificate, pay attention to it as well. Intermediates aren't required to have an EKU extension, but if they do have it, then all certificates issued by that intermediate are constrained by it. (If you obtain the certs commercially, shouldn't need to worry about it.)
Additionally, the certificate will have a basic "Key Usage" field which contains some very general restrictions: e.g. "Digital Signature" means the certificate is allowed to sign data (and therefore allows EDH/EECDH handshake in TLS); "Key Agreement" appears to be for static-DH/ECDH; "Key Encipherment" allows static-RSA handshakes.
The official X.509v3 documentation is very confusing in what usages are needed when... Commercial CAs will usually get this field right. But for private CAs it's worth double-checking.