My work blocks a bunch of sites on my laptop and I'm wondering if I access them through an Azure Machine if work can see what sites I'm visiting that way?
Ignoring the ethical implications:
An Azure Virtual Machine is going to have its own pipe to the internet, having nothing to do with your local network. That applies to dns lookups too: Azure VMs are not going to use your local network's dns servers (assuming your local network had dns servers).
That said: you need some way of connecting to your Azure VM, whether that's RDP, an vpn, or some other desktop sharing tool. And there's no telling what type of monitoring your company is doing, regarding software you're running locally.
One more thing: If you're using an Azure subscription owned by / billed to the company you work for, they can easily track resource usage (virtual machines, storage, bandwidth, etc.). So if you spin up a VM in their subscription, there's a chance they'll be able to track general usage patterns, and may potentially be able to infer non-work-related resource usage.