I am hosting WiFi from my ethernet-connected Windows machine. Is there any way for me to check how many devices are connected?
There are a number of free utilities to enumerate connected WiFi devices, such as Nirsoft's Wireless Network Watcher and alternatives. You can also use the router's web interface to view connections, or the (paid) professional version of Glasswire, which also displays network traffic.
The image below shows the interface for Wireless Network Watcher. Note that MAC Address, Device Name and User Text can be falsified, but if you know the parties using your WiFi, you can verify this information when each connects.
As with any software, check before using it that is does not contain malware with pestudio or VirusTotal.