There are a ton of ways to get this up, but I'll quickly go over some of the alternatives I am more familiar with.
I'll be referring to Azure Active Directory as AAD throughout this.
What is AAD
Let's quickly talk about what AAD is and is not. AAD is not Active Directory. About the only thing they have in common is the name. AAD is not LDAP, it does not support RADIUS, and it cannot act as an Active Directory endpoint for anything that requires an active directory endpoint.
AAD is a REST API based authentication service. It has support for OAuth 2, and I believe also SAML. Anything you configure will need to use these endpoints for authentication.
In a bit of irony, many of Azure's native services do not support AAD as an authentication endpoint.
Azure Application Gateway
This is essentially the Azure Application Proxy for services hosted on Azure. It may be the easiest solution for you to set up. It's been a while since the last time I used it, but it should support AAD authentication. There is plenty of Azure provided documentation on how to set up and configure an Application Gateway.
Azure Application Gateway Documentation
3rd-Party Authentication Proxy
If you're using Apache or nginx, or similar in front of your deployment, you can gate access to the website with a 3rd party authentication proxy like Skipper. You'll need to deploy a skipper service, and configure it to connect to an AAD OAuth endpoint for your customer.
Azure Application Proxy
This is not intended for use with Azure hosted applications, as you mention in your question.
3rd-Party VPN Gateway
I'm not aware of any VPN servers that support OAuth or SAML as a backend mechanism. I don't think OpenVPN supports it, and it's the most commonly used VPN server. It is possible an option may exist though.
Azure VPN Gateway
If you want to Authenticate with ADD, This isn't really an option, since the Azure VPN Gateway only allows authentication by either TLS or RADIUS, neither of which are compatible with AAD. As a result, this is not a viable option.