I actually managed to do this after some trial and error:
The VPS has the public IPs directly attached to its interfaces:
ens4 10.10.10.10
ens4:1 11.11.11.11
ens4:2 12.12.12.12
ens4:3 13.13.13.13
ens4:4 14.14.14.14
It is better not to use the first IP so that managing the VPS will be easier.
Then, I created a ZeroTier network and joined the VPS and the OPNsense firewall.
Let's use the first subnet: 10.147.17.*
I set the IP:
10.147.17.2 for the VPS
10.147.17.1 for OPNsense
Then, on the VPS, you should have iptables-persistent installed and route all traffic from the public IPs to the IPs from the ZeroTier network. Like so:
sudo iptables -t nat -A PREROUTING -d 11.11.11.11 -j DNAT --to-destination 10.147.17.111
sudo iptables -t nat -A POSTROUTING -s 10.147.17.111 -j SNAT --to-source 11.11.11.11
sudo iptables -t nat -A PREROUTING -d 12.12.12.12 -j DNAT --to-destination 10.147.17.112
sudo iptables -t nat -A POSTROUTING -s 10.147.17.112 -j SNAT --to-source 12.12.12.12
sudo iptables -t nat -A PREROUTING -d 13.13.13.13 -j DNAT --to-destination 10.147.17.113
sudo iptables -t nat -A POSTROUTING -s 10.147.17.113 -j SNAT --to-source 13.13.13.13
sudo iptables -t nat -A PREROUTING -d 14.14.14.14 -j DNAT --to-destination 10.147.17.114
sudo iptables -t nat -A POSTROUTING -s 10.147.17.114 -j SNAT --to-source 14.14.14.14
Next, you should create a bridge in OPNsense between your LAN and your ZeroTier (ZT) interface (at this point, you should already have set up ZeroTier in OPNsense correctly with the IP mentioned above).
The last step is to create a firewall rule in your LAN network to allow traffic from your ZT network to anywhere and from anywhere to your ZT network.
Finally, set the IP in the machine you want the full public IP address for:
IP: 10.147.17.111
Subnet 255.255.255.0
Default GW: 10.147.17.2
Thats it. Now you have all the traffic from the VPS to your local machine, almost nothing in between.