Yes this is possible and should be fairly easy to do.
Assuming you have the Windows 8.1 installation media:
- Boot from your installation media
- Click Install Windows
- Choose Custom Install
- Click Drive Options when asked where you want to install Windows
- Delete the partitions containing your current Windows and Ubuntu operating systems
- Create new partition for your clean Windows installation
- Continue installation as usual
Depending on how the dual boot was set up on your computer, you may run into booting problems or you may still see Ubuntu as a boot option. If this happens, just boot from your Windows installation. Instead of Install, choose the following:
- Repair Your Computer
- Advanced Options
- Automatic Repair
This should clear up any boot problems you have.
If you need more detailed information or pictures, there are plenty of websites that can help guide you, just try doing a few web searches and I bet you will find what you're looking for.