I've been experimenting a bit with the puppies bath in 4k video off 4ksamples.com as well as a few others. I got a combination that worked with my test video. I note reasonably fast storage is essential for quality playback. Considering some trial and error is needed, do backup the current settings. These settings work on a 3770, 16gb of ram and a gforce 980TI which I know is 4k capable
I've changed quite a few advanced settings to get it to work. In theory I want DXVA to be enabled but that causes VLC to crash.
Under simple mode in preferences go to Tools -> Preferences and Input / Codecs and in the drop-down box for Skip H.264 in-loop deblocking filter change it to All. This is recommended by VLC
Switch to show all under preferences and try these changes
I turned off all filters under Video Filters. This was the last thing I did, and seemed to help the most so I'd start here
Under "Video Codecs" - > FFMPEG I set my threads to 3. You might want to experiment with higher or lower numbers.
Make sure everything under output modules/DirectX is unticked.
These changes seem to have worked.
For reference, once I stripped out the commented lines, my vlcrc looks like
I have a gist with my full vlcrc for 4k here