I also have a Lenovo Yoga. I also had to deal with this. It's... Just stupid, I know. What follows is an explanation and a solution.
Why you're having this problem:
Windows 8.1 was designed with a heavy emphasis on touch screens ("traditionally" on tablets.) If a touch screen is present, Windows 8.1 assumes that a keyboard is not, so any time you poke your finger onto the screen within a text box, a keyboard pops in (because it thinks you don't have one available, so you'll obviously need a way to type in yonder text box, dig?) However, if you have a convertible machine like your Yoga, which has both a touch screen and a physical keyboard, Windows doesn't know that; It just thinks you have a touch screen ONLY, with no keyboard, so it keeps pushing that annoyingly redundant On-Screen Keyboard up in yo' grill even though you've already got a perfectly functioning "IRL" keyboard right in front of you. Dumb, ya? Dumb, dumb, dumb!
To make things more confusing, this behavior is only designed into METRO apps, and not Desktop apps (with SOME exceptions, like, say, Google Chrome, which have the Metro-style touch-text-box,-get-On-Screen-Keyboard-popup-automatically enabled as a "feature" you can't turn off)
Generally, with Desktop apps, you won't get a forced On-Screen Keyboard if you touch the screen, so this is why you may not see the strange behavior for hours at a time - you're using a desktop application.
All of this is why the keyboard popping in seems random to you; because the interface for Windows 8.1 was not designed consistently.
Does that make sense? No? Welcome to Windows 8.1.
As for the sudden disappearances, the On-Screen Keyboard will go away suddenly if you touch any of the physical keys. It's 8.1's way of saying, "Whoops! Sorry, I didn't see that physical keyboard there." Except it doesn't remember the whoops two seconds later.
I've called both Microsoft and Lenovo directly about this. Their take? The truth is, Microsoft did not design Windows 8.1 with convertibles (like your Yoga) in mind. If you have both a keyboard and a touch screen, you are going to experience this behavior. As of this point in time, there is no "fix."
...So, there's the "why".
As to the "how", first lets look at a proposed solution that came before this post:
Disabling the keyboard by stopping the service does work - it will nuke the On Screen Keyboard right outta the park - but the Yoga converts into a tablet as well as a laptop, and at that point you'll want that onscreen keyboard back - so the thought of digging into multiple screens and scrolling down to find that one tiny setting buried in a long scrolling list of other tiny settings each and every time you just want to flip your dam* computer into a tablet, is exasperating.
HERE IS A WORKAROUND THAT LETS YOU HAVE YOU CAKE AND EAT IT TOO:
We're going to make it so that the On-Screen Keyboard will NOT pop into view without your permission, but still have the On-Screen Keyboard available to you when you use tablet mode. Better yet, it's not that complicated to do... Ready?
Step #1. Go to the Charm search (Win + C)
Step #2. Type in "On-Screen Keyboard" An On-Screen Keyboard will appear, but it will look different than the one you're used to. This one has a Win7 style minimize button.
Step #3 Click that minimize button! The new-style On-Screen Keyboard will minimize to the Taskbar on your Desktop, out of your way.
Now, DON'T TOUCH IT! Just leave it open & minimized in the Taskbar.
As long as that new keyboard sits minimized in the Taskbar, when you touch or click on any text box in any context the OS will think that the On-Screen Keyboard is already enabled, except it won't shove it back in front of your face, it will stay neatly minimized. You won't see it.
Meanwhile, the hardware keyboard on your Yoga will continue to work like normal. Success!
When you want to use the computer in regular tablet mode again, just switch back your Desktop (Win + D) and close the On-Screen Keyboard like you would any regular Desktop app.
So now, when you flip your Yoga back to tablet mode, you'll still have regular access to that On-Screen Keyboard when you need it, in its intended context.
(You'll still have to reengage that alternate On-Screen Keyboard each time you come back to laptop mode from tablet mode, but it only takes a second to do so. ....I did say this was a workaround, not a fix.)