What's is the Up!?!
Hello. I love it when someone says;
"[...] how to debug it?"
I don't actually have an installation of the Windows 8 setup, and I haven't played with a Windows 8 install either, so I'm going to be assuming that there are some fundamental similarities to its predecessors. I'll try to call out assumptions as I make them.
Calling the shots
My guess, based on your description of the problem, is that the system is failing to record the proper entries in the registry so that when you run "open" after previously doing "open with..." you will see the application that you selected last time.
I will walk you through your issue and prove or disprove my initial guess. If you don't need all of my verbosity below you can skip/scan down to the Recon section where we start trying to uncover the issue.
Before we get into the more involved steps described below you should first investigate the Default Programs interface found in the Control Panel. You can get a similar effect going through the "open with..." menu, but you already said that was not working for you. **Make sure you are setting the checkbox that indicates to remember your setting as the default application for whichever application you have selected.
Under the Default Programs interface in the Control Panel there might be options for you to "reset defaults" or revert to a previous version. You can consider these various options to see if they solve your problem. Do these first, because they will be the easiest fixes. If they are not working then you should continue on to the options below.
These are the assumptions that I foresee. If any of them are way off base, then please let me know and if I have to I will take the time to set-up a Windows 8 install.
I think the biggest assumption that I'm making is that the OS still uses the registry and for places where I give examples I'm assuming that the registry directory layout/structure in Windows 8 is similar to its predecessors too.
Personally, I often prefer a command prompt so I will probably describe steps in the context of a terminal session. I'm assuming that Windows 8 still has cmd.exe; I'd be really surprised if it didn't.
If there is a registry, then there is probably a tool to view/edit the registry. I'll be assuming that the name of the GUI and commandline tools are the same as they were in the predecessors.
I will assume that the ways to invoke shell's with elevated privileges are similar enough to the OS predecessors that you will know what to do. If I am off-base here just ask me for some elaboration and I'll do what I can.
I would be extremely disappointed if these were not available in Windows 8. I can't believe that they would not be available because they are so fundamental to the developer/QA cycles that Microsoft and third-party developers go through.
WinDbg.exe, DDKs, and SDKs
I assume that you don't already have these installed. That's fine. We probably can solve the issue without requiring those heavy-duty tools. Go through the Recon section first and if you need (or just want) to go deeper, then will talk about these tools.
Always try to accomplish your task by the simplest means possible first. Starting with the Windows Debugger would be overkill for a starting point, and it would be a little more difficult to spot the piece/sequence of events that are relevant.
If you haven't already downloaded and installed The SysInternals Suite, then you should do that now. At the very least you will need Process Explorer and Process Monitor for the first attempts at spotting what's going wrong. Both of these tools are part of the suite, which you can download as a *.zip file, but if you have a poor network connection then you might choose to download the two tools individually to get it done faster.
Unzip the suite/tools to an appropriate place on your system (largely your personal preference). If you are using the default Windows unzip utility, then you will want to remove the Zone file before unzipping the archive. The zone file is how the OS knows "this file was downloaded from the Internet [...] run at your own risk [...] uncheck box to [...]". I'm sure you've seen a message similar to that in a pop-up box before. There is usually a little checkbox in the bottom left of the pop-up that will remember your decision going forward if you toggle it once.
Another way to see that checkbox is to right-click -> Properties on the file you downloaded. The checkbox should be at the bottom of the General tab. Toggling that checkbox once will remove the zone file from the binary (as a side note, the zone file is stored in an alternate data stream of the binary in question - but that's another discussion. For now, just use the pop-up windows or file properties interfaces to manipulate zone files.
Make sure you remove the zone file from the *.zip archive before unpacking it with the default Windows zip tool. If you don't remove it from the *.zip file then the ZIP utility will mark each unpacked file with a zone file and then it will be a pain getting the pop-up window for ever binary you run in the suite.
There are two tools that we are going to start with:
* Process Explorer (procexp.exe)
* Process Monitor (procmon.exe)
You can reference the corresponding *.chm file if you want to learn more about them. We are going to start with Process Explorer and if we can't quickly spot the issue then we will move to Process Monitor (which is a little more involved to run), and if we still can't resolve the issue then we will talk about pulling out the big guns. The fact is, though, that I often find what I need to know in the recon phase and don't need to start-up the debugger.
Background and Understanding
The registry is where the system stores the mappings that relate file extensions to the applications that know how to handle them. For example, you can use the command-line tools ftype and assoc to view/manipulate the mappings for extensions to applications.
To see the filetype associated with the extension ".txt" execute the following command from a terminal.
$> assoc .txt
This indicates that the .txt file extension is assigned to the txtfile class of filetypes. Then to see the application that is associated with the txtfile class, execute the command
$> ftype txtfile
These tools are just reading the registry settings for the system. You can view them yourself with a tool like reg.exe which you would execute from a terminal or most people would prefer the GUI registry editor regedit.exe. At that location you will see some of the extensions, filetype classes, and application associations you would simply look up under the HKEY_CLASSES_ROOT hive. Directly under that hive you should find a key for both ".txt", as well as, "txtfile" and of course much, much more.
Here are some more examples of using the assoc and ftype utilities.
Make sure you are feeding extensions to assoc. I think you already know this, but to be clear file extensions are the "dot followed by 3 or 4 characters" that terminate a file name. For example, application.exe, document.txt, or presentaion.ppt. The assoc utility expects you to provide the "." and the 3/4 characters as its parameter.
$> assoc .exe
$> assoc .txt
Separate and distinct from the file extension there is the file type. Multiple file extensions can be grouped into a single file type. Consider how ".html" and ".htm" map to the same filetype; that is, "htmlfile". The example below tries to demonstrate this point further.
$> assoc .html
$> assoc .htm
$> ftype htmlfile
htmlfile="C:\Program Files (x86)\Internet Explorer\iexplorer.exe" -nohome
Further, notice that the registry contains the following keys under the HKEY_CLASSES_ROOT hive.
If there is a convienent way for you to upload/attach/share a txt file then run the following commands, and I will look at the output they create.
$> cd \somewhere\easy\for\you\to\find\
$> assoc > extension_associations.txt
$> ftype > filetype_handlers.txt
You should now be able to open the folders that you wrote those files into and share them. The following commands should open the files too (if you can seem to find the files you created).
$> notepad \somewhere\easy\for\you\to\find\extension_associations.txt
$> notepad \somewhere\easy\for\you\to\find\filetype_handlers.txt
If you can't share whole *.txt files with me, then at least run these two commands and just put their results into the comments.
$> assoc .txt
$> ftype textfile
textfile="C:\Program Files (x86)\Notepad++\notepad++.exe" "%1"
If you don't see anything there then you should start by setting it to something. Make sure you have elevated privileges (that is, administrator privileges) when you make those changes, otherwise they probably won't work how you expect.
TODO if needed
TODO if needed
Installing Windows Debugging Environments