The wicked cool open-source application Notepad++, a free XML editing tool, is an example of an app commonly deployed in enterprises that not only has a shortcut to start the application, but also uses the Windows file explorer File Type Associations to allow the end user to launch Notepad++ from any file on the system by right-clicking.
This app has been difficult for many to package into MSIX, due to an issue with the Context Menu missing. Last year we worked out two solutions for this issue. The better solution involved using a new type of application extension that is only available on Windows 11. For Windows 10 deployments, we recommended converting the Context Menu into a Shell Menu as the solution.
But now in version 8.5.7 of Notepad++, the developer did some bizarre things that are causing us issues again.
Background
But first some background.
When it comes to File Type Associations, Windows has two types of file types. The standard ones are for files that have names that end in a file type extension, such as foo.xml. The special ones are for things like folders, drive, and a special case ‘*’ that means for any type of file. Notepad++ uses the special ‘*’ file type for its registration.
As I wrote in The Application Book, Windows has two choices when it comes to implement the registration of the FTA, the Shell Integration method and the Shell Extension method.
- The Shell Integration method is the simplest for the developer. In the registry, under the file type the developer adds a key called “shell”, and under there the developer can add verbs and the command line to be used. These verbs will then appear in the pop-up menu whenever the user right clicks on a file of that file type.
- The Shell Extension method is more dynamic, and requires the developer two write a special in-process COM based dll called a Context Menu Shell Extension. This gets registered under a ShellEx key under the file type, and here the Context Menu handler is defined by its COM CLSID GUID. The COM CLSID is also registered, which allows the dll to be found and loaded by the file explorer when needed. In essence, the file explorer asks the context menu handler “what verbs to you want me to display and what command to run if chosen”. This allows the handler to consider the context of the request and supply different lists for different situations.
There are some additional options, like ProgIDs, that I am skipping over in the above description, but it is good enough definition for our situation.
What Notepad++ did before
Notepad++ has, for quite some time, implemented a Shell Extension Context Menu. The handler, however, always returns the same list, which by default is a single item to ‘Edit in Notepad++’. The app does provide an xml file to customize that, but most folks use this default. If you aren’t customizing that, a Shell Integration for a verb would have sufficed, been more efficient, and would not have caused us the packaging headaches.
The problem originally was that the special file type ‘*’ was not supported in the MSIX schema set. This was later addressed in a schema update which solved the problem, but only for systems running Windows 11. So, we were fixing the package for Windows 11 by updating the AppXManifest to get the full feature, but for Windows 10 deployments our suggested method was to get rid of the shell extension and create your own Shell Integration verb (note: this also works for Windows 11).
What Notepad++ 8.5.7 did
It turns out that not only were people repackaging Notepad++ for deployment in MSIX having an issue, but that the native application installation on Windows 11 was also having an issue with the shell extension working as expected.
So in version 8.5.7 the primary author re-implemented the registration in a strange way, one that I’ve never seen before. And then he added another bizarre twist to get it working on Windows 11!
- First, the new installer uses a little-known and undocumented (as far as I can find) third method to perform the Shell Extension registration. This method uses the Shell key instead of the ShellEx key, but then defines the COM CLSID GUID in a special registry value named ExplorerCommandHandler. Of course the COM CLSID must still be added to the registry too. Not only had I never noticed this used before, but the Microsoft MSIX Packaging Tool didn’t know what this was either and ignored it. By the way, so does the Microsoft App-V Sequencer (which is how this issue was first reported to me).
- Now comes the second bizarre thing done. The native exe installation of Notepad++ 8.5.7 drops down a file called NppShell.msix next to the NppShell.dll file which is the Context Menu Handler. Then it tries to silently install this MSIX file.
The MSIX file dropped down is what is known as a “sparse” msix file, which means that it references things that are not in the package.
| Note: I’ve never heard of a sparse msix file used the way this one is being used. It normally is used in very large games that want to minimize the initial download size and then download new levels as the game goes on. |
In fact, the msix package dropped on to the system here pretty much consists of the AppXManifest file and no binary components. It is manifested to only install on Windows 11, so if you natively install the app on Windows 10 this MSIX package will silently fail to install and the traditional registration will be used, which works. On Windows 11, that doesn’t work, so once the MSIX package is installed, you get the context menu.
So, what is in this special 68kb msix file? Just enough to register the ContextMenu – exactly as we have been doing in our repackaging.
But how does this affect repackaging the entire app into MSIX?
Well, by default, our package will contain the extra msix package file, but it is ignored. It doesn’t matter if it was installed while in a packaging monitoring mode as those areas of the system are ignored. And since the registration of the Shell Extension is now under the Shell key, neither the Microsoft MSIX tool, nor my own tooling, knows to look for it there to add the extension into our full package MSIX file.
What can we do about this?
I am adding code to TMEditX to detect and fix this shortcoming (it will be in the release that follows the current 3.5 release). For anyone that can’t wait for that or are using other tools (like the App-V Sequencer), I recommend just dropping back to the old Shell Integration verb method of solving the problem.
