Do this by adding “-private” to the shortcut in the taskbar. Right-click on the IE logo in the taskbar. Then, right-click on “Internet Explorer” and click “Properties”. In the properties dialog, add a space and then -private to the end of the Target. OK.
If IE is not currently running, right-clicking the IE logo in the task bar and selecting “Open new tab” opens a new, non-InPrivate, tab.