Категория: In English @en

Windows, AHCI/RAID and INACCESSIBLE_BOOT_DEVICE

When you switch SATA from AHCI -> RAID or back in BIOS, Windows may fail to boot with INACCESSIBLE_BOOT_DEVICE. There are several pieces of advice on the internet that work only sometimes. You will now understand why. This is an ultimate guide to fixing every driver-related INACCESSIBLE_BOOT_DEVICE out there.

You can jump straight to instructions but life is messy and blind copying often doesn’t work.

TLDR Why this happens

Windows needs to see the disk from which it boots. Bootloader initially cheats and jumpstarts Windows, but then Windows needs to quickly load drivers that can talk to the disk properly.

INACCESSIBLE_BOOT_DEVICE happens if after loading all boot-time drivers Windows doesn’t see the boot disk runtime device around.

How Windows loads drivers

Every PNP device has an ID, something like VEN_8086&DEV_06D6&SUBSYS_86941043&REV_00. Each driver has an INF file which lists the devices it can service. Windows keeps a library of known drivers ("Driver Store"). When it discovers a new device it looks through that library for a driver that can handle it.

If Windows finds a compatible driver it installs it and saves the device permanently in the registry. Next time it sees the same device it will load that installed driver directly. The devices are saved in HKEY_LOCAL_MACHINE\System\CurrentControlSet\Enum.

A driver is a program running in the background. When Windows loads a driver for the device it explains which device has to be serviced (by giving it that registry key). The driver then creates a runtime device object. Only one copy of the driver will be running even if it services multiple devices.

So there are three different things:
– Hardware device attached to the PC, known to Windows by its PNP ID
– Registry entry for the device’s PNP ID
– Runtime device object

Hardware device that has no driver won’t get a registry entry and a runtime device object. You can still see such devices in the Device manager with an exclamation mark.

If you detach an installed hardware device its registry entry will remain. But a driver won’t be loaded and a runtime device object won’t be created. You can see such devices if you check "Show hidden devices" in Device manager.

You can check the attached PNP device IDs with a tool like devcon.exe. (Note that Windows won’t see PNP IDs of devices attached further to the devices it has no drivers for).

Boot devices

Boot devices are those that are critical for Windows to boot. Disk controller is a prime example.

Windows cannot install devices and drivers at boot time. The Driver Store and the registry itself are stored on disk which is inacessible. All the boot devices and drivers have to already be in the registry (which is read once at the very beginning, via cheats).

This is why you can replace your graphics card and Windows may find new drivers automatically but you cannot simply switch from AHCI to RAID and back. Your "AHCI controller" will disappear and a new "RAID controller" appear; Windows cannot install drivers for this new device at boot time.
You have to boot to install drivers but you have to have drivers to boot.

Thankfully many drivers will discover "their" devices even without registry entries. For these devices it’s enough to have the driver itself registered and started at boot; it’ll create runtime objects, you don’t need to have CurrentControlSet\Enum device keys.

This is why there’s advice to preinstall RAID controller drivers. If you preinstall the drivers and configure them to auto-start despite not having any devices to service (another piece of advice), the drivers just might see their devices and let you boot – where you can install the devices properly.

How Safe Mode is different

There’s advice to enable RAID and boot to Safe Mode first. Why?

In Safe Mode, Windows automatically boots all boot-time drivers even if there are no registered devices that need that. So this advice still needs you to preinstall RAID controller drivers, and it’s the same as configuring those drivers to auto-start, only safer and easier.

oem*.inf and installed third-party drivers

Once Windows installs a driver, it creates a key for it in the registry: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services. This key controls when the driver is started and lists devices associated with it.

Drivers can be forcefully installed by installing their inf files. There’s a number of ways to do that, one is to right-click the file and choose "Install". Another is pnputil /add-driver.
INF files that are installed this way are copied into Windows with names like oem0.inf, oem1.inf etc. You can list all third-party drivers with pnputil /enum-drivers or dism /Online /Get-Drivers or uninstall them with either of these utilities.

Simple case: Preinstall drivers

The standard advice just may work in many cases:

  1. Install latest drivers for your RAID controller, either via setup or by force-installing their INF files if the installer seems too smart for its own good.
  2. If you have older versions of the same driver installed you may choose to delete them to be on the safe side.
    To do this, reboot after installing the new version to hopefully let all devices switch to the newer versions. Use dism /Get-Drivers or pnputil to enumerate all oem*.inf drivers. Find the ones to be deleted and use pnputil to delete them.
  3. Reboot in RAID mode and go straight to Safe Mode. Hopefully your boot-time RAID controller driver will load, see the RAID controller and let Windows boot from it.
  4. Install RAID controller drivers again from Safe mode to register the device properly.

Why can this process fail?

  1. Your RAID controller driver does not auto-discover supported devices.
  2. Some INF files are written in a way that if they don’t see the device at the moment of installation they won’t install everything that’s needed to run it.
    A popular example is iaStorAC.inf. When preinstalled on a system without RAID controller attached it won’t install iaStorAfs filter driver. When you later reboot in RAID mode it sees a RAID controller but fails to initialize because iaStorAfs is not installed.

In both cases it’s not enough to preinstall a driver; you also need a registry key for a device.

Messy case: Preinstall a boot time device

Now that you know everything you may imagine a number of solutions such as manually installing iaStorAfs. But here’s a more resilient version.

  1. Prepare a flash drive with unpacked drivers for your RAID controller and a number of utilities, including devcon.exe. All utilities must be of exactly the OS bitness – 32bit emulation doesn’t work in recovery mode.
  2. Boot into recovery mode and open command prompt.
  3. Use devcon to list all devices in the system. Find the RAID controller and take note of its PNP ID. Open your driver’s inf file and check that it services this PNP ID. If not, you need newer drivers.
  4. Open your driver’s inf file and check that if it’s install sections have OS identifiers attached (".NTamd64-10.0.1") then your OS is supported. See INF format docs, Manufacturer section.
  5. Use Windows\System32\drvload.exe to load your RAID controller driver.
  6. If everything goes right, this recovery system now has that driver loaded!
    Check the device list again to verify that the RAID controller now has a more personalized name. Check that the device list now has your hard drive too or use diskpart list disk to verify that the disk is visible.
    If something went wrong, stop at this point. Figure out why the driver isn’t loading. It’s pointless to try and install a driver that doesn’t load.
  7. Start regedit. This system now has the driver and the device installed. Go to the following keys and export them as reg files to your flash or hard drive.
    • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\PCI\VEN_8086&DEV_06D6&SUBSYS_86941043&REV_00
      This is the device node. Find the one matching your PNP ID.
    • HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e97b-e325-11ce-bfc1-08002be10318}This is the class registration node. This one is for SCSI controllers; if you need a different one, find it by looking inside them

    For the good measure look up all places in the registry where oem0.inf is mentioned (that’s how Windows renamed the inf you’ve just installed). Export all those.

  8. Now disable the RAID mode and reboot into normal OS.

You now have registry export files with registration information for the RAID controller device. Your have to adjust these files and import them, making this copy of Windows think it saw this device.

What do you need to adjust?

  1. Preinstall the driver as per simple case. Take note of what "oem*.inf" name it got.
  2. You’ll only need to import the Enum\PCI\… and Control\Class\.. files. The rest are just in case.
  3. In both of those, replace oem0.inf with the oem*.inf for your imported driver.
  4. Go to the registry and find those keys. See which 0000, 0001 etc keys are already there; change your reg files so that they take the yet unused key numbers.
  5. Delete any parts of those files that duplicate the already existing keys. For instance, all key=value pairs directly inside Control\Class\{4d36e97b-e325-11ce-bfc1-08002be10318} are not needed. Leave only keys directly related to your device, the less you touch things outside of that, the better.
  6. Very carefully take ownership of both of the keys where you’re going to import these, and give your account full permission. Make sure SYSTEM, Administrators and other accounts that had permissions over those keys before get to keep them.
  7. Import the reg files. If there are any problems, stop here and solve them until the import works. Verify that what you have in the registry now matches the reg files.
  8. Reboot the PC normally. Now go to the Device Manager and enable "Hidden devices". You should see that RAID controller that you’ve just imported as a reg file. If you don’t, solve this before going further.

Okay, now you have an offline device for your RAID controller! From here on try these actions, starting from the simplest:

  1. Reboot into RAID mode. Things may just work.
  2. Reboot into RAID Safe mode. If successful, install the drivers.
  3. If neither RAID normal nor RAID safe mode boots, but you still see the RAID controller in "Hidden devices" in AHCI mode, try installing RAID drivers from AHCI mode, preferably via their setup.
  4. As a last resort, right-click the "hidden device" in Device manager and "Update drivers" this way. Then try booting in RAID Safe mode again.

Additional help

Use pnputil and devcon to manage drivers and devices.

Use dism /Online /Get-Drivers. Dism is available even in the recovery console and while it can’t uninstall drivers from /Online systems (use pnputil), in recovery console it CAN uninstall from the main OS.

If you’re stuck in a recovery loop (without Windows even attempting to boot into normal mode), do bcdedit /set {current} recoveryenabled No. After that you’ll be getting a bluescreen instead of going into recovery, and what would you think! You can STILL go into recovery from that bluescreen, but you also get a problem description and you can also restart in Safe Mode and other things.

To access the main OS registry from recovery, open regedit, select HKEY_USERS and File > Load hive. Load the Windows\System32\drivers\config\SYSTEM for HKEY_LOCAL_MACHINE from your main OS. It’ll load under HKEY_USERS so importing .reg files won’t work but you can edit things by hand.

For the purposes of this guide drive letters don’t matter. In earlier Windows versions drive letters could become mixed and Windows would fail to find C:\Windows because it’s now D:\Windows. This can still happen later in the boot process but the kernel and the bootloader are now mostly drive letter agnostic, they know how to tell one physical disk-thing from another. If it says INACCESSIBLE_BOOT_DEVICE then it’s really inaccessible.

Logrotate creates empty compressed log files

Case study: After some changes I’ve noticed logrotate flushes the main log (as it should) but all gzipped archived copies it creates are empty (20b). Logrotate destroys logs instead of preserving them.
You had one job, logrotate!

Turns out my logrotate file looked like this:

/var/log/httpd/*_log /custompath/log/*_log {
..
create ...
}

And the latter had been symlinked to the former.

It made sense when I wrote that. Cover all the bases. But the way logrotate works, it first scans all paths for files that need rotating and then applies rotation to them all.

So if I have /custompath/log/error_log reflected as /var/log/httpd/error_log, logrotate notices two files in need of rotation. It then compresses the first one, replaces it with empty log (create), then compresses the second one (now empty) and replaces the first .gz with an empty archive.

Nice job, logrotate. Make sure the collected filenames resolve to unique files? Nope. Maybe at least don’t overwrite already existing gzips? Nope.

Anki — How to review regularly

Here’s what helps me review regularly:

  • I study when there are no better things to do. When I commute it’s either reading or drilling. It’s easier to resist temptation when there’s less temptation.
  • I have a pact with myself that I will keep review counts low by whatever means necessary (even moving learned cards back to new), but in exchange I have to review all cards every day.

This way I have less temptation + less frustration ("I’m not going to finish these 1000 cards anyway") + more motivation ("Gotta review these 300 cards faster and have more time for reading").

It’s all about the balance of incentives. It sucks more to betray my routine, miss my usual satisfaction when I finish the reviews and pile up work for tomorrow than it is to drill an always manageable number of cards then feel good and have fun.

As for game-like elements, there’s "% learned" (overall progress), "review %" (today’s performance). Increasing both feels good because better scores == less cards to repeat tomorrow == less reviews in the coming days and in the long run == more time for reading.

Fix Logitech MK520 Mouse Middle Button

I’ve changed two sets of Logitech MK520 Wireless Keyboard+Mouse sets and while otherwise they’re great, the middle mouse button starts to fail after 6-8 months of use. Pretty annoying.

Turns out there’s an easy way to fix this:

  1. Remove the batteries, unscrew and remove the cover.
  2. Find the middle-button push thingie on the board. The thingie is held together by 4 black dots at the corners. Take a small screwdriver and carefully destroy these.
  3. The thingie will come apart. Remove its square cover, the button layer and the conductive layer.
  4. Clean the conductive layer carefully. If it’s black in the middle that’s why it doesn’t work. Scrub off that black until the middle is of uniform metallic color. Also clean the base under it.
  5. Assemble the thingie back as it were. Insert the battery and test the middle button.
  6. Very carefully use a toothpick to put just a tiny bit of glue at each of the four corners of the thingie where the remains of the black dots are. Have some more toothpicks and some napkins ready to remove excessive glue quickly. You don’t want to glue the whole button together!
  7. Assemble the mouse back.

Takes around 30-60 minutes and probably less if you know what you’re doing.

Posting to console from GUI app

In Windows, apps can be marked as either GUI or console. GUI apps have windows with controls, console apps run in a black box, receive text input and produce text output. Inputs and outputs can be redirected, allowing to chain applications and write results to file:

dir | find ".dll" > list_of_dlls.txt

For apps which do not output streams of data, console is usually a good way to print status messages. There’s a whole class of apps that could benefit from functioning both ways: if run as is, they display GUI, but with command-line switches they do the job silently and exit.

myapp.exe — runs graphical interface

myapp.exe /source dir1,dir2 /output dir3 — does the job silently and exits

Could these apps detect when they’re running from console and output status messages? And otherwise behave entirely like a console app?

This is an interesting and complicated topic and here are my findings.

(Read the article)

Background

Console and GUI subsystems

Each EXE file has a flag that marks it either as GUI or CONSOLE subsystem application (there are other subsystems but they are not important for this article). Apps marked as console or GUI behave differently.

Console apps:

  • When run as is, create a new console window for itself.
  • When run from another console app (such as cmd.exe), inherit its console window and do not return until they’re done.

GUI apps:

  • When run as is, are invisible (until they create some GUI windows manually).
  • When run from a console app (such as cmd.exe), return the control immediately and work in the background.

Nevertheless once started, they are identical. They can call the same functions, they have the same rights, and so far as Windows is concerned, each can be reconfigured to be another.

There’s no principal difference between Console and GUI app, once they’re running.

Consoles

Each process can have up to one attached console. Consoles are windows where you type the input and where the output is printed. They are created automatically by the system when a Console-marked application starts and has no inherited console.

Each console provides three standard handles, for input, output and error output. They are passed to the application as described below. When you write to an output/error handle, that data is displayed in the console. When you read from an input handle, the console lets the user type some characters and returns that.

Attaching a console means you can use those handles. Without a console attached even if you somehow get those handles, they will not work.

Attaching goes deeper than that: once the console window is closed, all attached processes will be terminated and there’s no way to stop that, though you can receive a notification and be allowed to finalize.

When a console-marked app starts another console-marked app, the latter inherits the console. No new window is created, and it receives the same input, output and error handles.

GUI-marked app never inherit the console, no matter how they are started. But they still inherit the handles, which will be broken (see below).

By default in modern systems console window is implemented by conhost.exe. But there are apps which replace it when installed.

Attaching to a console

Any application, including a GUI-marked one, can attach to a console with AttachConsole(), by passing the process ID of a process with a console (copying a console from it).

If two applications are attached to the same console, both can read and write to it. Unless they coordinate, they’ll produce a mess.

Any application can create a new console for itself with AllocConsole(). This will spawn a new console window, identical to the one Windows creates automatically for console-marked apps.

In fact, for console-marked apps Windows just calls it automatically, if no console is inherited.

Detaching from a console

Any application, including a console-marked one, can detach from a console with FreeConsole(). In that case standard handles stop working.

Yes, that means a console app can stop being a console app, if it wishes so.

Console windows are destroyed automatically once no one is attached to it. If you’re the only one attached (because you created the console with AllocConsole() or because it was created for your console-marked process automatically), once you FreeConsole() the window disappears and you continue running.

If you inherited the console from a parent process or attached to it manually, it’ll stay for that other process.

Standard handles

To input and output data in console apps Windows provides three read-write handles which can be obtained from GetStdHandle(): STD_INPUT_HANDLE, STD_OUTPUT_HANDLE and STD_ERROR_HANDLE.

These are standard Windows handles, same as returned by CreateFile().

For a console app, all of these point to an attached console. If the input is redirected, they will point to files or to pipes to another app. For a GUI app started from Explorer, all will be zero.

By default these handles are inherited from parent to child. If you start a process from console, it will have its standard handles set to that console.

There are two ways to override handles for another app:

  • Or you can SetStdHandle() for your own process temporarily and start a child process like that (since standard handles are inherited).

To redirect input/output, the usual approach is to create an inheritable pipe, pass one end to a child process as an input and write to another. That’s more or less what cmd.exe does when it chains applications. But you can pass any handles which support reading/writing, including network handles and file handles.

How console-marked apps are started

All processes under Windows are ultimately started with a variation of CreateProcess(). This function returns immediately once the process is started.

How is it, then, that when you run a console-marked app from a command line (cmd.exe) it does not return until it is finished?

The answer is that cmd.exe itself does that. It starts the process and if it notices that the process is a console-marked one, it waits for it to return, and sets ERRORLEVEL to its exit code.

Why does it do that? Because child command-line processes inherit the console and start writing to it, and if cmd.exe continued to use the console at the same time, the result would be a mess. So it waits until the console is free again.

In other words it’s not built into the OS. If your command-line app starts another command-line app, by default it’ll still return immediately, and that app will write to your console in parallel with you.

What if I don’t want to share my console?

You can pass a flag CREATE_NEW_CONSOLE or DETACHED_PROCESS to attach a new console to a console-marked app, or to start it without any (like a GUI app!)

Can I share my console with a GUI app?

By default, no. There’s no way to start a GUI-marked app with inherited console. There’s no way to attach a console to another application. Only the application itself can call AttachConsole to attach one.

Can I start a GUI app with an attached console?

Yes, but only with a new one (pass CREATE_NEW_CONSOLE).

Reading and writing to console from GUI app

Since standard handles are inherited, GUI apps run from console receive handles to that console. But those handles will not work, because the console itself is not attached.

To make them work, it’s enough to do AttachConsole(ATTACH_PARENT_PROCESS).

There’s a host of cases that needs to be covered:

  • Standard handles (from GetStdHandle()) may be 0, which indicates that either you were started from Explorer, or that whoever started you wanted you to throw away your output. In this case, you do not have to AttachConsole().
  • Standard handles may be redirected to a file, or a pipe, or elsewhere. In which case you still do not have to AttachConsole(). You should just write where pointed. How to check what those handles are? GetFileType() == FILE_TYPE_CHAR, but it wouldn’t work — see below.
  • Standard handles may look like a console handles, but come from your own console, attached to you automatically because someone started you with CREATE_NEW_CONSOLE. It would not be nice if you detached from it and attached to a parent one, so you must not call AttachConsole in this case. How to tell if you have your own console? GetConsoleWindow().
  • Standard handles may come from a console, but an entirely different one because someone screwed up. In which case there’s nothing you can do and the behavior is undefined, but it would probably be better to just write wherever pointed, and not AttachConsole (since parent is unrelated).

Summarizing, what do we do?

If we have a console (GetConsoleWindow() <> 0), do nothing! We have explicitly been told exactly where to write.

If all the handles are zero, and you cared to check, do nothing! Either we don’t have a console-enabled parent (nowhere to attach), or been explicitly told to be silent.

Otherwise we could check GetFileType() to see if any of our handles is FILE_TYPE_CHAR. But there are two troubles with that. First, FILE_TYPE_CHAR also means stuff like LPT printers and not only consoles. And second, while the console is not attached, console handles will return FILE_TYPE_UNKNOWN.

So there’s no clear way to tell. As an optimization, you could check whether all of the handles are FILE_TYPE_PIPE and FILE_TYPE_DISK and in this case skip the AttachConsole(), but there’s plenty of cases where you don’t need AttachConsole() and can’t tell it.

So just do it. AttachConsole() returns FALSE if there’s no console underneath, and has no other consequences; we’ve already determined we have no other console to lose, so even if we attach to a parent one for no reason, it won’t hurt. Or will it?

What are the consequences of attaching to a parent console?

There are two:

  • Your process will be terminated if a console closes (see the section on Consoles)
  • Parts of your code which write to console may write to the parent console unexpectedly for the parent process.

Remember that for GUI-marked apps, cmd.exe returns immediately and the user may be typing another command, as you steal the console from them and output your messages. Therefore attaching to a console must be accompanied by a parent application waiting until you finish (how to achieve that will be described in the sections below).

Since there’s no way to guarantee that you’re being run in that fashion, it may be sensible to not attach to a console by default. Only if you’re passed a flag (say, "/console") indicating that the user (or an intermediate app) wants to run you in a console mode, only then should you attach a parent console.

How to run a console-attaching program

By default, cmd.exe does not wait for your process to return. It returns the control to the user immediately. If you attach the console and start writing to it, you’ll be interfering with user’s actions.

Even if you do not attach the console, your error code will not be delivered to the cmd.exe, and will not be put into ERRORLEVEL, as expected by scripts.

There are two approaches:

  • start "" /WAIT will start the app and wait for it to return. It will also properly handle the error code. This is entirely sufficient if your app does AttachConsole() and it works (see below about problems with runtimes).
  • You can write your own console-marked wrapper to start the app, wait for it to finish, then query and return it’s error code. This is preferable if you want a neat console shortcut to your GUI app, and when more complicated handling is needed (see below).

Consoles, standard handles and runtimes

AttachConsole() is fully enough to make ReadFile() and WriteFile() on console handles work just like they would in a console app.

But in a real world, most people write to console through some kind of language abstraction (printf() in C++, writeln() in Pascal and so on).

These abstractions are wrappers around platform-dependent functions, and on Windows resolve to querying GetStdHandle(STD_OUTPUT_HANDLE) and doing WriteFile() to it. But they introduce problems.

There’s two kinds of problems I’ve encountered:

  • Buffering, and
  • Handle caching

Buffering

Most runtimes do not write to files immediately as you call the equivalent of printf("text"). Badly written code often outputs data char by char, and it would’ve been too many system calls. Instead, they collect the data locally and push it to the operating system only occasionally.

C/C++ for example has three modes of buffering available for CRT FILEs: no buffering (output immediately), line buffering (output on each CR/LF) and block buffering (output when something like 2048/4096 bytes are collected).

CRT tries to be smart about it and chooses different buffering schemes for different files. For stdout it chooses line buffering, as it assumes we want to see each line as soon as it’s printed, not ten minutes later when enough data has been collected.

But when it detects the output is redirected (i.e. standard handles point to something other than FILE_TYPE_CHAR), it switches stdout/stderr to block buffering.

For instance, if you run a C++ app with it’s output redirected to a pipe, and if you read from that pipe and put that on the screen, you will not see the output line by line as the program executes. You will instead receive it in blocks of several lines, which is not how we expect console apps to function.

Similar buffering exists in most runtimes, although sometimes it’s smaller and less noticeable.

In C Runtime it can be disabled by calling setvbuf(stdout, NULL, _IOLBF, BUFSIZ) to enable line mode, or setbuf(stdout, NULL) to disable buffering at all. It needs to be done manually even if you output to console to which you attach with AttachConsole, because the buffering is decided before entering main(), and at that time console is not yet attached, so GetFileType() returns FILETYPE_UNKNOWN.

In particular, in Visual C Runtime line buffering is not implemented at all, so setbuf(stdout, NULL) should be used instead.

Other runtimes have their own ways of disabling buffering. If there’s none, sometimes you’ll have to bear with it.

Handle caching

Some runtimes, notoriously Visual C Runtime, inspect standard handles once and then remember the results forever. If your application starts with one set of standard handles, and then you use SetStdHandle to redirect your own output, that redirect will not be applied to stdout because it has already decided where to write.

This is especially bad with ATTACH_PARENT_CONSOLE method. By the time you get to do AttachConsole, C Runtime has already inspected the standard handles, called GetFileType() on them, seen that they’re FILE_UNKNOWN_TYPE (because their console wasn’t attached) and marked to never write anything to stdout/stderr.

Is there anything you can do? No.

There’s no way to make C Runtime re-decide. You can open a new FILE for a standard handle with _open_osfhandle, but you cannot assign it to stdout for it is a constant. You can freopen stdout, but only to a file name, not to a handle already opened. Finally, some sources teach to freopen("CON"), but this will always return a handle to attached console (no matter if it’s not set as your output), what if your standard handle is actually redirected to a file? Or a pipe?

So in short, there’s no reliable, non-hackish way (which doesn’t involve patching internal CRT structures).

In this case ATTACH_PARENT_CONSOLE method is not going to work, and you’re restricted to a console-marked wrapper with pipes (see in Solutions).

Compiled solutions

Summarizing all the knowledge we’ve accumulated, we’ll list approaches to making a GUI application behave in a console-like way.

ATTACH_PARENT_CONSOLE

When our app detects its parent process has a console, and it doesn’t have a console of its own, and perhaps it’s started with a "/console" flag, it calls AttachConsole(ATTACH_PARENT_CONSOLE) and ignores the result.

If used with a "/console" flag, it then can adjust buffers and other settings as expected from a console app.

if ((GetConsoleWindowHandle() == NULL) && (haveArg("/console"))) {
 AttachConsole(ATTACH_PARENT_CONSOLE);
 setbuf(stdout, NULL);
 setbuf(stderr, NULL);

}

It can then be run with start "" /WAIT app.exe params.

Console wrapper

A small console application can be written which starts the target application, passes it’s entire command line unchanged, waits for it to return and returns the error code.

Usually there’s not even a need to change the GetCommandLine() before passing it as lpCommandLine to the CreateProcess(), as the first argument in it needs not to be equal to the actual executable name. It’s enough to pass a correct new executable into the lpApplicationName.

Such a wrapper may include any additional flags you wish, for instance it may pass "/console" flag to your app to make it adjust its behavior (see ATTACH_PARENT_CONSOLE).

You do not need to override any of the standard handles, as they’ll be automatically inherited.

Console wrapper with pipe redirection

When ATTACH_PARENT_CONSOLE method doesn’t work, you’ll have to write a wrapper with pipe redirection. Do the same as in the previous step, only create three pipes in addition.

These pipes must be created with "Inheritable" flag set.

SECURITY_ATTRIBUTES sa;
memset(&sa, 0, sizeof(sa));
sa.nLength = sizeof(sa);
sa.lpSecurityDescriptor = NULL;
sa.bInheritHandle = TRUE;
CreatePipe(out, in, &sa, 0));

For each pipe, pass one side as a standard handle to the child process and close it after the CreateProcess() call, keep the other side and read or write to it (make sure to keep the side you need, one is for reading, the other for writing).

Start a process with inherit handles flag set to true, then wait for it to finish, meanwhile translating input to it and reading and posting output. You’ll probably need a separate thread for input (readln() and write to input pipe in cycle), and you can handle both output and error output pipes in the main thread, by looking with PeekNamedPipe() and reading if there’s something to read.

This way when the target app starts, it already has valid input/output redirected handles. You may need cooperation of the target app to configure buffers and such like in ATTACH_PARENT_CONSOLE method.

Controlling the lifetime of a child process

Since your wrapper is a console-marked application, it is attached to a parent console and will be terminated if you press Ctrl-C, or if a parent console is closed.

But your child process is a GUI app. If it does not do ATTACH_PARENT_CONSOLE, it will remain in the background even after you’re terminated.

To work around that, set your own SetConsoleCtrlHandler(). This function will receive notifications when Ctrl-C is pressed or the console you’re attached to is about to go down.

You will not be able to prevent that, but you’ll be able to terminate the child process with you.

tee

There’s a quick and dirty way to test redirection before writing a wrapper. Use tee, an utility from unix-utils, available from cygwin. It’s purpose is to redirect the output both to file and to the screen; if you omit the file, you’re just getting the redirection wrapper:

MyGUIapp.exe | tee

Console app + GUI wrapper

Another way of having a dual console/GUI app is to mark it as console and FreeConsole immediately after start if you’re running in a GUI mode. But that still produces flickering console window (ugly).

But why, you can make a thin wrapper. It’s only role is to start the console app with DETACHED_PROCESS flag. This flag instructs Windows to not create a console window. Your app can then detect that it has no attached console (with GetConsoleWindowHandle) and run in a GUI mode.

This is the simplest approach of all, and with the least amount of code.

FAQ / TLDR

Yes, you can have a chameleon console/GUI app, but reliably only through a secondary helper app.

Can I write to console from my GUI app, if it’s run from one?

Yes, so long as you AttachConsole(ATTACH_PARENT_CONSOLE) or AllocConsole() and use WinAPI functions. But you don’t want to do this.

Why don’t I want to do that?

Because the parent application does not expect this. You should only ATTACH_PARENT_CONSOLE if you’re run in a certain way.

How should I be run so that I can attach parent console normally?

By start "" /WAIT appname.exe or through a special wrapper app.

How do I know that I’m run like that?

You don’t. The usual way is to pass you a flag, something like "/console". But you may decide to be reckless and just always attach the console.

Will my default printfs/writelns work everywhere in a program?

Sometimes. Some runtimes permit this, others don’t. C Runtime doesn’t, Delphi does. If your runtime doesn’t, WriteFile will work, but printfs won’t.

Can’t I do something so that my normal printfs work?

You can make a console wrapper around your app which runs it with redirected handles. It’s more work, but it will be almost transparent to your GUI app.

Can my GUI app tell if it has a console?

Yes, by checking GetConsoleWindowHandle(). But most of the time, your GUI app will NOT have a console. It does not inherit the console by default, even if started from one. You have to connect to one manually.

Okay, can my GUI app tell if there’s a parent console to connect to?

It probably can, but it’s easier to just go ahead and try to connect. If it fails, it fails. But make sure you don’t have your own console beforehand.

Can I make some existing GUI app (pre-compiled) write to console? (I know it calls printfs, even though it has no console)

Yes you can, by writing a wrapper. But there may be inconveniences (buffered output). One simple way to test it is to use tee from unix-utils.

Further reading

Mercurial local per-repo .hgignore

What do you do if you need to ignore some local files, but would prefer not to commit that rule to everyone in .hgignore?
(E.g. you’ve created a folder in the repo for your own needs)

There’s a global .hgignore which you can configure from %PROFILE%\hgrc, but using it to list all exceptions from everywhere is ugly.

Turns out you can add per-repository .hgignore overrides this way too! Edit repo’s hgrc:
[ui]
ignore = .hg/.hgignore-local

It wouldn’t be committed as it’s inside .hg, and it would be parsed in addition to repo’s normal .hgignore.

Solved: Delphi XE3 64-bit debugger fails to run

Symptoms:

Delphi XE3 sometimes fails to run 64-bit applications under a debugger. Code would compile, but the part where Delphi switches to debug layout never happens, Delphi just pops a message saying "Cannot run the debugger".

32-bit debugging continues to work normally, and so does "Run without debugging".

The funny part is that this happens irregularly. Sometimes the first attempt would succeed, and then the debugger would run all the time in all instances of Delphi. But if it fails the first time then it would always fail even if you restart Delphi.

I also noticed that the earlier I launch Delphi + debugger, the higher is the chance it would run (and then continue working). It seemed like there was something I was doing or the computer was doing sometime after boot that broke the debugger if I hadn’t launched it yet.

Solution:

Stop the "Internet connection sharing" service and restart Delphi.

What might have contributed:

– Uninstalling older versions of Delphi on the same PC.
– Disabling Windows Firewall
– Disabling Windows Defender

(Diagnostics process)

Diagnostics process:

Looking at the successful and failed debugger launches with Process Monitor, in both cases Delphi runs a remote debugger. But on the successful run it’s dbkw64_17_0.exe (64 bit) while failed runs spawn rmtdbg170.exe (32 bit). Both are Delphi debuggers, but I suspected that the second one is only supposed to be used for 32 bit debugging.

Further investigation showed that in both cases dbkw64_17_0.exe launches initially, but in the second case it terminates shortly afterwards. Delphi then tries to connect to it through TCP, unable to do so, and restarts it automatically. But the code that does the restart probably wasn’t updated to 64 bit and launches 32-bit rmtdbg170.exe instead.

Anyway, the problem lies in the initial instance of dbkw64_17_0.exe terminating. Comparing Process Monitor logs, both successful and failed runs load the libraries and then work with winsock. Stack in the final calls indicates ws2_32.dll‘s socket() is running – the debugger is probably trying to open it’s command socket for listening – after which failed instance abruptly terminates (Thread Exit, Process Exit). I figured socket() probably returns with an error.

Using rohitab’s Api Monitor I tried to find out the error code, but this didn’t work out. Api Monitor successfully traced all the calls until roughly WSAStartup(), but no further – the last bunch of calls just before the termination always got lost, perhaps the injected driver wasn’t being able to send it back to the main app in time before the application terminated.

Then I opened dbkw64_17_0.exe for debugging in Visual Studio. I set a breakpoint to {,,ws2_32.dll}socket, caught the execution there and studied what happens step by step. Turns out, socket() was successful. It was followed by setsockopt call, also successful (to know which functions we were stepping into, I used VS’s standard ability to load Windows DLL symbols from Microsoft servers). Then dbkw64_17_0.exe called bind() which failed.

My initial guess was that someone else occupied the port it needed. Checking bind() parameters at MSDN, I looked into RDX, RCX, R8, R9 registers which host parameters in x64 calls, namely the memory referenced by RCX, which kept the requested family and port number. It turned out to be 0xC0F3 but it was unoccupied.

I then traced the call to bind() and from the internal call to WSPBind() got the error code: 0x1D27, that is 10013 (WSAEACCES: Permission denied. An attempt was made to access a socket in a way forbidden by its access permissions).

This code has no single specific reason for it. From the internet it looks like it appears when some driver or network-related service misbehaves. I tried stopping network related services one by one, until finally bind() succeeded. The infringing service was "Internet connection sharing (ICS)". As long as I stop this service, the debugger launches normally, and so long as ICS is running, the debugger would not start.

The reason why sometimes the debugger would run and then run always, is probably that ICS hadn’t yet been started or did not yet harm the network stack at the time. If the debugger run at that point, it would bind the socket, and for whatever reason binding at that port would then continue working later. But if the debugger was initially launched after the harm has been done, it wouldn’t be able to bind to the port neither once nor at all.

Sale rankings are not indicative of popularity

Sales might not be indicative of the anime’s popularity, as understood by those who watch anime regularly.

E.g. if asked what were the best animes that you watched, what most otaku would say.

First of all, there’s a large number of people which only stay in the anime for 1-2 years (and even more who only watch less than 10 series). They will vote for Shingeki no Kyoujin because that’s what they watched and it was good. But if you watched anime for the last 10 years, you wouldn’t necessarily say Kyoujin was the best that you’ve seen (even though it’s OK).

So even other reasons aside, sales mostly show “what was hip at the time it came out”, not “what was etched into the hearts of long-time fans the most”.

Now, if you look at the list, there’s only Gundam SEED in the top 10 which can’t be considered relatively modern (let’s say modern era starts with 2006 Haruhi / Geass). There’s only SEED, FMP and Initial D in the top 30!

Even if you count “Modern” from 2009 / Bake, there’s only SEED, 00, Macross F, Geass, Haruhi, FMP, Lucky Star and Initial D (10 slots total) which haven’t been made in the last 5 years.

Modern animes are disproportionately present in the top sale charts.

It’s not that anime has gotten just that much better. Delivery prices, channels and anime popularity vary. Perhaps anime DVDs cost more or weren’t as widely available, or it wasn’t as common at the time to buy them, or economy was worse, or anime in general had less fanbase.

Recent animes also have a better potential fanbase. Older otaku may still vote for them, while younger otaku will not, in general, watch lots of older animes, and even if they do, there’s usually no way to buy one (increasing the sales).

HOWTO: Assign checkable TAction to TSpeedButton

To make TSpeedButton work with TAction.Checked when it’s a singular option (either On or Off), make sure that at design-time:

SpeedButton.Action = Action
SpeedButton.GroupIndex = 0
SpeedButton.AllowAllUp = true
Action.GroupIndex = 0
Action.AutoCheck = true //only if you need AutoCheck

Then add this to FormCreate:

SpeedButton.GroupIndex := 17; //any non-used group index

SpeedButtons are linked to Actions through TSpeedButtonActionLink. It only updates their Down property if AllowAllUp is set and SpeedButton.GroupIndex property is NOT 0.

But when Action is linked, SpeedButton.GroupIndex gets rewritten by Action.GroupIndex on load.

And if Action.GroupIndex is 0 because it’s a singular option, then no matter what you put into SpeedButton.GroupIndex at design-time, it’s going to be rewritten with 0 at load, so TSpeedButtonActionLink does not update Down property.

The simplest solution is to set SpeedButton.GroupIndex to something in FormCreate.

How to maybe fix Gigabyte RF-G90B and certainly void your warranty

Disclaimer, read first. I have no qualifications to give any advice on this topic. I give no assurances or guarantees of any kind. This article is not meant to serve as an instruction, it’s just a description of what I did. Whatever you do, do it at your own risk, after properly studying and following safety measures.

I have a Gigabyte GZ-G90B1 Power Bank which seems to be more commonly known as RF-G90B. It’s also appears to be the same as Enerpad MG-9000 which is the brand name for Samyatech contents (board + cells imported from Japan, or so they say). So Gigabyte just sells that.

I haven’t used it for a while and when I tried to use it, it appeared dead. No matter if I tried to charge it, use it to charge mobile, press the button and/or hold it, the device wouldn’t react and lights would stay off.

As I’ve already lost my receipt I had no hope for replacement, so I unscrewed it (there’s a single screw). The box wouldn’t open at first because the cells are glued to both halves of it, but there’s no way around it but to pull. I detached the chip from the cells and then attached it back. This seems to have rebooted the chip and it started working.