First bits of the second edition

[phil-opp]

This PR adds the first two posts for the second edition, “A Freestanding Rust Binary” and “A Minimal Rust Kernel”. The largest changes in comparison to the first edition are:

• Instead of GRUB, we use our own bootloader (written in Rust) and our bootimage tool. This removes the dependencies on GRUB and nasm. Note that both tools are still experimental and might contain bugs.
• Support for Windows and Mac: Without GRUB, there's nothing preventing us from building on Windows or Mac OS anymore! We added additional CI jobs at travis and appveyor to ensure that the project always builds on all three platforms. (At the moment, users still need to install LLD, but with the LLVM 6 update of rustc we should have a builtin LLD soon.)
• No assembly in the main posts. Instead, we're creating a no_std executable and relying on our custom bootloader to perform the initial page mapping and the switch to long mode.
• No linker script: Our bootloader loads the kernel in a clean address space, so we can just use the default executable layout and don't need a linker script. (This also means that users that want a higher half kernel just need to update the mapping in their linker script. However, I'm not sure if it's a good idea to add this to the blog. Maybe later, when we begin to run user programs.)

These changes only land in “beta mode” with this PR, which means that they're not linked from the front page yet. We will do that in a follow up PR.

[phil-opp]

bors r+

[bors]

Build succeeded

• continuous-integration/appveyor/branch
• continuous-integration/travis-ci/push

[le-jzr]

(This also means that users that want a higher half kernel just need to update the mapping in their linker script. However, I'm not sure if it's a good idea to add this to the blog. Maybe later, when we begin to run user programs.)

Just a random thought: Maybe in face of the Meltdown vulnerability, don't do the higher half kernel mapping at all, and instead just keep kernel in a separate address space from the start.

[IsaacWoods]

Maybe in face of the Meltdown vulnerability, don't do the higher half kernel mapping at all, and instead just keep kernel in a separate address space from the start.

While I think it's a bit preemptive to be worrying about Meltdown, mitigation of both Meltdown and Spectre are still possible with a higher-half kernel (KAISER/KPTI in Linux and probably similar in the Windows kernel), and there are benefits to the higher-half kernel: it's probably easier to understand how different processes run in different memory spaces if kernel addresses are clearly different, especially if you're new to kernel development, and it allows the linker to make some nice optimisations with relocations.

You also need at least some part of the kernel mapped for when interrupts occur in ring 3. Even if you keep the kernel in a whole different address space, you're gonna need some trampolines somewhere (preferably not interfering with process address spaces) to reload CR3 and jump into the kernel. Keeping the whole kernel, along with the interrupt handlers (even if they're the only things mapped during execution of a usermode process), in the higher-half makes more sense imo.

[phil-opp]

How about mapping the kernel (partially) to the higher half as soon as we implement user mode? This way user mode and kernel mode addresses are easily distinguishable. Discussing Meltdown then would be very interesting too.

it allows the linker to make some nice optimisations with relocations.

Can you expand on that? Sounds interesting

[IsaacWoods]

Sure! The idea is that if you choose to place the kernel within 2GB of either the top or the bottom of the virtual memory space, then the relocation offsets can be represented by 32 bits. For example, I place my kernel at 0xffffffff80000000 (-2GB) so that kernel symbols can use R_X86_64_32S relocations instead of R_X86_64_64 (twice the size). This should also allow userspace applications (mapped in the lower 2GB, since the kernel isn't in the way there), to use R_X86_64_32 and R_X86_64_32S relocations.

However, I have looked at the actual ELFs rustc is producing (before linking to my bootstrap), and it looks like that, by-and-large, it isn't realising it can do this and is actually just resorting to using R_X86_64_64 in most cases :( In hindsight it's relatively insignificant, and it'd probably be better to just do whatever would be easiest to understand for people who don't care what the linker's doing.

[le-jzr]

@IsaacWoods I think you need to specify code-model: kernel in your target specification to tell rustc it's allowed to do that.

[le-jzr]

I couldn't find the llvm documentation for it right now, but it should be equivalent to -mcmodel=kernel GCC option (see https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/x86-Options.html).

[IsaacWoods]

@le-jzr I already had code-model: kernel, which wasn't playing ball, but playing around, I also added "pre-link-args": ["-mcmodel=kernel"], which appears to let something in the pipeline use R_X86_64_PC32 instead of R_X86_64_64 (which was unexpected, since I didn't think ld would've been consulted but there you go). Anyways, I'll have another play if/when I change to using lld to link, but I'm anxious not to hijack the issue with linker jargon ;)