Zebu Operating System

It’s been over a year since I posted here last, mostly because it had been a year since I’d done any work on Zebu. I’m writing now because I’ve spent the last few days making some progress on something.

I decided to re-write Zebu entirely from scratch. There are a number of reasons for this. First of all, I wanted the kernel to be entirely (or as much as possible) in C++ rather than C. Secondly, as I progressed before, I realized that I should have done a number of things slightly (or significantly) differently. I wanted to make it easier for me to maintain Zebu, by making the code easier to read and doing things in ways that make more sense (now that I have more of a full understanding of what writing an OS involves).

So, my current progress on this rewrite (which I started on Friday) is this:

I’ve written a bootloader that works on a hard drive (master boot record). It’s nothing special — it does no more than, say, the old DOS MBR. It looks for the active partition on the drive and executes the first sector of that partition.

I wrote a bootloader that resides in that first sector of a hard drive partition. Eventually this will become the first sector or two of a filesystem, but right now it’s just code. The code here loads the kernel (from a fixed position on the partition) into memory. It loads it to the 1 MB mark in memory, which is where main memory begins. (My previous implementation kept the kernel below the 640K mark, which was somewhat restrictive.) It switches directly from 16-bit real mode into 64-bit long mode, and then executes the kernel.

The kernel itself is, as I mentioned, almost entirely written in C++. It’s compiled entirely as 64-bit code (AMD64, EM64T, or whatever you want to call it). It’s highly object-oriented (that’s why I wanted it in C++), which I’ve found to be increase my development speed greatly. It currently has a physical memory manager, it can handle some interrupts, and it can create page tables. There’s still a lot of work to do to catch up to what I had before, but it doesn’t seem like it’ll take long to get there.

I discovered a very useful tool which has helped me quite a bit: I compiled Bochs (an emulator) with its graphical debugger enabled. This is proven to be extremely useful in debugging problems along the way. I wish I had done this the last time around.

I’ll continue to post updates here as I make more progress with it.

I’ve been working on Zebu a bunch again lately. Here are a few of the things I’ve done:

• Made screen updating faster by only updating portions of the screen which have changed.
• Sped up floppy disk reading by caching the most recently used 5 cylinders of the diskette.
• Created driver framework for video (graphics) drivers, and created an SVGA driver (using VESA BIOS Extensions (VBE) with the Virtual 8086 mode I added previously).
• Fixed Virtual 8086 mode so it will actually work properly when called more than once, and so I can get data out of it when it’s done.
• Added a RAM disk driver for creating a pretend disk in memory. This will be used for a couple things. First, if I ever boot of a CD or DVD, it can be used for storing data that would normally be stored on a hard drive while running (current settings and such). Second, I can use it for testing filesystem implementations without having to deal with a real disk, although I already have a real hard drive driver that works.

I also fixed various bugs here and there. I was working on trying to improve the floppy driver so it works on a real machine, but no luck with that so far. It works only in VMware (with either a real or a virtual floppy drive) at this point. Oh well.

Next steps? I haven’t decided for sure yet. I’m thinking maybe I should add file writing support to my FAT driver, which currently only supports reading existing files. We’ll see.

I did two big additions today. The first is Virtual 8086 mode, which allows me to run 16-bit programs… in my case though it’s just for performing BIOS interrupt calls. That will allow me to better utilize VESA for video modes and things.

The other thing I did was add support for one process to start another process, and added such a capability to the shell. Thus, I can now do this:

/ > cd floppy0

/floppy0 > ls
Listing files in /floppy0:
KERNEL.BIN
STAGE2.BIN
ZSHELL
TESTAPP2

/floppy0> run TESTAPP2
This is my first little test program!
Sleeping for 10 seconds, then exit.

/floppy0>

It takes a bit for the program to load (it’s coming off a floppy, and the code isn’t very efficient and is outputting all sorts of debug info to the serial port), but it does load, then wait 10 seconds, and exit, returning to the parent process. Note that both processes could run at the same time, but in this particular use case I choose to wait for the child process to exit.

More later!

I realized all this work trying to make a windowing system was holding me back from getting more productive work done on Zebu. In fact, I haven’t been doing much on it because any time I think about it, I would go “ugh, gotta implement more windowing stuff.”

So I decided to turn off most of the windowing system for now. Now I just display an almost-full-screen console window (120×50 characters) and I added necessary system calls to do input and output. I’m now working on a command-line shell. So far it allows directory navigation and listing, like this:

/ > ls
Listing files in /:
partition101
floppy0

/ > cd floppy0

/floppy0 > ls
Listing files in /floppy0:
ZSHELL
KERNEL.BIN
STAGE2.BIN

/floppy0 >

I guess one of my next steps is to get it so it can run start other programs… then I can write a program that would, for example, format the hard drive, plus whatever else I might like to run, of course.

I haven’t posted in a while for a couple reasons. The first is that I haven’t worked on Zebu as much lately as I was for a while. The second is that I’ve been working on something big, and didn’t want to write about it until I’d made a lot of progress on it.

That something, as you may have guessed from this post’s title, is a windowing system. I have implemented an extremely basic windowing system which features a very basic window manager, window classing, and message passing/handling.

In the process I also added a few nice utility graphics functions (previously SetPixel was the only option). These functions include DrawLine (using Bresenham’s algorithm), DrawCircle (similar), DrawFilledRectangle (trivial), and DrawBezier. The last one is the only function which requires the use of floating-point mathematics. The others do not, and are thus in general very fast.

I may post a screenshot in the next few days. I would like to make it work a little better and look a little better first. I need to get z-order implemented right, I’d like to port freetype, and some other things. So keep an eye out for that!

I fixed the crash that was occuring when executables where exiting, so everything seems to be working well now. A program can run, do stuff (print out to the screen, really, and not much else at this point), then exit, and the OS keeps running along without it (as you would expect). Now I need to start adding more system calls so programs can actually do more real things, like read/write files, get input from a user, etc.

I’ll let you know how that goes!

Although I have what seems to be a minor problem, and very little interface between programs and the kernel, I can actually run programs on Zebu now! I wrote a “Hello, world!” program and it is able to execute (it puts “Hello, world!” on the console) and then it exits, without a problem.

The minor issue I have is that there seems to be a problem (either in newlib or something I’m doing) when I attempt to use printf() to output. I can use sprintf() followed by write(), and it works just fine. Very curious. I’ll keep working on that issue.

Meanwhile, it’s great that I can now run programs. I now need to further develop my API (I currently have only one system call, to print out text) and start writing programs that use it! I will also, at some point, need to implement shared libraries, but that’s not super important at this stage, I don’t think.

I’m working torward getting Zebu to the point where executable files can be loaded and run. I did some work on the toolchain (binutils and gcc) so they now include a target “i586-pc-zebu” and these tools are then used to compile newlib, which provides me with libc, among other things. With these tools and a C library, I should be able to start compiling programs for Zebu.

Then I just need to get Zebu to be able to actually run these programs. I’ve started creating some of the necessary structures for loading an ELF executable, but have a little ways to go still.

After a lot of work, I’ve finally got virtual memory paging working. Previously, I did not have paging enabled, so every address I saw was identical to a physical memory address. With paging, an address I see in a process can be anywhere in physical memory, or not even in physical memory.

In my case, all the code still runs at the kernel level and most of the pages are still mapped to their physical addresses. The only exception to this is the SVGA video memory, which I’ve mapped to a particular location rather than its physical location (which is off in la-la land somewhere). I verified this worked because I can see stuff on the screen (previously I was getting page faults, which means something was trying to access a page that wasn’t present).

I have to say that was one of the hardest things to wrap my mind around so far, and consequently also one of the hardest to implement. But now that I’ve done that, I have a multitasking & paging kernel, which is more that 99.9% of the people on Earth can say!

My next step, I guess, is to load at executable (I’ll start with a flat binary) from the floppy and map some pages to it so it runs from the virtual address that it wants to run from. Then I can move to more advanced loading (ELF probably) and start working on an interface between user-mode and the kernel.

Have a great day!

I added support for serial ports today. But before that, I had to create the infrastructure for stream device drivers; so I did that. A stream device just takes a bunch of bytes and “streams” them somewhere or streams in a bunch of bytes. My previous drivers (floppy, hard drive) are called block drivers, because they deal with blocks of data that can be randomly accessed.

On top of the serial driver, I added some debug output support. The debug build of the kernel will write out debugging information to COM1 (the first serial port). Then in VMware (or any other emulator) I can map the serial port to a file on the hard drive, and anything I output from the kernel goes into that file. I use it as a text-based debug log so I can see what’s going on without having to rely on looking at the screen.