What makes a good job?

Lately I've been reflecting on previous positions I've held and thought about the good and bad of each. It's probably a good idea to think about what job traits lead to satisfaction or dissatisfaction. There seem to be a few common properties which effect me most.

Teammates

I've pretty much always stayed in a team well after the expiration date because of great team members. I often find myself missing peers more than any other positive aspects of a job. Because of this I feel my peers are an extremely important factor in job satisfaction.

Good teammates turn into family friends and lifelong contacts. I recently had lunch with a guy I worked with over 3 years ago. It was the highlight of my week and I'd love to work with him again. If you want to know if you're in a cohesive team you only need to count the number of times team members have done things together away from work.

Technology

If I'm honest with myself I think I have to admit something: What I'm working on seems to be much more of a deciding factor when choosing the next job than contributing to actual happiness once there. I get caught up looking for the next shiny thing. However, there is some practicality in allowing technology to influence my next role. Usually, I try to choose a technology area that I believe will be most useful for my long term career. This is when I try to make sure I don't get stuck or pigeon holed into one area. I remember an engineer I worked with who was getting progressively bitter about always working on the same thing. He hated what he was being asked to work on but didn't seem to want to change positions. I suppose this is the most important reason to consider technology when choosing a job. You don't want to whither away as an expert of a dead-end technology.

Manager

My manager is probably the single most important influence on job satisfaction. Unfortunately, companies (and their managers) don't seem to understand this fact very often. I've worked in company cultures where the only way to grow was to enter management. This is horrible. It takes a company's most productive employees and mashes them into a completely different area of expertise -- one in which they most likely are not qualified. One great thing about Microsoft is the dual path they support. You can be a "people manager" or you can be an "individual contributor" (IC) and obtain the same compensation.

In my opinion, even worse than the employee who enters management because company culture doesn't permit IC growth is the employee who enters management for the prestige of it. These "prestige managers" seem to think being a boss is a more important role than being an IC. The reason this is the worst possible situation is these people don't understand a fundamental rule of leading: The leader works for the team.

Someone looking for prestige in a leadership role is under the impression the team works for the leader. Paradoxically, the only time the team truly will work for the leader is when the leader has won the respect and admiration of the team members. This does not happen through silly mandates, make-work meetings and tasks, managing appearances rather than results, or overbearing "guidance." It does not come from business buzzwords or MBA degrees. It comes from facilitating the team members. It comes from getting out of the way when you're a hindrance, proactively removing obstacles, and internalizing team member requirements. A bad leader will say, "I hope you're empowered to fix this." A good leader will not need to talk about empowerment because their ICs already are.

At one point I naively thought these misguided managers simply never had the benefit of an excellent manager mentor. I don't believe that's the case anymore. I've personally observed multiple examples of ICs with solid examples of good leadership turn into some of the worst prestige managers. They are incapable of observing what makes the solid leaders excellent. Since they are blind to these traits they are also incapable of applying the principles in their team. It's easy to spot these managers. Their teams bleed talent and whither. Team members who stick around are demoralized, have low job satisfaction, and through apathy eventually give up trying to make any positive change. A bad leader can poison a team or organization the way cancer moves through the body.

Partner teams

Much like the culture of a team, the culture of partner teams is extremely important. For the same reasons a bad manager introduces a cancer to the team partner teams do the same. Partner teams are by definition peers or customers. Nobody wants to work with a jerk and nobody wants a jerk for a customer. Talented ICs will eventually leave a team forced to deal with horrible partner teams.

Work-Life balance

"I don't live to work, I work to live." I believe a balanced personal life creates many more motivated employees and directly impacts the success of a company, organization, or team. For me, when work cuts into my personal time I end up despising work. More importantly, when work-life balance shifts too far toward work it's an indicator the team, organization, or company will eventually fail. ICs are asked to work too much because of scheduling / planning issues or staffing issues.

If a team, organization, or company has poor planning the quality of the final product will eventually suffer. Quality will suffer because bugs are not caught (or are caught too late to fix and reset testing to make a release date). Quality will also eventually suffer because the best employees will eventually leave. Common "fire drills" point to a process or cultural issue which the team, organization, or company cannot or is not motivated enough to change.

If a team, organization, or company has a staffing issue one must ask why. Do they suffer from high attrition? Do they not compensate properly? Do they expect too much of ICs? Can they not find qualified candidates? All of these are related to the simple fact that it's not a good place to work. Most of the issues behind the questions above are obvious, but what about the lack of qualified candidates? I believe there are quite a few extremely smart people out there looking for something new. If you can't entice quality candidates how do you expect to keep current talent?

Impact

I want to change the world. I want to work on something wonderful that makes the world a better place. I believe life is too short to not try to make the world better while we are here. There are many ways to make the world better. Entertainment, creating tools for others, basically anything that helps someone else. Someone also pointed out to me that developers typically want to work on things that ship. I fully agree -- it's pretty hard to have much impact on the world if your project doesn't ship.

Conclusion

I started writing this to help me decide what I value the most should I look for a change. It appears leadership is a top influence for me at this time. However, I think leadership is perhaps a proxy for general team, organization, or company culture. The leaders will dictate the culture of a workplace.

DarkOS: Hobbyist OS architecture taking shape.

I've been tinkering with some things in my "hobbyist OS" for a bit now. There was a bit of hiatus when my son was born but he's getting into a sleeping schedule more conducive to projects.

First, I've been messing around with a few emulators. VirtualBox seems to be the best for my work. A few key bits of functionality make it favorable:

• It doesn't require some of the more advanced CPU virtualization extensions needed by HyperV.
• It does run using some virtualization, so it's much faster than something like Bochs.
• It has a "debugger" mode with windbg style commands to dump registers, gdt, etc.

Bochs is still pretty useful, but it started throwing odd crashes during protected-to-real transitions (more on this in a bit).

http://wiki.osdev.org/ has been very helpful in all my development.

I've got a plan of attack for the OS now:

• Using long mode (amd64).
• Using ELF x86-amd64 Linux ABI. Cross compiler running in Cygwin has been built.
• No real/protected mode ping-ponging.
  
  • The real/protected transitions were proving more annoying (and variable) between Bochs and VirtualBox. I'd get one working and then the other would fail. Since this whole idea of switching back and forth is full of issues I'm just not doing it.
  • This requires slightly odd bootstrapping of components (loading kernel, drivers from disk but not processing the ELF format at first).
  • Doing this, I hope to spend more time focused on the OS and less on the quirkiness of various BIOS implementations.
• POSIX user mode environment with a Windows NT -style kernel.

Key components and TODO list:

• Bootloader
  
  • Enable A20 support.
  • Get RAM configuration from BIOS.
  • Read kernel boot options from disk or perhaps keyboard.
  • Load kernel stub, kernel, and all modules indicated as required for boot. (The kernel and modules are streamed into higher address RAM directly from disk, no ELF image loading done yet).
  • Switch video to VESA linear frame buffer mode (preferably 1024x768 24/32bit color.
  • Fill out a table of boot and machine parameters for use by the kernel stub.
  • Setup basic identity paging for long mode (amd64).
  • Enter long mode and jump to the kernel stub.
• Kernel Stub
  
  • Load kernel from raw file images the Bootloader copied from disk to full on ELF relocated modules.
  • Convert the Bootloader "boot and machine parameters" to a table suitable for calling KeMain(...) with.
  • Calling KeMain(...) to initiate kernel startup.
• Kernel
  
  • Initialize a very basic set of kernel debugger stubs accessed through a serial port.
  • Initialize the memory manager based on Kernel Stub provided parameters (used memory, etc.)
  • Initialize the object manager.
  • Create a "system" process (virtual address space, etc.)
  • Call the loader functions in the kernel for each of the Bootloader provided boot critical modules.
  • Call ModuleInit(...) for each boot critical modules.
  • Call ModuleEnter(...) for each of the boot critical modules to initiate PCI/PnP/Hardware enumeration.
  • Using file system and storage modules loaded as part of the boot critical modules group, load and start additional modules required by the system.
  • Create a user mode process ("init") to bootstrap the rest of the system from user mode startup scripts.

Once I get the basic kernel booting I hope to crib quite a few module tidbits from a free *nix distribution like FreeBSD or NetBSD. I'll have to evaluate the licenses I'm choosing for my project and the ones provided by FreeBSD and/or NetBSD.

Some modules will be pretty straight forward. For example, the VESA module will mainly just be a set of helper functions for drawing and outputting text to the linear frame buffer. Other modules like EHCI/XHCI will be much more in-depth and probably won't every be tackled.

Ideally I'd like to be able to boot from USB thumb drive, access the network card, and if the POSIX environment works out, perhaps get enough libraries and components running in user mode for a web browser and self hosting a build of the OS.

Well, that's the plan anyway. I hope to make gradual progress with a few planned breaks in the next few months (moving houses... maybe a vacation...).

Why is my default timezone Monrovia, Reykjavik?

I recently sent a friend an email from work and he pointed out my email timezone is set to Monrovia, Reykjavik and that I should refer to http://support.microsoft.com/kb/2298834.

Here is my response...

Oh, I gave up running Outlook and I'm accessing email via the Office365 interface.

Outlook had (has) some nasty bug where it takes a UI lock in a thread and then posts blocking web calls from the same thread. When the blocking calls actually take a bit of time (because the company isn't good at running data centers?) my dev machine would lose the ability to CHANGE WINDOW FOCUS or otherwise process windows messages.

I tried debugging this issue and after attaching windbg to Outlook.exe and discovering ~150 CLR threads I decided this effort was hopeless. I'm still unclear why they use the UI thread to post blocking web calls in an environment where they already have 149 OTHER threads they could use for this. Thus, I decided to use the https://outlook.com/owa/XXXX.com site instead for corporate email.

HOWEVER, more comedy ensued since MSFT apparently isn't very good with cookie management and continually tried to use my XXXX.X.XXXX@live.com tokens to access my corporate email -- the concept that I'd want to be logged in as XXXX.X.XXXX@live.com and NNNN@XXXX.com at the SAME TIME to different Microsoft website properties is completely alien.

Thus I've settled on my current solution, a shortcut: "iexplore.exe -private https://outlook.com/owa/XXXX.com"

This works very well since it ignores any persistent cookies on the system via InPrivate mode. However, I guess persistent settings like my actual timezone cannot be accounted for and I'm not even prompted for them (some UX designer probably focus grouped this and convinced themselves it was the optimal solution to just use the null timezone setting).

I thought about using Chrome exclusively for my Outlook.com needs but alas, Google seems to think the key to a successful browser is launching FIVE Chrome.exe processes utilizing ~75MB OF MEMORY to provide access to their useless "Hang Outs" feature. Which I hear now supports (THANKFULLY, the world was empty without this feature) Mustaches and Hats so I guess 75MB of RAM is a small price to pay for MUSTACHES AND HATS in a failing social media landing point that replaced the highly effective and useful googletalk.

Gone are the days of a trim gtalk.exe client and Outlook.exe (not .com) just working (for various levels of "working" exceedingly greater than the current implementation). I realize I'm becoming a crusty fossil in the industry; (already?!) clinging to concepts like trim code, easy to understand and debug projects, and my generate-the-object-code-I-intend C coding methodology...

Dispatch Objects

It seems like the next step in my OS development is going to be laying down some kernel work. I thought I'd be able to delay this, but doing so would only result in less useful work being done. A kernel is responsible for many things. I'm sure many more things than I even know about right now (such is the nature of this whole undertaking). However, my experience writing Windows drivers has offered a decent skeleton to start with.

One piece of this skeleton is dispatch objects. Basically, this is any sort of thing that can be waited upon. A timer, an event, a lock, a thread (joining a thread, etc.). I figure this is as decent a place to start as any. The way a dispatch object will work in my kernel is it will be a light weight structure containing a list of waiting threads. Of course, this means I need to start defining a thread structure as well. I imagine I'll continue to flesh out the guts of the kernel (or at least their structures) by jumping off on this one spot.

Since a dispatch object is essentially a list, the first structure I need to create is a list structure. There are many ways to create a list. You can create a linked list, a vector, or various other structures. Again, leaning on my Windows background I'll choose something very similar to the LIST_ENTRY structure in the NT kernel. In more data structure books I've seen a doubly-linked list defined as:

typedef struct _LIST_NODE
{
    struct _LIST_NODE* pPrev;
    struct _LIST_NODE* pNext;
    void* pData;
} LIST_NODE;

Here, I'll rely on something clever that Windows does with it's list structure -- there's no "data" pointer. The structure is intended to be embedded directly into any other data structure:

typedef struct _LIST_NODE
{
    struct _LIST_NODE* pPrev;
    struct _LIST_NODE* pNext;
} LIST_NODE;

typedef struct
{
    LIST_NODE  ThreadListHead;
    ...
} DISPATCH_HEADER;

typedef struct
{
    UINT32     ThreadId;
    ...
    LIST_NODE  DispatchListNode;
    ...
} THREAD_OBJECT;

The way this works is the list pointers are offset from to find the original object the list points to. In the DISPATH_HEADER above, this is simply the same address. However, say I had additional members before the ThreadListHead member, you'd then SUBTRACT back to get the original object pointer. This is done using a member offset macro, where the offset of a member into a structure is computed and then subtracted off the list pointer. While this may appear clunky at first it has one very nice advantage - you don't need to dynamically allocate small chunks of memory. In the example above, you would simply have a list insertion function that you'd pass a pointer to the THREAD_OBJECT::DispatchListNode member variable to.

There are multiple reasons this is a good thing. First, the previously mentioned avoidance of small memory allocations. Second, by not relying on any sort of memory allocation this data structure can be used in the kernel before memory allocation is even possible -- obviously, you need to write an allocator before you can use one. Third (but related to the first), the absence of smaller allocations for what is a widely used construct can add up to some pretty substantial memory savings. Any memory allocation will require a chunk of memory for bookkeeping. This is usually going to be at least the size of a few pointers. For small allocations of only 3 pointers (in the first list structure definition) the overhead of the allocation may very well be larger than the actual data usage. Considering how widely used the LIST_NODE structure will be, this is a substantial savings in memory as well as access speed (there isn't an indirect memory access to get to the actual structure, if you have the pointer to the list, it's only a subtraction to get the actual pointer versus another read).

So, there it is, the beginnings of a DISPATCH_HEADER, a THREAD_OBJECT, and a LIST_NODE. This kernel is practically writing itself...

Rolling Your Own Operating System

I started an extremely ambitious project a while back -- working on my own operating system. This isn't one that I ever intend to gain any ground or even boot on real hardware. (I'm targeting VirtualBox, VirtualPC/HyperV, and Bochs emulator environments right now.) It's just a place for me to play around with OS concepts. At work I write driver code for windows, but this code relies on a system of interfaces and mechanisms that have been designed and implemented by people much smarter than I. I'd like to start working on those more basic concepts. Sort of like peeling back the layers of an onion by recreating it. I anticipate I will learn much not only about how my OS works and can work, but about the tradeoffs that most OS designers have encountered along the way.

My current "OS" is very limited. It uses the standard DOS (well, Windows 7 -- because there are differences...) bootloader from the partition table. I created my own volume boot record to receive control from there. I'm able to load from the beginning of the partition. I'm still working on the loader. This is not be choice, but actually some unexpected first bit of learning. Here's the deal...

The BIOS loads and calls into the MBR code. The MBR code loads and calls into my volume boot record code. Now the fun starts. There are numerous extremely important interfaces the BIOS provides to early boot code. Before protected mode, paging, multi-tasking, and all the other things that really make an operating system an operating system you could just happily keep calling these BIOS interfaces to get this functionality. However, the BIOS code is legacy... It's so legacy that it isn't all that compatible with the way a modern OS sets up the CPU. (There are a few more compatible extensions -- VESA graphics BIOS stuff for example, but even this is severely limited)

So, it's already decision time. A lot sooner than I thought it would be. How do I start laying the ground work for the more modern features of my OS when I need to use the BIOS routines to talk to the hardware? Well, the short answer is I need to stop at some point, and probably the sooner the better. I don't want to litter my kernel code with a bunch of BIOS dependencies -- anything I write that requires this cannot really be reused anywhere else. Ideally, I would switch over from BIOS interfaces to my own all at once, however that will be difficult. Instead, I've decided a better approach is to rely on the BIOS interfaces until I'm ready to create my own. So, what does the BIOS give me that I need to reproduce?

1. VGA/VESA support -- in order to know my OS is doing something, I should probably have a way to see it working. ¹
2. Keyboard support.
3. Disk IO support.
4. ...

I'm sure there are many more but these are the interfaces I'm immediately aware of. The first item, VGA support was easiest. This is probably because it's also the only one that's moved past legacy support. I am able to select a video mode which allows me to draw 24/32 bit RGB directly into a flat buffer space. This buffer space is directly mapped to the screen buffer, so I'm set as far as rudimentary drawing is concerned. I don't get any hardware acceleration and I'm stuck having to implement all my own line drawing, bit-blitting, and other APIs, but at least it's doable. The most annoying thing about this is I cannot adjust for changing the monitor resolution, but if I look at my above goals of not really caring if this runs on real hardware this is acceptable.

Now comes disk IO. This is a bit sticky, since there is no easy way to go about this. I'm probably going to have to support at least IDE and AHCI. I'm not exactly sure if I can get away with IDE only and rely on hard disks to support the legacy commands... I think so, but not sure. I also cannot keep thunking back and forth between BIOS code for this support -- every time I do that I'll probably have to tear down the things I really wanted to learn about -- memory paging, scheduling, etc. I see some work on a storage driver in my near future... It does indeed look like I can just go with IDE. Hopefully, IDE will be pretty easy to work with. It looks like the specs are open as well -- http://www.t10.org/t13/technical/d98120r0.pdf

Here's what my volume boot record code looks like so far: http://www.darkautomata.com/blog/os/vbr_2013_11_29.asm.txt

---

¹ I can just use a COM port interface for logging what is actually happening in the OS, but there is going to come a time relatively early in the OS where I'll actually want to SEE what I've created. Maybe this isn't as big of a requirement as I think it is right now...

Screen DPI vs Remoting issues

It seems to me the user experience for high DPI displays like the Apple Retina display will be horrible for certain scenarios.

If I'm remoting into a windows machine using Remote Desktop or displaying an X application using X11 remoting then the host DPI settings are probably going to result in a bad experience. I already see this in when using my work laptop to remote into my desktop.

At work I have multiple 24" 1920x1080 monitors. When I RDP into the machine from my 14" 1920x1080 work laptop I get text in the code editor that is barely legible. I can't imagine how bad this would be on the Retina display!

The Remote Desktop and other remote display clients should enable a pass-through mechanism for client DPI. This way the host can probably format the display for remoting.

To expand on this idea, if all UI settings relating to size were stored in a device independent way (for example: twips, 1/1440 of an inch) then a translation could take place to always match the physical dimensions of the UI. You could even have a constant scaling factor so you can change this for a device. For example, I may not want my cell phone displaying at full physical dimensions, maybe it's okay to go with 50% or 25% and I know I have to squint at it, but I want my laptop or tablet to use 80-90% physical dimensions, etc...

If no one has thought of this yet well it's here first... prior art recorded.

Native or Managed?

I want to start a project with a friend to upgrade the UI of windbg. It's not too bad right now, but more importantly I think it could be so much more.

So here's the problem: Making an "awesome" UI is much easier in C# than in C++ using native code. However, the APIs for the debugging engine are all COM based and there aren't bindings for C#. There are a few unofficial bindings, but nothing really available. So, we'd probably have to roll our own bindings, which is not something I want to do.

I think a hybrid option would be to use C++/CLI. At least we'd be able to use the COM methods pretty easily and maybe even use the header files without any modification. We could also then use the managed API for the GUI. I think I'm going to suggest this as the route to take.

There's some concerns that .Net wouldn't be installed on all the environments you'd want to use the new UI but I don't think that should stop us. It's pretty easy to install and a version of .Net ships with modern Windows distributions already.

A bit to think about, but I think the C++/CLI approach is going to be the way to go. It will get me writing code faster and having SOMETHING working much faster than a native solution. I've recently worked on native GDI+ code and although it's nicer than GDI it's still not something I'd want to force myself on for a GUI intensive project.