Handmade Hero: A Systems Programming Odyssey

Table of Contents

Section 1: Why learning from Handmade Hero still matters

Section 2: Reasons for creating a GNU/Linux port of Handmade Hero

Section 3: Is yet another Xlib post necessary?

Section 4: Why use Xlib for graphics display

Section 5: Client-Server Architecture

Section 6: Installing dependencies

Section 7: Developing an X Client application

Subsection 7-A: Headers

Subsection 7-B: Connecting to the XServer

Subsection 7-C: Creating a Window for the Game

Subsection 7-D: Mapping the Window

Subsection 7-E: Expose Events

Subsection 7-E-1: Handling Expose Events in X11-based Linux desktops

Subsection 7-E-2: Handling Expose Events in Wayland-based Linux desktops

Subsection 7-F: Pausing the Game

Subsection 7-G: Destroying the Window

Subsection 7-H: Closing the Display

Subsection 7-I: Initial Platform Layer of the Game

Subsection 7-J: Compilation

Subsection 7-K: Running the Game

Subsection 7-L: Checking Memory Leaks with Valgrind

Section 8: Conclusions

Section 9: Final Thoughts

Section 10: References

Section 12: Ports This post is the start of my Handmade Hero journey documentation. For those who do not know what Handmade Hero is, it is a completed game development series done by legendary game engine developer Casey Muratori. In the series, Casey shows how to build an entire game from scratch in Windows to teach us how computers work and write performant software. For me, and many others who have followed the series, the best way to solidify the knowledge is by writing the code on a different platform (not Windows) and posting about it. The reason for choosing a different platform is that you are forced to explore the platform (with its distinctive characteristics) on your own rather than following an established path or solution. However, writing this post is not all about learning but about sharing knowledge and experiences that might be useful to other developers. Because of my intention to share knowledge, I have decided to write this post with an academic-like format so that it could be used as a reference for low-level systems programming. The next section presents a table of contents for quick navigation in the hope that this resource can be used as reference for building low-level window applications in Linux. Section 1 explains why Handmade Hero remains relevant even in these times where AI can generate code at scales never seen before. Section 2 reveals my motivations for porting Handmade Hero to Linux. Then, I talk about the value of posting about libX11 (commonly known as Xlib)—the library that we are going to use to put graphics on screen in Section 3. Then, in Section 4, I answer the difficult question of why to choose Xlib for graphics display, despite that even X11 developers recommend other alternatives. After that I talk briefly about the client-server architecture of the X11 protocol in Section 5. Installation instructions of the development files for developing X11 client applications on Linux are presented in Section 6. The X11 client development dive starts in Section 7, the development is broken down into subsections to make the explanations more manageable; here is where I talk about the headers, creating the window, mapping it to the screen, handling events, and closing the display. Compilation, execution, and checking for memory leaks is covered for completeness. The complete listing of the source code is given after that with notes for those who prefer to head directly there. After the development section I share the lessons learned in the conclusions section. That is followed by my realization of the importance of Xlib to the current software development world as I was writing this post. A list of references for future study is given and also I credit the work of other developers that helped me learn about Xlib in the early days. My post concludes with a list of Handmade Hero ports that might be of interest to enthusiasts and software developers alike. Use the table of contents to view an outline of the post and to navigate to the sections that you might want to (re)visit. If you are here for the source code, you can copy it from Subsection 7-I. Handmade Hero has never been more relevant than ever for software developers. Largely because of the increasing pressure of shipping faster by integrating software development with AI, this trend has intensified the work without leaving much time to developers for crafting better solutions by writing exploratory code. Instead, AI proposes possible solutions that at best are based on well-known patterns; leaving little room for developers to innovate and drive solutions through the hindsight that stems from writing the code. As a result of this trend, software developers write code less frequently, and so their ability to craft innovative solutions by interacting directly with code could diminish. It is not hard to see that developers would write code less often and would be less motivated to do so in settings where productivity is measured in terms of AI token usage and the number of lines of code generated by AI. Working in an environment like that can lead to an eventual loss of confidence or interest in writing code because you are not perceived as productive by their metrics. If you are happy about the way things are, all of this probably sounds foreign to you. However, if you feel dejected by this I just want you to know that there are things that you can do to overcome this. A possible way of overcoming this could be as simple as working on an engaging side project that keeps your ability to engineer solutions through the act of writing code honed. When you are concentrating on an engaging project, it can help a lot and make you feel better with yourself, family, friends, peers, coworkers, and the environment. Doing this can also mean that your craftsmanship not only remains but evolves and enables you to find ways to integrate AI better into your workflow and so you could get the best of both worlds. For me, that engaging side project has been Handmade Hero. Handmade Hero is a low-level systems engineering series that legendary game engine developer Casey Muratori streamed for a period of two years to teach developers and enthusiasts alike how computers work and how to write performant software by developing a game from scratch (no libraries and no engines). This series has helped me encounter a balance between using AI to generate code for some of my work while keeping my problem-solving and coding skills honed in my spare time. Even though developers have written about the series many times, each post is unique and it always matters because it is a transformative experience. I just want to add that posting about that experience makes it even more meaningful. Handmade Hero is about learning how computers work, and Casey did that by showing us how to use the Win32 API to build a game in Windows. And he used Windows because it's the platform he is most familiar with. By analogy, I want to build the game in Linux because I am comfortable with the command-line (it gives me that "at home" feeling) and because I really want to know my platform better by writing systems programming code. On a side note, I am not adamant about trying to convince anyone to switch to Linux. You will know if it is for you after interacting with it for a while. I would like to add that I do not intend to skip the Win32 platform because Handmade Hero is a cross-platform development experience. The scope of this post is to share what I have learned about Xlib to create a game window in Linux by following the way of Handmade Hero craftsmen. To me, that meant that I had to read the Xlib man pages, dive into the source code to peek at its implementation, and also borrow ideas from the Quake-II engine. There are books and many great posts about libX11 (commonly known as Xlib), is there really a need for another post? The authors of the Xlib guide for new developers express that there is plenty of documentation but a large portion of it is outdated or not particularly well-written. They also express that learning X is tricky, fortunately the learning curve may not be as steep as one might think. Nevertheless, if you try to dive directly into the Xlib man pages without prior exposure to the C Language X Interface, or "The Specs", you will probably hit a wall (as happened to me) because of the nomenclature and concepts that you need to understand. So the man pages are an excellent resource as a reference of the API; however, they were not written to teach you anything in a tutorial kind of way. Unfortunately, the new developer guide is not considered complete, not even by the authors and for that reason there is a gap of knowledge that can be closed (to some extent) by studying the source code and experimenting with it. I would like to add that probably the best way to learn Xlib is by interacting with the X developers and contributing to the project. I wish I could say that I have had that experience but cannot say so at the time of writing this post. To answer the original question, any post about Xlib that tries to look at it from a new angle or that is written with the intention to teach or to share experiences is valuable. I strongly recommend that if you are serious about learning Xlib that you invest some time into reading the guide for new developers, for it is a great starting point because it provides high-level descriptions about the X Windowing system. I would like to end this section by saying that now that programming is more widespread, it is more likely for professionals from different domains other than computer science to write code. And I think that is great because they learn about computers from a different standpoint that perhaps is more relatable to non-computer scientists and software developers. So I believe it is better for the world to not just have people from different domains writing software but also writing about it. So why use Xlib for putting the graphics of the game on the screen when the official documentation from Xorg recommends not using Xlib but instead using the X C Bindings (XCB) for low-level development or toolkits such as GTK+ or Qt for GUI applications? To answer that question it is important to have in mind that Xlib development began in the 80s and it has remained backward compatible since then. If it didn't, then devices equipped with older hardware (such as embedded devices and medical equipment with long life cycles) still in use today would be rendered useless. We should recognize that Xlib developers have done the best to preserve backward compatibility while modernizing it. At present, there is not much that can be done to improve its performance without compromising the existing architecture. To give you an idea of what this means, some Xlib calls incur roundtrips that hinder its performance on modern hardware but those roundtrips are features that are needed to remain backwards compatible. This is one of the major reasons Xlib developers do not recommend using it for new projects. The other reason is that modern GUI applications, such as desktop environments, leverage toolkits tailored for modern applications, such as handling complex international input and output. The toolkits have better programming models than the underlying Xlib layer and for that reason applications should call Xlib sparingly (only to reach out for features not provided by the toolkits). For modern low-level application development on embedded devices the obvious choice is XCB. I don't think that anyone would object to that. I strongly recommend reading the official Xorg guide for new developers for those wishing to learn more about this. The other thing that we have to keep in mind is the philosophy of Handmade Hero—to write exploratory code to learn about the problem space and after getting an idea of what the solution looks like, optimize it. Since we are just starting the development we are content with any code that allows us to display a window for our game on the screen. We do not care if it looks clean or if its performant. What we care about is that it works and that we know (to some extent) how it works. And the latter is an important point for me, we have the privilege of having the opportunity to learn from source code written by professional developers because of open-source. This is why I do recommend reading even if it is just the headers of Xlib so that the code that you call is not a black-box to you. This is one of the advantages of developing the game for the Linux platform over doing that for Windows, that you can take your time to study the source code of all the libraries that your game depends on. Now that we have talked about some general reasons for using Xlib, we can consider some key specific reasons for using Xlib. In the handmade spirit, what we need for displaying the graphics of our game is a framebuffer structured in a way that the screen can understand. Also we are not going to need widgets, menus, text, buttons, etc. for our game; this is why we do not need toolkits. What we do need is a window to put the game graphics for the delight of our players and we can achieve that through Xlib calls (see Xorg-wiki). What Xorg developers discourage is reinventing the wheel of the entire stack because it disregards what took several years for professional developers to implement and improve. For this use case the Xlib code is relatively straightforward to write. The client code that one has to write essentially reads by itself, you don't need to know the entire Xlib API to understand it. This means that the Xlib code that we are going to write is readable and that is a good enough reason for using it in my book. Another reason for using Xlib is that it has been modernized, it leverages the X C Bindings (XCB) under the hood, it also supports multi-threading (if XThreads are initialized on the client side), and it is still maintained to this day by Alan Coopersmith, a veteran Xlib developer, and many others. Surely a properly implemented client code with XCB will perform better than its Xlib counterpart but that depends entirely on your ability to write that code (as mentioned in this XCB tutorial). Working directly with XCB entails that you must have a fairly good understanding of the X11 protocol. In the context of handmade game development, reaching for XCB does resonate with the Handmade Hero spirit only after you have determined that Xlib calls limit performance. Casey also advocates that it is best to write a working code first to get a good idea of what the platform layer should be; and only then work on optimizing it if there are factual reasons to do so. Thus my rationale for using Xlib is that I want to push it to its limits and then after identifying that performance limitations stem from Xlib itself then I would consider using XCB. And if I were to focus on performance bottlenecks, I would look at the game loop code because the less overhead the better so that the game would hit the target framerate. I would not bother too much about the Xlib code preceding the game loop. The expectation is that the game will at some point leverage XCB in the game loop and use Xlib elsewhere for readability. To be clear, I do find it appealing to develop the platform layer of the game with XCB. However, choosing XCB over Xlib right now without having a playable game would be an early optimization for me. We cannot talk effectively about performance until we have a system running. This is why I would rather spend time getting a working system sooner with Xlib and then consider the performance bottlenecks. One has to consider that Xlib supports multi-threading and extensions such as the shared memory extension are features that should be explored in addition to speaking directly to the X11 protocol with XCB. It is worthwhile to mention that Xlib and XCB code can coexist, that means that you can have the best of both worlds by using XCB for optimizing the rate limiting features while keeping around the convenient functions that Xlib provides. After conducting the due research I would be at a better position to assess performance differences and post about what worked best. Another reason for using Xlib is that Handmade Hero has not been my first experience with game development. I started learning about game development by reading Quake-II's source code and so by the time I found out about Handmade Hero I knew that Quake uses Xlib to put graphics on the screen in Linux. It was only natural to avail myself of the experience of diving into Quake's source code to develop my own Linux port of Handmade Hero. I would like to tell more about why not using a toolkit with more details here. The strongest reason for not using a toolkit is because it hides the difficulties of dealing directly with visuals, a lesson on how computers work that I do not want to skip. Some of those difficulties translate to the platform layer of the game. For example, if the visual is TrueColor (modern monitors) one has to work directly with RGB bitmasks so that the colors shown on the game window look right. It is important to mention that the X Server (more on that later) still does the heavy lifting to determine the properties of the visual, such as if the visual is a TrueColor or PseudoColor type, whether it has a 16-bit or 24-bit depth, the RGB layout in memory (bitmasks), etc. To be precise, you can still experiment with RGB bitmasks with the GTK2 toolkit (see man page) but you must consider the pros and cons of adding another layer of abstraction to your application; probably, the answer to that lies with you (or with your team). The last reason is the most personal one. By the time I began my transition from Windows 7 to Ubuntu 9.10 (code named Karmic Koala) that Linux desktop shipped with libX11 version 1.2.2 (that can be verified via the manifest). At that point Xlib already had the modernized XCB transport layer that made my Linux desktop experience so memorable. Knowing that GTK2 at that time was leveraging Xlib code heavily (as can be verified in the source) to create the desktop environment makes me want to build the platform layer of my game with Xlib (call it nostalgia if you may). Also my desktop is using a version of Cinnamon (a fork of GNOME2) that still depends on Xlib and so it is natural for me to stick with it. I just want to add here a little note about possible issues that you could stumble upon if you try to run the game (without modification) on a Wayland-based desktop. Xlib blocking calls, such as XNextEvent and XWindowEvent, must be replaced with their polling alternatives to avoid potential deadlocks due to Wayland's asynchronous design. Xlib has a client-server architecture as illustrated in the following diagram: The diagram shows that the X Server receives the user input from the keyboard and mouse. The diagram shows that the applications that we use such as the browser and the console are clients, and that even the desktop environment is a client (in some Linux distributions). I would like to comment here that the shown diagram is a simplified illustration that does not delve into the fact that the server has both device-independent and device-dependent code. If you wish to look at a closer depiction of the architecture see the one provided in the X Window Concepts section of the guide for new developers. An interesting finding for me that helped me realize that even desktop environments can also be X clients can be found in the Cinnamon desktop environment source code. For context, the Cinnamon desktop environment is actively developed by Linux Mint developers. If you do look at its source code you will see that a connection to the X Server (via XOpenDisplay()) happens in the main source file. The communication between the client and the server happens over the network via the X11 protocol; this means that the client could be on a remote machine as well. For security ssh forwarding (see man ssh) is commonly used to encrypt the communication between the client and server. If both the client and the server are in the same machine, a Unix socket is commonly used for communication. It is worth mentioning for clarification that the server does not capture the input directly from the hardware, that is the job of the Linux kernel. And that is important because it hints that the server implementation is modular. In general, a client application tells the X Server what operation it wants to do (such as drawing) and the server responds to the request by performing that asynchronously. The motivation for this architecture is that it solves the problem of multiple clients competing for the same portion of the screen. The server also knows the window the user is currently using (for input) and also knows the client that owns it, and so by extension it knows the client that will respond to the input events. It helps to bear these aspects of Xlib in mind when reading X client applications. If you want to follow along you would probably need to install the development libraries for creating client X11 applications. If you have not done that before it is likely that they may not be installed on your system even if your desktop environment still uses Xlib. On Debian-based distributions (such as Ubuntu and Linux Mint) you can do so by invoking the package manager from the command-line: For example, if you wish to consult the documentation for opening a display (more on that later) you can use the command-line string: if that does not work use the 3 tells man that what follows is a function from a library. Now we can start writing our X client code to create a game window for our game. Xlib is the original Application Programming Interface (API) of the X11 protocol, written in C. Although there are bindings to make it possible to call Xlib from other languages such as Python, Go, and Rust, you would want to write C code if you wanted to work closer to the metal. Another reason for using C is that you will be free of the overhead of mental context switching when reading the Xlib source code or documentation. The Xlib code will not seem foreign to you as it would if you are calling it from Python. If you are familiar with the Handmade Hero series you know that Casey essentially writes C code on stream despite the fact that he compiles the source code of the game with a C++ compiler. He did not leverage C++ features heavily until implementing vector math (via operator overloading) to make the operations read more like math to the next developer. C programs typically start with a header section that specifies the function prototypes (declarations) and types (data structures) that the program needs to compile the source down to machine code. The minimal set of headers that we may want to use for creating the game window, having an improved debugging experience, and logging useful information to the developer (or user) is the following: The standard input-output header <stdio.h> provides functions for formatted printing (such as fprintf) which can be used to channel messages through the standard output stdout and standard error stderr streams. By doing it this way, we can redirect error messages to a file for debugging without the informative messages that we may want to show to the user. The <X11/Xlib.h> header provides the necessary definitions for the Xlib data structures, macros, and functions that we need for putting our game graphics on the screen. The Xlib header should be located on your system at the standard path /usr/include/X11/Xlib.h. If you read it you are going to find out that client X window applications that set the XLIB_ILLEGAL_ACCESS macro to a truthy value before including <X11/Xlib.h> obtain (limited) access to the Display data structure which contains all Xlib internals. As we advance in the discussion we are going to use it to reinforce our understanding that Xlib is asynchronous. I do not recommend using this hack to write the platform layer of the game, instead use the macros to make your client code portable, readable, and maintainable. With those definitions we will be able to open a connection to the X Server, create the window for our game, and put graphics on it (in a future post). The first step towards displaying a window with Xlib is to establish a connection with the X Server via the function call XOpenDisplay() which takes the hardware display name as an argument. In Linux it's okay to pass NULL, in which case the parameter resolves to whatever the shell environment variable DISPLAY holds. If the call succeeds the function returns a pointer to the Display structure. In the Xlib context this encompasses not only the monitor for graphics output but also the keyboard, mouse, and other peripherals for capturing user input. On the other hand, if the connection to the X Server fails, the function returns a NULL pointer and the code should stop and return an error message for the user. It's okay to put all the Xlib code in the main function, this aligns with the spirit of exploratory development philosophy that Casey advocates throughout the series. We are discovering what our game needs from the underlying platform to put graphics on the screen and (maybe in a future post) handle user input from peripherals (keyboard, gaming console controller, etc.). Behind the scenes XOpenDisplay() in Linux opens a socket in non-blocking mode to connect to the X Server so that the server may handle requests from multiple clients asynchronously. If the X Server is running in localhost then the connection to the X Server happens through a Unix socket for performance. It is interesting to note that the code that performs the socket connection is not present in Xlib, but rather in libXCB. The file descriptor of the socket is stored in the Display structure. I am mentioning it because Xlib keeps the file descriptor around for subsequent manipulations via fcntl calls, not because we are expected to read (write) from (to) the socket ourselves. Display also contains other useful info that's fetched from the X Server such as the minimum and maximum values of the keyboard codes (we care about this for handling user input), the number of screens, the default graphics context, etc. From the X Server we also get valuable information about the screens such as the default dimensions (width and height), the values of the white and black pixels, the screen depth, and the visual information. In particular the visual info stores the RGB masks so that we know what the RGB layout that the X Server expects for packing the pixel data; otherwise, the colors that you see on the screen are not going to be what you might expect. Xlib also gets the default screen by parsing the DISPLAY environment variable by calling dedicated libXCB utils. For example if DISPLAY is :0 we know that the X Server is running in localhost and that the default screen number is zero. This matters because the communication to the X Server goes through a Unix socket; otherwise the communication falls back to TCP/IP if you supply an IP address 127.0.0.1:0 as the hardware name. Note that the TCP/IP fallback happens even if the IP address happens to match localhost. This matters to us because this tells us that X11 developers care about performance (communication over a Unix socket is preferable when the client and server are in the same machine). Shortly you will see the default screen is zero by peeking at the Display structure. This is where the fun part really begins. I strongly recommend you to consult the Xlib documentation (man pages) as you read the code snippets to familiarize yourself with some aspects of Xlib that we cannot possibly cover in a post like this. The other recommendation is to read the Xlib source code that the Quake engine uses to display graphics on screen with Xlib. That should give you an idea of the Xlib code needed to develop a solid platform layer of the game. Chances are that you will need to adapt the code for the Handmade Hero game but that is part of the exploration that you want to get into. The following snippet borrowed from the Xlib official documentation shows that to create a simple window via XCreateSimpleWindow() one needs to have an active connection to the X Server (a display), a parent window, coordinates with respect to the top-left corner of the parent window, window dimensions (width and height), a border width which can be zero for our purposes, and colors for the border and background: For the purposes of putting graphics on screen for our game and keeping things simple we can tell Xlib that we want the root window as the parent of our game window via the macro DefaultRootWindow() which takes as argument our display. The macro will return the Window ID of the root window. We can pass zero for both the x and y coordinates, we could specify standard dimensions for our window such as 1600 x 900 where the width, 1600, and the height, 900, are in pixels. If you wish to make your code more portable you may get the default screen of your display and use query macros, WidthOfScreen() and HeightOfScreen(), to use the default dimensions: There is also a query function for getting the black pixel value, BlackPixelOfScreen(), for the screen to use it for the border and background colors: Thus to create the window of the game with a black background and default dimensions you should write: Bear in mind that this alone will not make the window visible and this could be a little surprising at first because we are used to the Object-Oriented Programming OOP mindset. We might expect that the Window is an object but that is not the case, it is an Xlib resource ID (XID) of 64-bits in size (typedef unsigned long XID;). What happens under the hood is that the request for creating a window is stored in the Display data structure. That means that the X Server knows nothing about this until we explicitly ask the server to process this request (more of that later because we still have work to do). Xlib behaves this way for performance, it stacks our requests until the time is right for processing them. When the server does process the request it will allocate memory for the window but that memory region is not accessible from client applications (it is private to the server). From looking at the display data with gdb we can determine that we have a request that has not been processed by the server (only showing some of the fields): The sequence number of the last request considered by the server is 6 (last_request_read = 6) and that happened when we called XOpenDisplay() and the sequence number of the current request is 7 (request = 7). This means that the request is stored locally in the display structure. The server has not seen it because we have not called XFlush() ourselves to push the requests to the server. Xlib's implementation is conservative on what function calls have the side-effect of flushing the output buffer (we will call one soon enough) and so client applications seldom have to call XFlush() directly, as stressed in the official documentation. Only functions that require an immediate response from the server flush the output buffer and block until the response has been received. I hope that this has convinced you of the asynchronous nature of the X protocol. You may also wish to name your window as Handmade Hero at this point as Casey did to get that hello world experience when your game window appears: Calling XStoreName() also updates the output buffer with the request to change the name of the window shown in the window manager, the function does not trigger a synchronization request with the server as in the previous cases. It is interesting to see that it forwards the operation to the XChangeProperty() function (which implements the corresponding X11 protocol request): As can be seen from the code snippet, the function we just called is a convenient wrapper that makes the Xlib code easier to read. The display and window handle arguments are easy to see, the new ones are Atom macros,XA_WM_NAME and XA_STRING, which are defined in the standard header X11/Xatom.h, that help identify the property name and the type of the property. The underlying size of the Atom is platform dependent (either 32 or 64-bit integer). In this case the name of the window and the string type. The magic number 8 tells the server that the property should be interpreted in chunks of 8-bits which make sense for strings. The PropModeReplace instructs the server to replace whatever it had stored for that property. If the name that we provide is a NULL pointer or if its length exceeds the maximum storage size it returns zero to indicate that an error has occurred. This behavior is not explicitly mentioned in the man page; however, the official Xlib documentation states that zero status is used to hint errors. In this case it is up to the client application to check the status and up to the developer to determine what was the cause. This is a good example that the source code itself is the ultimate documentation. The stored name remains in the server until it receives a request to destroy the window, it is not stored on the client side. It is important to note that to make the window visible we will need to make a mapping request, and that is the topic of the next section. To make our game window visible, we need to add a window mapping request to the X Server. The Xlib function that allows us to place that request is XMapWindow(). But before doing that we have to modify the attributes of our game window so that it responds to Expose events. By design, the X Server will not send events unless the client application requests them. From the context of the ongoing discussion if our game window is not configured to handle (graphics) Expose events the server will not send any. The interested reader might want to consult this resource for verification. So, if we were to call the function that looks for events (right now without changing the window attributes) the call would block until the specific event is received from the server. It is not hard to see that the server will not send any such event because we have not requested that from the server and so the client application will hang there indefinitely (a deadlock). To change the attributes of our game window we must call XChangeWindowAttributes() with an instance of the XSetWindowAttributes data structure. For the objective of making the window ready for graphics display all that we need to do is to tell the X Server that we want to respond to exposure events and therefore we must set the event_mask field with the ExposureMask in this way: It is worth mentioning that if we wanted to capture user input from the keyboard we would bitwise OR the event mask with the KeyPressMask and KeyReleaseMask as in this example: Note that these event-masks are actually bitmasks and they can be found in /usr/include/X11/X.h: Just for displaying our game window we only need to set the event_mask with the ExposureMask and nothing else, really (a future post will address user input). The function XChangeWindowAttributes() enables us to change the window attributes, which has the following signature: It takes the usual display pointer and window ID, a bitmask that corresponds to the window attributes that we want to change, and a pointer to the XSetWindowAttributes data structure. Because the only thing that we want to tell the server is about the events the game window will respond to we only need to pass the CWEventMask bitmask, this one is also defined in the <X11/X.h> header: You may also find it worthwhile to read about Xlib's window attributes. The call in our code would look like this: Now we are in a good position to place the request of mapping the game window to the X Server to mark it as eligible for display via the XMapWindow. The function takes as arguments our display and Window ID: It's worth noting that at this point we are still adding requests to our display structure locally, the X Server is unaware of our intentions until we call XWindowEvent to wait for the Expose event so that our game window becomes visible. This can be confirmed by looking at the display data: We can see that the sequence number of last request read has not mutated but the request sequence number has increased to 10 after calling XMapWindow(). We have requested a name for our game window, requested attributes change, and performed a window mapping request (explaining the sequence number increment). As we saw in the previous section mapping the window does not make it show up, instead it batches the request into the output buffer. To make the window visible we need to flush the output buffer and wait for the server to send the Expose event. Xlib provides blocking and non-blocking function calls, it is up to the developer to make the code on the client side capable of handling asynchronous requests. To make this clear, if the server is operating asynchronously a blocking call might result in a deadlock—a situation in which the client is stuck indefinitely traversing an outdated event queue that does not contain the event the client cares about. This is particularly important for X11 client applications running in a Wayland-based Linux desktop. And this is even more relevant now than ever as major Linux distributions, such as Ubuntu and Fedora, are switching to Wayland. Therefore, this section aims to answer the question how to handle events of X11 client applications in both X11-based and Wayland-based Linux desktops. We start by talking about the blocking (or synchronous) approach that is compatible with X11-based Linux desktops in the context of Handmade Hero. In the context of writing the code that makes the game window visible all we need to call is the XWindowEvent(). The signature of that function is the following: again we have the display and the window ID, we also need to pass the event-mask ExposureMask of the Expose event, and a pointer to an XEvent data structure. It is interesting to see that the XEvent is an union of all events, and that the X11 protocol reserves the first four bytes to store the type of the event. This is a clever way to send different events consistently over the network. The code that we need to write to wait for the game window to become visible is the following based on what we have found about the X11 protocol: The XWindowEvent() call will block until the game window comes into view, which is one of the conditions that causes the server to send the Expose event, as stated in the official documentation for expose-events. Because of our use of XWindowEvent() the only event that gets pushed out of the event queue is the Expose event, all the other events are preserved in the event queue. We can verify with gdb that the sequence number of the last request read now matches the current one: The game window should now be visible on your screen. Wayland supports running X11 client applications via XWayland, which is based on the original code for the X11 server. In a nutshell, XWayland acts as an intermediary layer that translates the X11 protocol between client X11 window applications and the Wayland compositor. In modern software development you can think of this as another layer of abstraction that adds an overhead to the process of displaying graphics and handling user input. The polling version of the code compared to the one we used for X11-based Linux desktops could be written this way for the Wayland counterparts: Note that the choice of looping indefinitely is deliberate to expose the asynchronous nature of the communication with the XWayland layer; we don't know when the client will receive the Expose event. We are not going to do anything special with the event itself and this is why it is scoped to the while block. The XCheckWindowEvent() polls for the specified Expose event, if the event is found in the event queue, the function moves the event into the XEvent structure and returns true, otherwise it returns false to signal that such an event is not in the queue (in this case the event structure retains its zero initialization). The XCheckWindowEvent() function can be thought to be an efficient find and remove from the queue operation in one go for performance. To avoid wasting CPU cycles on the polling you may want to use a high resolution clock such as clock_nanosleep just before stepping into the next while-loop cycle. To avoid shifting the attention into that I am not going to discuss that here. Just mentioning it to motivate you to explore this on your own. Regardless of the platform you should be able to see the game window. One thing that you want to do is to pause the game to be able to see the window on your screen and this can be done simply by requesting a read from standard input: where the fprintf shows an instructive message to the user (the game is paused and press enter to continue) on standard output. And the function fread is used to read a byte from the input stream stdin. The fread call is a blocking call and that means that the code will not proceed until a character has been read, effectively pausing our game. Note that the variable c is just a placeholder for the byte to be read. The sizeof() function is commonly used by Linux kernel programmers to write type independent code (the placeholder may change but the fact that we intend to write into the entire placeholder will not). The magic number one after sizeof tells us that we only want to read one item of size sizeof(c). This could be a little strange at first and this is why I prefer to use read instead but to keep things to a minimum I decided to stick with fread. As mentioned in the creating a window section the X Server allocates resources for the game window and so the right thing to do is to request the server to destroy it before we close the connection. The server will destroy all the properties associated with that window and decrement the global property registry accordingly. Destroying the game window can be done by calling XDestroyWindow(): As you can see the function takes the usual display pointer and window resource Id. If you look at the implementation of XDestroyWindow() in Xlib you will find that it stores the destroy window request in the display structure. There is no implicit synchronization, it is not really needed, because closing the display does the final synchronization. That's when the server will receive the destroy request and when the resources allocated on the server-side will be freed. At the end of the program, you should close the display via XCloseDisplay() so that Xlib's internal data structures are freed from heap memory and the socket used for communicating with the X Server is closed: As mentioned in the preceding section the X Server performs a final synchronization via XSync() to send requests in the output buffer (this feature is also mentioned in the man page). Here is the complete source code to create the game window, the code should work on both X11-based and Wayland-based Linux desktops. Normally, I write comments to inform about things that are not evident. (As Casey mentioned in the series various times "comments are always outdated" and so the source code is the source of truth about what the code does.) However, for the source code posted in this section to be useful to developers that are new to Xlib I have added some notes. As you familiarize with Xlib, you will see that the X11 client code reads by itself. If you bear in mind that many of the calls just push requests to the output buffer on the client side, and that few calls like XWindowEvent() flush the output buffer and block until the server responds, you will see that developing X client applications is easier than you think. If in doubt about what function calls block you are recommended to read "The Specs" or the man pages. Note that the XLIB_ILLEGAL_ACCESS definition has been excluded from the source code to reinforce that it should not be used except when you have strong reasons for looking at Xlib's internals, such as debugging and learning purposes. Use Xlib's macros to write portable and readable code. For simplicity we have opted to write all the source code in a single source file linux_handmade.c that goes with the convention established by Casey in Handmade Hero to prefix the platform name for the platform layer code. The command-line for compiling the code with GCC is the following: where we are telling the compiler that we want our code to conform to the C99 standard with GNU extensions, this is so that we can initialize structs the way Casey did on the stream; we are also asking the compiler to enable all warnings, generate debugging symbols in the DWARF version 4 format that valgrind (memcheck tool) understands, and apply optimizations that will not interfere with the debugging session. The last one -lX11 is for the linker so that our code gets dynamically linked with Xlib. It's important to mention that Casey uses a batch file to compile the source code and that's what I also did at first when I was working solely on the Win32 implementation (I started following along on Windows). However, to have a consistent build I am currently using a Makefile that can be used for compiling the source in Windows (via MinGW) and Linux. The advantage of using a Makefile is that one can extend it for other platforms—"one Makefile to build them all". To run the game from the command-line: where the dot slash (./) prefix tells the shell to look for the executable in the current directory. This is important because if you are new to Linux, you don't know that commands and executables must be in the PATH in order to run them. For those executable that are not in the path the dot slash prefix is required as in this case. Valgrind is a tool suite for debugging and profiling programs. I use it often to check for memory leaks at the end of the program. In the context of our simple client application we are making sure that by calling XCloseDisplay() all the internals allocated in the heap memory are freed. In Xlib, there are some resources that we need to free ourselves by calling XFree; there are others and so it is important to check the official documentation for the right function call to free a resource. To use the tool to check for memory leaks use the following command: where the -s flag informs valgrind that we want a list of detected (and suppressed) errors so that we can address them if any. The output for this code is the following: We can see that in the heap summary reports 91 allocations and 91 frees and so this means that by calling XCloseDisplay() we have made sure that the client application releases its memory to the operating system. Even if we don't do that the operating system will reclaim the memory anyways but it's a good practice to do so. The benefit of doing these checks periodically during development is that you can find errors related to memory more easily, reducing the time needed to find the faulty line of code. I would like to mention that leaving out XDestroyWindow() from the client code would not affect the total heap usage results. The reason for that is that the memory for the window is allocated by the server; this request instructs the server to destroy the window and its properties. And even if we miss the call (due to a crash for example) the server will free the resources allocated to the client and that of course includes the window. In this post we delved into some of the Xlib internals to close the gap between the information presented in the official documentation and what we can understand from reading it in order to be in a better position to build the foundation of our game based on Handmade Hero. As a result of doing this exercise we now have a better idea of what happens behind the scenes, we no longer perceive Xlib code to be a "black box" through which we can make a window visible in our desktop environment that a user can interact with. These are some of the most important achievements and things that we learned from going through this exercise: We have an initial idea of what the platform layer of the game (for graphics display) with Xlib might look like. In the process we learned that Xlib has a client-server architecture that enables multiple client applications to draw to the screen (a shared resource) concurrently; it is the server that processes and resolves those requests to update the screen's framebuffer. We also found out that in Xlib the display refers to the connection to the X Server which manages the screens (for graphics output) along with the peripherals such as the keyboard, mouse, or game console controller (for user input). We have realized that Xlib provides convenient macros for getting at the screen, visuals, etc. in a portable way. We have found that under the hood these macros cast the "opaque" display structure into a known type (private display) and subsequently dereferenced to obtain, for example, the root window ID, screen dimensions, or the black pixel value for the screen. These macros enable developers to change Xlib internals while not breaking the existing client code. Additionally, we learned that Xlib allocates the resources for the window on the server side and that the client code (our game) allocates the window resource Id (or handle). We used the GNU debugger gdb to demonstrate that client applications batch their requests into an output buffer. We also saw that Xlib commands that need synchronization such as XWindowEvent flush the output buffer and block until the server responds. To verify that our code has no memory leaks we used valgrind's memcheck tool. In the end we succeeded in making our game window visible and ready to put graphics on it. In this current world of software which consists of multiple layers of code that we cannot simply discard or choose not to use because they are integral to the code that we write, at least we can dive into it to write better software. For example, if you are a frontend engineer developing with Next.js maybe taking your coding to the next level means diving into React's implementation to find out what happens under the hood. Maybe by doing just that you will know enough about React to write better code and discover things about JavaScript that you did not know. And I mention this because by diving into Xlib's implementation I was able to see why just calling XCreateSimpleWindow will not just make the window visible and also learned some interesting things about C such as the comma-operator. The comma operator can be used for chaining multiple arithmetic operations into a single expression as shown in Xlib's internals header. It is not hard to imagine that this technique was used by Xlib developers to write maintainable code for abstracting away operations that are frequently used throughout the codebase. During my researching for writing this post I did not just learn how to write the X client code for the initial platform layer of my Handmade Hero game. By looking at pieces of the history of the development of Xlib from the commit logs I have truly developed a fondness for the Xlib project that I could not have obtained otherwise. I discovered the dedication of the developers that laid out the foundation for the desktop environments for Linux. The list of Xlib contributors is extensive but I would like to mention some notable ones. Keith Packard who has worked on the development of the X Windowing since the days of the XFree86 project and has continued doing so for Xlib until 2022. Alan Coopersmith has authored several contributions to Xlib for over 20 years and still maintains it. And Jamey Sharp modernized Xlib with the XCB transport layer and contributed the Xlib project from 2006 to 2011; it is also remarkable that during that same time he was also developing libxcb. But what it is even more impressive is that he kept reviewing code submitted to the Xlib project until 2017. It was interesting to see a commit log from Keith Packard mentioning that Jamey Sharp was looking over his shoulder while he was working on fixing an issue with the XCB transport layer (here is the link to the commit log). I am certain that there are many other outstanding contributors that made possible the Linux desktop experience that we enjoy that should also be credited but that would probably require many more posts like this one. I think that the Xlib developers deserve more credit than what they get, for the most common thing I have found are complaints about the X Windowing system. They modernized the internal implementation while keeping the API intact and still support older hardware. It is impressive to find out that some of the Xlib code that we considered in this post such as XCreateSimpleWindow() has only undergone minor changes in a period of over 20 years. This makes me reflect on how starkly different the world of software development must have been in the 90's when some of the veteran Xlib developers were already involved in implementing the X11 protocol (for the XFree86 project). I am assigning the readers (that have made it up to this point) the task of answering to themselves what of that craftsmanship evinced by these developers they need as a foundation to be outstanding software developers. Stay tuned for the next post. This section presents a list of the most relevant resources to learn about the X11 protocol and the C Language Interface libX11 (or commonly known as Xlib): Xlib - C Language Interface The official Xorg Technical Specification of the X11 Protocol with the C bindings. The X New Developer's Guide Provides a high-level documentation of the X11 Protocol that is tailored for new contributors. It provides information about modern extensions, the asynchronous X C Bindings (XCB), a fresher presentation of the original C Language Interface libX11, and contributing guidelines. Xlib Repository The official Xlib repository hosted in GitLab. This is the definitive Xlib documentation written in source code form. X11 Protocol Headers Repository The official GitLab repository hosting the X11 protocol headers. Some of the X11 protocol definitions were referenced from the standard X11 protocol headers. Tronche's Xlib - C Language Interface This is the legacy (release 6 of the) X11 protocol C Language Interface by Christophe Tronche. It is worth mentioning that Tronche brought the Xlib documentation to the web before the X Consortium and this is why it is still a relevant, highly indexed, and searchable resource. Handmade Network Tour through Xlib and related technologies A practical tour of Xlib (creating the window, handling events, and working with the framebuffer) to get you on the fast track for developing the Handmade Hero game in Linux written by Florian Behr. Xlib Learning Notes by Faison Zutavern. A series of study notes that covers topics of interest for developers just getting started with Xlib, such as opening a window, toggling fullscreen mode, and using OpenGL. This was the first GitHub repository that I found on Xlib when I was exploring the Quake engine. I am citing this work as a token of respect. Xlib Examples A repository of Xlib examples by Charles Qiu. This is another repository that I found useful when studying the Quake engine. I am sharing it as well as a token of respect. I would like to share some of the well known ports of Handmade Hero to Linux. I found out about them while working on this post. I am sharing them because you may also find them to be useful. First Linux port of Handmade Hero A series by David Gow on developing the platform layer for Handmade Penguin (a Linux port of Handmade Hero). David leverages the Simple Direct Media Layer (SDL) for handling graphics, sounds, and user input. This is useful for people that want to defer the hardships of implementing the platform layer with Xlib and ALSA. Multi-Platform Handmade Hero A comprehensive implementation of Handmade Hero on various platforms by Laszlo Korte. He has succeeded in porting the game to MacOS, iOS, Android, and Linux (both Wayland and X11). His work demonstrates that a well-designed game layer written in one platform can be easily ported to other platforms. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse