In the early 1980s, Bob Scheifler of the Department of Computer Science at the Massachusetts Institute of Technology (MIT) set out to formulate rules for a new windowing system. He decided to name it X because the window system was an improvement on the W window system, which was derived from the V system. The X Window System will revolutionize the graphics field. X did become the standard graphics server for all UNIX-like windowing systems at the time, because its features and philosophies far surpassed those of the competition. In just a few years, the UNIX community has fully embraced the X Window System.
In this article, we'll take a look at the evolution of the Linux graphics stack, from the original X client/server to the current Wayland effort.
X has become so special, it has to be said that it is a legend. X is the first web-oriented, distributed-based graphics server. On a time-sharing system, an X server can serve multiple clients at the same time. X's network protocol enables local windows to be displayed on remote machines. In fact, X is network-oriented, and the protocol is independent of hardware. By sending a display request through a network protocol, an X client running on a certain UNIX-like platform can be displayed on a UNIX-like system with a completely different hardware platform.
The window appearance is independent of the X server. The X protocol defines the display device and window primitive information, and the specific appearance of the window is only related to the control library, window manager and desktop environment.
Under the leadership of Bob Scheifler and the management of MIT, more people began to be interested in it. For the better development of X in the future, the leading companies at that time, such as Digital Electronics Corporation (DEC) in the United States, can obtain the source code for free. At the same time, a group of vendors asked MIT if there was some way to maintain the integrity of the source code so that X could benefit all interested organizations. MIT agreed, and soon after, the MIT X Association was formed, and the source code was released, including DEC's improvements to X. The release of the X source code is indeed an event worthy of attention. Manufacturers realize that X already has high value. It is necessary to prevent a company from gaining control of X to better protect everyone's efforts. Maintaining the open source of X is the most important principle. At the same time, the MIT X Association has always Owning the copyright to X source code avoids X being owned by private commercial companies.
In 1988, the association recruited a senior developer, Keith Packard, who would reimplement the core of the X server. Now, Packard is well known in the Linux graphics stack world.
Despite X's dominance in the UNIX and Linux graphics stack arena, due to its popularity, X-based software that generates a lot of draw requests and a lot of software eventually weighed down X itself. In the 1990s, Linux quickly became popular. For X Server/client running on the same machine, X configuration files began to be used, and X began to be bundled with many Linux distributions. The X's web-based design is useless for local displays, a once vaunted advantage that is now a bottleneck for video rendering.
During this time, sales of personal computers surged, the complex nature of professional graphics cards began to surpass the processing power of the X, and more high-performance graphics cards continued to be developed.
The advent of Translation Table Mapping (hereafter referred to as TTM)
Around 2004, some Linux developers began to grow frustrated with the slow development of X. In 1992, they moved to OpenGL, an image rendering API originally used to display 2D and 3D graphics (developed by Silicon Valley Graphics). After several years of X's attempts to draw 3D graphics on graphics cards, none of them worked.
Then in 2007, Thomas Hellstrom, Eric Anholt, and Dave Airlie developed a memory management module they called TTM. TTM is used for buffered copies between video memory and system memory. There was a lot of buzz in the Linux community at the time. It holds promise for graphics applications that have stringent 3D performance requirements. This method is to pass the memory buffer as the first class object to the application, while allowing the application to allocate and process the graphics information in the memory buffer. The TTM manages the buffers of all applications on the host and provides synchronization between the GPU and the CPU through a special signal fence. When the GPU completes the display processing of a buffer, it passes the control of the buffer to the application corresponding to the buffer through the fence signal, and the CPU takes over the buffer again.
In all fairness, the standardization of how applications access the GPU, TTM does make a brave attempt, it is the memory manager on Linux for all graphics card drivers, in short, TTM tries to provide all operations that all graphics applications may need. Unfortunately, the TTM code size and API are too large, and each individual open source driver only needs a small subset of the API set. The sheer number of APIs can easily confuse developers, who need to decide which API to choose to use. The biggest complaint is that TTM still has some performance problems, which may be caused by the signal synchronization mechanism (fence) between the CPU and GPU, or the copy efficiency of the buffer object is too low. The TTM probably does have a lot to deal with, but that's no excuse for it being slow.
In 2008, Reenter Keith Packard announced an alternative to the TTM. Keith is still working at Intel so far. Thanks to Eric's lessons in developing the TTM, and with Eric's help, they rewrote the TTM. The new API is called Graphics Execution Manager (hereinafter referred to as GEM). Generally, developers can roughly guess what to do next through the API. Compared with the huge API of TTM, developers will not get lost in the API of GEM.
Compared with TTM, GEM has many improvements. One of the most significant improvements is that the design of API interface is very rigorous, and the concept of fence signal that is troublesome in TTM has also been removed. Keith and Eric let the application take over the part of the memory on top of the API, while the GEM concentrates on the part of the memory accessible to the GPU and handles the context information for the graphics card execution. The interactive access between the application and the kernel space is through the ioctl() call, and there is no constant buffer copy like TTM. GEM is more of a stream-oriented API than a memory manager.
GEM allows applications to share memory buffers, and the memory space processed by the GPU does not need to be reloaded. Here are the original release notes:
"Gem provides a simple mechanism for the Linux operating system to manage graphics data and control the flow of execution. Many kernel subsystems implement this mechanism."
随着GEM在2008年5月的出现，给Linux的图形栈带来了更多的希望。GEM并不处理所有事情。例如，GPU执行的命令由设备对应驱动提供。因为Keith和Eric在Intel工作，GEM对Intel驱动支持的开源代码编写工作理所当然落在了他们身上。同时希望GEM能够支持更多其他的驱动，例如三个最大的显卡制造商。
然而，采用GEM的非intel显卡驱动很慢。一些迹象表明AMD驱动选择了一个“类似GEM和TTM管理器”的方案，预示着AMD并不愿意加入到GEM的阵营，GEM面临单枪匹马的危险。
TTM和GEM尝试在X中整合对GPU的操作以解决Linux图形栈上的3D加速问题。两者都是为了更好的提升不同图形库的显示性能，例如OpenGL(依赖于X), Qt(依赖于X)和GTK+（也是依赖于X)。问题是X处于所有图形库和内核之间，只有内核能够访问设备驱动，然后驱动才能访问到GPU。
X已经老掉牙了。X拥有数百万行代码，很多代码已经很陈旧，那是还没有GPU，没有专业的晶体管处理可编程着色技术或者旋转操作以及顶点翻转，既不知道高密度采样和用插值法减少反锯齿效果的概念，也不清楚如何创建极高精度的颜色空间。此时不同往日，X已经不能满足现在的需求了。
Wayland:一个新的显示管理器
2008年，一个叫Kristian H?gsberg 的软件工程师在波士顿的市郊上开着车，可能是去往工作的路上，亦或是回家，软件工程师正深入的思考着，有时他们会花费大量时间解决复杂的问题，将问题打散并进行重构。当他们工作疲劳休息时，会产生一些灵感，这种灵感往往能够为项目带来更大的突破。当人们在洗澡或者在厨房烹饪时，或者在驾驶途中，这种灵感更易产生。 当H?gsberg驱车于马萨诸塞州Wayland一个很小的村庄时，他的大脑中迸出一个想法并渐渐成型，这个想法用他自己的话说：
“核心思想是所有窗口被重定向，我们可以让所有客户端自己去渲染自己，然后传递一个缓冲句柄给显示服务器，显示服务器中的混合器完成窗口显示工作。其中一个目标就是让X服务器运行于Wayland上，首先支持全屏窗口（例如Xnest)，然后支持父窗口等，直到X被最终淘汰。”
他的想法是写一个全新的显示管理器，管理器可绕过X直接向内核发送3D显示请求。X自己就是显示管理器的一个客户端，正如以上作者所说，新的显示管理器取名为“Wayland”，以此表达对这个小镇的敬意。
这仅仅只是一个想法。很多人每天都会产生大量聪明的想法，但最终都被生活中的杂事所磨灭。 H?gsberg正从事于图形渲染库方面的工作，他大概觉得，如果应用程序都能直接访问GPU，没有X在中间搅局，那一切事情都变得简单了。他着手写代码解决这个问题。根据Intel的员工Keith Packard所说，两周后，H?gsber完成了雏形。
Wayland的一个关键特点是完成渲染工作的API完全不依赖X。为了保持兼容性，X服务器本身就是Wayland的客户端,所有X所做的渲染直接发给Wayland。像X一样，Wayland仅仅定义协议。Wayland的体系结构和X本身的特点，给X客户端的移植带来了很大便利，同时还兼容以前的X客户端程序。
Wayland显示管理器会使用内核中的GEM(图形执行管理器)、evdev(输入事件驱动)和kms(内核模式切换）。Wayland有自己的混合器，这与X形成鲜明的对比，X依赖外部的混合器处理显示相关的内存缓冲。
Wayland也会使用DRI2。Wayland混合器和Wayland客户端都能够访问屏幕显示相关的资源。当客户端更新了这个缓冲，混合器将更新桌面，并重绘屏幕。
Wayland确实解决了很多X无法解决的问题，大家对此项目感到兴奋，一群聪明的人正在实现它，并且Intel和redhat也在支持这个项目。
但是还有一些障碍需要克服。两个最大的障碍就是开发兼容NVIDIA和AMD的开源驱动。第三大显卡厂商Intel已经兼容了Wayland，因为GEM内核模块就是基于Intel驱动开发的。
谁会为Wayland更新AMD和NVIDIA的开源驱动？对开发者来说，Linux下开源显卡驱动的开发很折磨人，因为你没有完整的硬件规范手册，甚至什么资料都没有，只能通过枯燥的逆向工程获取显卡规范信息。
nouveau驱动就是一个很好的例子。当时，NVIDIA开发者放出话来，表示我们没有支持Wayland的计划。所有工作压向Linux社区，有时有一些厂商的支持，有时什么都没有。在nouveau项目中，开发者正通过逆向工程积极的开发NVIDIA驱动。一个叫做Renouveau(nouveau的逆向工程）的执行流程如下：
1. 记录MMIO寄存器的内容；
2. 绘制图形；
3. 记录寄存器中的新值；
然后向Renouveau工程的ftp服务器发送绘制前后内存转储的文本格式差异信息，这些文件是为以后的显卡分析做准备。
相对于NVIDIA，AMD这方面的工作进展更顺利。过去几年中，新成立的团队正在开发AMD的开源驱动，根据开发团队周期性发布的规范说明，各地开发者皆可持续驱动的研发。驱动名称是fglx(FireGL and Radeon for X）,Linux社区每月都可以从AMD获得显卡的最新信息。
Wayland在Linux图形栈方面很有优势。最近，Ubuntu计划在Wayland上使用他们自己的窗口管理器Unity。为了确保硬件驱动能在新的图形架构上发挥最大性能，Intel强力支持协助开发GEM，雇佣了Wayland的开发者们。
除此以外，图形领域正在两极分化。AMD和NVIDIA正忙于争夺市场份额的激烈竞争，开源社区的开发需求还排不上他们的议程。
Linux社区是基于协作来完成开发。多年以来，通过合作和开放，Linux急速发展，而Linux图形栈似乎不为所动。图形硬件厂商可选择是否向开源社区贡献力量，但很多厂商持不情愿开源的态度。那些拥有高性能图形显卡的厂商难道不想让他们的用户拥有最好的用户体验？将产品信息隐藏起来，这些难道不会伤及他们的产品线？可以毫无疑问的说-他们违反了开源精神。
In this article, we'll take a look at the evolution of the Linux graphics stack, from the original X client/server to the current Wayland effort.
X has become so special, it has to be said that it is a legend. X is the first web-oriented, distributed-based graphics server. On a time-sharing system, an X server can serve multiple clients at the same time. X's network protocol enables local windows to be displayed on remote machines. In fact, X is network-oriented, and the protocol is independent of hardware. By sending a display request through a network protocol, an X client running on a certain UNIX-like platform can be displayed on a UNIX-like system with a completely different hardware platform.
The window appearance is independent of the X server. The X protocol defines the display device and window primitive information, and the specific appearance of the window is only related to the control library, window manager and desktop environment.
Under the leadership of Bob Scheifler and the management of MIT, more people began to be interested in it. For the better development of X in the future, the leading companies at that time, such as Digital Electronics Corporation (DEC) in the United States, can obtain the source code for free. At the same time, a group of vendors asked MIT if there was some way to maintain the integrity of the source code so that X could benefit all interested organizations. MIT agreed, and soon after, the MIT X Association was formed, and the source code was released, including DEC's improvements to X. The release of the X source code is indeed an event worthy of attention. Manufacturers realize that X already has high value. It is necessary to prevent a company from gaining control of X to better protect everyone's efforts. Maintaining the open source of X is the most important principle. At the same time, the MIT X Association has always Owning the copyright to X source code avoids X being owned by private commercial companies.
In 1988, the association recruited a senior developer, Keith Packard, who would reimplement the core of the X server. Now, Packard is well known in the Linux graphics stack world.
Despite X's dominance in the UNIX and Linux graphics stack arena, due to its popularity, X-based software that generates a lot of draw requests and a lot of software eventually weighed down X itself. In the 1990s, Linux quickly became popular. For X Server/client running on the same machine, X configuration files began to be used, and X began to be bundled with many Linux distributions. The X's web-based design is useless for local displays, a once vaunted advantage that is now a bottleneck for video rendering.
During this time, sales of personal computers surged, the complex nature of professional graphics cards began to surpass the processing power of the X, and more high-performance graphics cards continued to be developed.
The advent of Translation Table Mapping (hereafter referred to as TTM)
Around 2004, some Linux developers began to grow frustrated with the slow development of X. In 1992, they moved to OpenGL, an image rendering API originally used to display 2D and 3D graphics (developed by Silicon Valley Graphics). After several years of X's attempts to draw 3D graphics on graphics cards, none of them worked.
Then in 2007, Thomas Hellstrom, Eric Anholt, and Dave Airlie developed a memory management module they called TTM. TTM is used for buffered copies between video memory and system memory. There was a lot of buzz in the Linux community at the time. It holds promise for graphics applications that have stringent 3D performance requirements. This method is to pass the memory buffer as the first class object to the application, while allowing the application to allocate and process the graphics information in the memory buffer. The TTM manages the buffers of all applications on the host and provides synchronization between the GPU and the CPU through a special signal fence. When the GPU completes the display processing of a buffer, it passes the control of the buffer to the application corresponding to the buffer through the fence signal, and the CPU takes over the buffer again.
In all fairness, the standardization of how applications access the GPU, TTM does make a brave attempt, it is the memory manager on Linux for all graphics card drivers, in short, TTM tries to provide all operations that all graphics applications may need. Unfortunately, the TTM code size and API are too large, and each individual open source driver only needs a small subset of the API set. The sheer number of APIs can easily confuse developers, who need to decide which API to choose to use. The biggest complaint is that TTM still has some performance problems, which may be caused by the signal synchronization mechanism (fence) between the CPU and GPU, or the copy efficiency of the buffer object is too low. The TTM probably does have a lot to deal with, but that's no excuse for it being slow.
In 2008, Reenter Keith Packard announced an alternative to the TTM. Keith is still working at Intel so far. Thanks to Eric's lessons in developing the TTM, and with Eric's help, they rewrote the TTM. The new API is called Graphics Execution Manager (hereinafter referred to as GEM). Generally, developers can roughly guess what to do next through the API. Compared with the huge API of TTM, developers will not get lost in the API of GEM.
Compared with TTM, GEM has many improvements. One of the most significant improvements is that the design of API interface is very rigorous, and the concept of fence signal that is troublesome in TTM has also been removed. Keith and Eric let the application take over the part of the memory on top of the API, while the GEM concentrates on the part of the memory accessible to the GPU and handles the context information for the graphics card execution. The interactive access between the application and the kernel space is through the ioctl() call, and there is no constant buffer copy like TTM. GEM is more of a stream-oriented API than a memory manager.
GEM allows applications to share memory buffers, and the memory space processed by the GPU does not need to be reloaded. Here are the original release notes:
"Gem provides a simple mechanism for the Linux operating system to manage graphics data and control the flow of execution. Many kernel subsystems implement this mechanism."
随着GEM在2008年5月的出现，给Linux的图形栈带来了更多的希望。GEM并不处理所有事情。例如，GPU执行的命令由设备对应驱动提供。因为Keith和Eric在Intel工作，GEM对Intel驱动支持的开源代码编写工作理所当然落在了他们身上。同时希望GEM能够支持更多其他的驱动，例如三个最大的显卡制造商。
然而，采用GEM的非intel显卡驱动很慢。一些迹象表明AMD驱动选择了一个“类似GEM和TTM管理器”的方案，预示着AMD并不愿意加入到GEM的阵营，GEM面临单枪匹马的危险。
TTM和GEM尝试在X中整合对GPU的操作以解决Linux图形栈上的3D加速问题。两者都是为了更好的提升不同图形库的显示性能，例如OpenGL(依赖于X), Qt(依赖于X)和GTK+（也是依赖于X)。问题是X处于所有图形库和内核之间，只有内核能够访问设备驱动，然后驱动才能访问到GPU。
X已经老掉牙了。X拥有数百万行代码，很多代码已经很陈旧，那是还没有GPU，没有专业的晶体管处理可编程着色技术或者旋转操作以及顶点翻转，既不知道高密度采样和用插值法减少反锯齿效果的概念，也不清楚如何创建极高精度的颜色空间。此时不同往日，X已经不能满足现在的需求了。
Wayland:一个新的显示管理器
2008年，一个叫Kristian H?gsberg 的软件工程师在波士顿的市郊上开着车，可能是去往工作的路上，亦或是回家，软件工程师正深入的思考着，有时他们会花费大量时间解决复杂的问题，将问题打散并进行重构。当他们工作疲劳休息时，会产生一些灵感，这种灵感往往能够为项目带来更大的突破。当人们在洗澡或者在厨房烹饪时，或者在驾驶途中，这种灵感更易产生。 当H?gsberg驱车于马萨诸塞州Wayland一个很小的村庄时，他的大脑中迸出一个想法并渐渐成型，这个想法用他自己的话说：
“核心思想是所有窗口被重定向，我们可以让所有客户端自己去渲染自己，然后传递一个缓冲句柄给显示服务器，显示服务器中的混合器完成窗口显示工作。其中一个目标就是让X服务器运行于Wayland上，首先支持全屏窗口（例如Xnest)，然后支持父窗口等，直到X被最终淘汰。”
他的想法是写一个全新的显示管理器，管理器可绕过X直接向内核发送3D显示请求。X自己就是显示管理器的一个客户端，正如以上作者所说，新的显示管理器取名为“Wayland”，以此表达对这个小镇的敬意。
这仅仅只是一个想法。很多人每天都会产生大量聪明的想法，但最终都被生活中的杂事所磨灭。 H?gsberg正从事于图形渲染库方面的工作，他大概觉得，如果应用程序都能直接访问GPU，没有X在中间搅局，那一切事情都变得简单了。他着手写代码解决这个问题。根据Intel的员工Keith Packard所说，两周后，H?gsber完成了雏形。
Wayland的一个关键特点是完成渲染工作的API完全不依赖X。为了保持兼容性，X服务器本身就是Wayland的客户端,所有X所做的渲染直接发给Wayland。像X一样，Wayland仅仅定义协议。Wayland的体系结构和X本身的特点，给X客户端的移植带来了很大便利，同时还兼容以前的X客户端程序。
Wayland显示管理器会使用内核中的GEM(图形执行管理器)、evdev(输入事件驱动)和kms(内核模式切换）。Wayland有自己的混合器，这与X形成鲜明的对比，X依赖外部的混合器处理显示相关的内存缓冲。
Wayland也会使用DRI2。Wayland混合器和Wayland客户端都能够访问屏幕显示相关的资源。当客户端更新了这个缓冲，混合器将更新桌面，并重绘屏幕。
Wayland确实解决了很多X无法解决的问题，大家对此项目感到兴奋，一群聪明的人正在实现它，并且Intel和redhat也在支持这个项目。
但是还有一些障碍需要克服。两个最大的障碍就是开发兼容NVIDIA和AMD的开源驱动。第三大显卡厂商Intel已经兼容了Wayland，因为GEM内核模块就是基于Intel驱动开发的。
谁会为Wayland更新AMD和NVIDIA的开源驱动？对开发者来说，Linux下开源显卡驱动的开发很折磨人，因为你没有完整的硬件规范手册，甚至什么资料都没有，只能通过枯燥的逆向工程获取显卡规范信息。
nouveau驱动就是一个很好的例子。当时，NVIDIA开发者放出话来，表示我们没有支持Wayland的计划。所有工作压向Linux社区，有时有一些厂商的支持，有时什么都没有。在nouveau项目中，开发者正通过逆向工程积极的开发NVIDIA驱动。一个叫做Renouveau(nouveau的逆向工程）的执行流程如下：
1. 记录MMIO寄存器的内容；
2. 绘制图形；
3. 记录寄存器中的新值；
然后向Renouveau工程的ftp服务器发送绘制前后内存转储的文本格式差异信息，这些文件是为以后的显卡分析做准备。
相对于NVIDIA，AMD这方面的工作进展更顺利。过去几年中，新成立的团队正在开发AMD的开源驱动，根据开发团队周期性发布的规范说明，各地开发者皆可持续驱动的研发。驱动名称是fglx(FireGL and Radeon for X）,Linux社区每月都可以从AMD获得显卡的最新信息。
Wayland在Linux图形栈方面很有优势。最近，Ubuntu计划在Wayland上使用他们自己的窗口管理器Unity。为了确保硬件驱动能在新的图形架构上发挥最大性能，Intel强力支持协助开发GEM，雇佣了Wayland的开发者们。
除此以外，图形领域正在两极分化。AMD和NVIDIA正忙于争夺市场份额的激烈竞争，开源社区的开发需求还排不上他们的议程。
Linux社区是基于协作来完成开发。多年以来，通过合作和开放，Linux急速发展，而Linux图形栈似乎不为所动。图形硬件厂商可选择是否向开源社区贡献力量，但很多厂商持不情愿开源的态度。那些拥有高性能图形显卡的厂商难道不想让他们的用户拥有最好的用户体验？将产品信息隐藏起来，这些难道不会伤及他们的产品线？可以毫无疑问的说-他们违反了开源精神。