A Security Analysis of the Direct Rendering Infrastructure

Rickard E. Faith and Kevin E. Martin Precision Insight, Inc.

$Date: 1999/05/11 22:45:24 $, $Revision: 1.8 $


This paper examines a typical system running an implementation of the Direct Rendering Infrastructure (DRI) from the standpoint of security. The entities and access control present on the system are defined and potential security flaws are identified and discussed. Since the 3D direct-rendering client has access to low-level hardware, the implications of this access on the running X server and kernel are discussed, along with hardware features that could help make the system more secure. Further, this paper discusses software or hardware enhancements that may increase the stability or debuggability of the system without making the system more secure in a formal sense. Familiarity with the DRI design documents is assumed [OM98, MFOA99]

1. Preamble

1.1 Copyright

Copyright © 1998-1999 by Precision Insight, Inc., Cedar Park, Texas. All Rights Reserved.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

1.2 Trademarks

OpenGL is a registered trademark of Silicon Graphics, Inc. Unix is a registered trademark of The Open Group. The `X' device and X Window System are trademarks of The Open Group. XFree86 is a trademark of The XFree86 Project. Linux is a registered trademark of Linus Torvalds. Intel is a registered trademark of Intel Corporation. All other trademarks mentioned are the property of their respective owners.

2. Entities

The X server is a privileged process running as root with an elevated IOPL. In this paper, the X server and the kernel-level device driver are assumed to be secure, trusted entities. This paper is concerned with the other entities running on the system, as discussed below.

2.1 Objects

Objects are passive entities that contain or receive information. In this paper, objects are generally communications channels through which subjects communicate with each other and with the underlying hardware.

X Protocol Stream

The X protocol stream is a communications channel between the X server and the X client that is not specific to the DRI. The X protocol stream usually uses a Unix Domain Socket (i.e., a named pipe) for local clients and a INET Domain Socket (i.e., a TCP/IP socket) for non-local clients. This paper is primarily concerned with local clients, since non-local clients do not have direct access to the hardware or to the other objects listed below.

SAREA

The SAREA is a DRI-specific shared memory area that is used for communication between the X server, the 3D client, and the kernel.

Control Device

The control device, /dev/drm, is a kernel-level device that is used for general control of the graphics hardware and for setting up access to the graphics device.

Graphics Device

The graphics device (called ``sub-driver'' in related documents), /dev/drm0, is used to send DMA buffers to the graphics hardware and to obtain the hardware lock. If more than one graphics card is present in the system, there will be more than one graphics device (e.g., /dev/drm1, /dev/drm2, ...).

Hardware Lock

The hardware lock is a single lock for each graphics device that is used cooperatively to schedule direct access to the low-level graphics hardware (i.e., the frame buffer and the hardware registers).

Frame Buffer

The frame buffer is an area of memory that provides direct access to the video memory of the low-level graphics hardware.

Hardware Registers

The hardware registers are one or more memory-mapped regions that provide direct access to the programming registers for the low-level graphics hardware. Some hardware may separate the registers into non-privileged registers and privileged registers, which can be mapped as two different areas. In this case, the 3D client will only have access to the non-privileged registers, whereas the X server and kernel will also have access to the privileged registers.

2.2 Subjects

Subjects are active entities that can access or manipulate objects. In this paper, subjects are classes of executing programs that have various purposes. None of these programs are being run as root: this paper is only concerned with the security implications of running programs as regular users.

Non-X Programs

Non-X programs are programs that are running on the system without a connection to the X server (i.e., no X protocol stream).

2D Clients

2D clients are programs running on the system that are not doing direct rendering. These client require a regular connection to the X server (i.e., via the X protocol stream).

3D Clients

3D clients are programs running on the system that are doing direct rendering. In addition to a regular connection to the X server, these clients also require access to the SAREA, graphics device, hardware lock, frame buffer, and hardware registers.

3. Non-X Programs

3.1 X Protocol Stream

Access to an X protocol stream connection is controlled by standard X authentication mechanisms. If the non-X program somehow creates an X protocol stream connection, then it becomes a 2D client, and will not be discussed further in this section.

3.2 SAREA

If shared memory segments are implemented with the shm* system calls, a process can attach to the segment by using the segment's key, which is a 32-bit quantity. Since the SAREA data has to be shared between processes that have different user IDs, the segments must allow reads and writes by any user. Hence any non-X program could attach to the SAREA by guessing the value of the key. This situation must be prevented.

If authorization to use the SAREA is controlled by the graphics device (i.e., via process ID and memory mapping), then access to the SAREA will require access to the graphics device. This will prevent non-X programs from attaching to the shared memory segments and corrupting the data.

3.3 Control Device

The control device can be opened by any user ID, but non-root users are limited to a small set of functions. This limited set of functions cannot be used to create a graphics device or to allocate or destroy resources. Further, a request for exclusive access to the control device is not permitted, so a non-X program cannot create a denial of service by holding exclusive access to the control device.

3.4 Graphics Device

Access to the graphics device is limited to those processes that have registered as 3D clients with the X server via the X protocol stream. Further, access is limited to a specific Unix group so that system administrators can restrict direct-rendering access to a list of trusted users. A client that is not in this group cannot open the graphics device. A client that is in this group can open the graphics device, but cannot use any of its functionality without requesting authentication via the X protocol stream.

3.5 Hardware Lock

Access to the hardware lock requires access to the SAREA or the graphics device.

3.6 Frame Buffer

The frame buffer is mapped via the graphics device, so access to the frame buffer requires access to the graphics device. Non-root programs do not have sufficient access rights to map the frame buffer without assistance from a privileged entity.

3.7 Hardware Registers

Same as frame buffer.

4. 2D Clients

4.1 X Protocol Stream

A 2D client will have complete access to the X protocol stream, authenticated using standard X authentication (e.g., xauth). If the 2D client uses this access to gain access to the graphics device, it will be considered to be a 3D client, and will not be discussed further in this section.

The addition of the DRI does not make the X server more secure: all of the common 2D client exploits and denials of service can still proceed, just like they can in an unmodified X server. A 2D client can deny service by doing a server grab, can read or write to the windows of other clients, and can read keystrokes from the keyboard (even if another client has done a server grab). When analyzing the security impact of the DRI, the reader should keep in mind that X lacks any notion of privileges and that all clients are treated equally and are capable of performing any X function [EP91[.

4.2 SAREA

Same as for non-X program.

4.3 Control Device

Same as for non-X program.

4.4 Graphics Device

Same as for non-X program.

4.5 Hardware Lock

Same as for non-X program.

4.6 Frame Buffer

Same as for non-X program.

4.7 Hardware Registers

Same as for non-X program.

5. 3D Clients

5.1 X Protocol Stream

A 3D client will have complete access to the X protocol stream, authenticated using standard X authentication (e.g., xauth). The 3D client will use this access to gain access to the graphics device and, via this device, to the SAREA, hardware lock, frame buffer, and hardware registers.

5.2 SAREA

The 3D client will have complete access to the SAREA. The well-behaved client will access the SAREA via calls to the driver libraries: the application programmer will never have to access this area directly. However, there is nothing to prevent a malicious (or buggy) client from overwriting portions of the SAREA that are in use by other 3D clients.

5.3 Control Device

Same as for non-X program.

5.4 Graphics Device

The graphics device can only be opened by a process that currently has an open X protocol stream connection -- if the user is using xauth authentication, for example, then all 3D clients will have to be authenticated. This is a requirement so that basic X security is not compromised by the DRI.

The following algorithm [Plumb99] is used to authenticate access to the graphics device:

  1. The client opens the graphics device. This connection is unauthenticated and none of the graphics device functionality is available for use by the client, with the exception of the ``magic number'' ioctl.
  2. The client uses the ``magic number'' ioctl to obtain a magic number from the kernel.
  3. The client sends the magic number to the X server, via the X protocol stream.
  4. The X server requests that the kernel authenticate the currently open-but-unauthenticated connection associated with that magic number.
  5. The client can now use the full functionality provided by the graphics device.

On a fork or thread create, the file descriptor for the graphics device is duplicated, and the child process/thread is allowed to continue direct rendering using this file descriptor. This is reasonable, since the new (light- or heavy-weight) process shares a lot of intimate knowledge with the parent. This is a requirement to ease the implementation of threaded applications.

On an exec, the file descriptor for the graphics device is closed. An exec starts a completely different process that will have to do its own X authentication. When the graphics device is first opened, the equivalent of fcntl(...,F_SETFD) is performed to set close-on-exec flag. However, a malicious client could reset this bit, and allow the file descriptor to remain open across the exec. In this case, the parent is cooperating with the new process and could just as easily perform an attack without creating a new process. Hence, this is not considered an additional security risk.

The DRI policy for preserving graphics device access across forks and execs should be made well-known to programmers and device driver implementors.

5.5 Hardware Lock

Denial of Service

On low-end graphics hardware, there is one hardware lock (for each graphics card) that is shared, cooperatively, between all the 3D clients, the X server, and the kernel. Well-behaved clients will access this lock via well-known macros or calls to the driver library: an application programmer will never have to make use of the lock directly. A malicious client could grab this lock and hold it indefinitely, thereby denying access to the hardware by all the 3D clients, the X server, and the kernel (in much the same was that a 2D client could do a server grab for an indefinitely long period of time).

Further, a malicious 3D client could perform direct hardware accesses without first getting the lock. This has the potential, especially on low-end hardware, of causing the hardware to fail such that it is unusable until after the next power cycle (low-end hardware often lacks robust software reset capabilities). This situation could be prevented by unmapping the frame buffer and hardware registers when the 3D client does not have the lock. However, this would require that lock acquisition always uses an ioctl, thereby making an efficient two-tier lock implementation impossible. Further, the benefits are small: a malicious 3D client could also get the lock and then perform operations that would lockup the hardware. Even when debugging a well-behaved client, the benefits are minimal, since the vast majority of (potentially buggy) hardware accesses will be taking place while the lock is held.

A well-behaved 3D client could receive a SIGSTOP signal while holding the lock, thereby causing an inadvertent denial of service. This situation must be prevented in the kernel driver. If the client always acquires the lock via an ioctl, then the kernel could mask this signal. However, when using a two-tiered locking algorithm, the kernel driver must watch for all SIGSTOP signals, and mask the signal when it is going to a process that currently holds the lock.

A well-behaved 3D client could also receive a SIGKILL (or other terminating) signal while holding the lock. In this case, the graphics device will be closed during _exit(2) processing, and the kernel will release the lock. For this mechanism to work, it is imperative that the process holding the lock has the process ID of the process that requested the lock (i.e., the lock cannot be shared across a fork -- see below for details).

Forks and Execs

On a fork or thread create, the child process should assume that it does not hold the lock, and should acquire the lock when necessary: the child cannot free a lock that was obtained with its parent's process ID. This leads to an important matter of policy that driver implementors should understand: the driver library is not allowed to fork or to create threads while a lock is held. Handling of locks across forks much be done via strict policy, since the kernel-level driver cannot detect a fork by a process that holds the lock without instrumenting the kernel in ways that will degrade overall kernel performance.

If a lock is held on exec, the kernel will free the lock when the parent process closes the graphics device during _exit(2) processing. This assumes that the close-on-exec flag is set for the graphics device file descriptor.

5.6 Frame Buffer

All 3D clients have a mapping, obtained via the graphics device, of the frame buffer and must access the frame buffer cooperatively using the hardware lock. This mapping must be preserved across forks and thread creation.

5.7 Hardware Registers

All 3D clients have a mapping, obtained via the graphics device, of the hardware registers and must access these cooperatively using the hardware lock. This mapping must be preserved across forks and thread creation.

Depending on the robustness of the hardware implementation, an uncooperative client could place the hardware in a state which requires a power-cycle to reset the hardware. The impact of this exploit is minimized by requiring that all 3D clients are authenticated by the X server and by abstracting direct hardware access via the device driver. We expect this to be a problem for the device driver implementor, but not for the end-user, even when debugging a complex graphics application.

On some low-end hardware, there is no mechanism to separate privileged registers from non-privileged registers (e.g., both kinds of registers are present on the same page of mappable address space). If both kinds of registers are mapped into user-address space, a grave security risk is created: the potential would exists for the 3D client to initiate, independently of the kernel driver, DMA to arbitrary physical addresses. Assuming that DMA could only be performed to or from graphics hardware memory, a malicious client could DMA from kernel memory to the frame buffer on the graphics hardware, modify memory, and then DMA from the frame buffer back to the kernel memory. This would allow a malicious client to read and write any physical address, and would provide the ability to obtain root access or otherwise compromise the security of the whole system.

Driver implementors must be made aware of this potential risk and of the solutions provided by some hardware manufacturers to avoid the risk. Drivers for cards that do not provide a separate area for privileged registers should not map any registers into 3D client space, although the 3D client should still be able to use kernel-mediated DMA access to the graphics card as well as direct access to the frame buffer. Any register access that cannot be performed via DMA must be performed via a device-specific ioctl. Fortunately, our experience is that the 3D client performs the vast majority of performance-critical operations via DMA and frame-buffer access. Hence, special-casing a few instances where the 3D client requires register access is not expected to degrade performance.

6. References

[EP91] Jeremy Epstein and Jeffrey Picciotto. Trusting X: Issues in Building Trusted X Window Systems -or- What's not Trusted About X? In Proceedings of the 14th Annual National Computer Security Conference, Washington, DC, October 1991.

[MFOA99] Kevin E. Martin, Rickard E. Faith, Jens Owen, Allen Akin. Direct Rendering Infrastructure, Low-Level Design Document. Cedar Park, Texas: Precision Insight, Inc., 1999.

[OM98] Jens Owen and Kevin E. Martin. A Multipipe Direct Rendering Architecture for 3D. Cedar Park, Texas: Precision Insight, Inc., 15 September 1998. Available from http://www.precisioninsight.com/dr/dr.html.

[Plumb99] Colin Plumb, personal communication, 20 March 1999.