Picking and Hit-Testing in Metal

In this article, we’ll see how to interact with our 3D scenes using a technique called picking. Picking helps us answer the question “What did the user just tap on?” More precisely, picking is the process of determining which object or objects in our scene are being indicated by an interaction like a mouse click or a touch on the screen. Picking is frequently used in 3D modeling and CAD programs to select objects for further manipulation, but many applications eventually require the user to interact with the scene, and picking is an essential tool for enabling such interactions.

Picking is mostly synonymous with hit-testing, which is the more commonly-used term when discussing 2D graphical user interfaces. In fact, you may already be acquainted with UIKit’s model for hit-testing. For the purposes of this article, picking and hit-testing are identical.

You can view and download the sample code for this article here.

A demonstration of selecting objects with picking
A demonstration of selecting objects with picking

Read morePicking and Hit-Testing in Metal

Rendering 3D Text with Core Text and libtess2

In this article, we’ll discuss how to create and render 3D text with Metal. Most applications need to render text, and there are many techniques for drawing 2D text with graphics APIs, from pre-rasterized font atlases, to signed-distance field methods like Chris Green’s seminal work at Valve or GLyphy, to cutting-edge vector-based solutions like Slug. Most of these techniques generalize to positioning 2D text in 3D environments as well. Some even allow text to conform to the contours of 3D objects.

The technique discussed here is not intended for general use in graphical user interfaces (GUIs), where you often see 2D text. Rather, this article is about how to create an extruded 3D mesh that represents a string of text. This kind of mesh has many uses, such as labeling objects in virtual/augmented reality or educational applications. Any time you need a piece of text to have some heft to it, you might consider using extruded text.

An example of the 3D text you can create with the technique described in this article.

You can download the sample code for this article here.

Read moreRendering 3D Text with Core Text and libtess2

Writing a Modern Metal App from Scratch: Part 2

In the previous article, we wrote enough Metal code to get the spinning silhouette of a teapot on the screen, but that still leaves a lot to be desired as far as a “modern” app is concerned. In this article, we’ll further flesh out the app and introduce lighting, materials, texturing, and managing multiple objects with scene graphs.

Remember that you can clone this Github repository to follow along with the sample code.

Read moreWriting a Modern Metal App from Scratch: Part 2

Compressed Texture Formats in Metal

In this article, we will consider several GPU-friendly compressed texture formats. These formats allow us to trade some image quality for substantial improvements in disk usage and performance. In particular, we will look at the ETC2, PVRTC, and ASTC formats.

The sample app showcases a variety of compressed texture formats
The sample app showcases a variety of compressed texture formats

Read moreCompressed Texture Formats in Metal