What’s New in Metal (2019)

With WWDC 2019 over, it’s a good time to reflect on everything that was announced and start seeing how we can put it into practice. The purpose of this article is to round up the changes to the Metal framework and tools and provide pointers to where you can learn more. This was a banner year, with significant improvements to Metal itself; major advancements in related frameworks like MetalPerformanceShaders (MPS), CoreML, and ARKit; and the exciting introduction of an all-new augmented reality engine, RealityKit.

Since this was such a big year, this post can’t hope to cover all the improvements that are included in macOS 10.15 Catalina and iOS 13. In particular, raytracing with MPS was a hot topic, but you’ll have to look elsewhere for coverage. This session should get you up-to-speed.

Read moreWhat’s New in Metal (2019)

Rendering Physically-Based ModelIO Materials

In this article, we’ll take a look at a portion of the ModelIO framework we haven’t used so far: materials. In previous articles, we’ve used ModelIO to do the heavy lifting of loading 3D geometry, but have loaded and applied textures manually. This ad hoc approach works when we only have a base color map, but quickly grows tedious as materials become more complicated. Here, we’ll take a deep dive into ModelIO materials and learn how to use them to perform basic physically-based rendering (PBR).

You can find the sample code for this article here.

This is not a tutorial on physically-based rendering. PBR has been a hot topic for the past several years, and other people have written excellent tutorials on the subject. Joey de Vries’ series is probably the most approachable. Also consider taking a look at the documentation for Google’s brand-new mobile-first rendering engine Filament. For more theoretical background, consult the following papers and resources:

The focus here is on how ModelIO represents materials, deemphasizing how a shader might render them. For this reason, the shaders for this article are a pared-down version of Apple’s LOD with Function Specialization sample, which is worth studying.

An example asset rendered by SceneKit, from the 2016 WWDC Session, Advances in SceneKit Rendering (https://developer.apple.com/videos/play/wwdc2016/609/)

Read moreRendering Physically-Based ModelIO Materials

Translucency and Transparency in Metal


One of the topics we’ve handily avoided so far in our exploration of Metal is rendering of materials that are not opaque. In this post, we’ll explore a couple of related techniques for achieving transparency and translucency: alpha testing and alpha blending.

The sample scene for this post is a desert containing many palm trees and a few pools of water. The leaves of the palm trees consist of a few polygons each, textured with a partially-transparent texture, and the water is rendered as a translucent surface via alpha blending, which we’ll discuss in detail below.

The sample application from this article demonstrates alpha testing and alpha blending
The sample application from this article demonstrates alpha testing and alpha blending

Read moreTranslucency and Transparency in Metal

Instanced Rendering in Metal

In this article, we will discuss an important technique for efficiently drawing many objects with a single draw call: instanced rendering. This technique helps you get the most out of the GPU while keeping memory and CPU usage to a minimum.

The sample app for this post renders several dozen animated cows moving on top of a randomly-generated terrain patch. Each cow has its own position, orientation, and movement direction, all of which are updated every frame. We do all of this drawing with only two draw calls. The app consumes only a few percent of the CPU, but maxes out the GPU, drawing over 240,000 triangles per frame. Even with this large load, the device manages to render at an ideal 60 frames per second.

Dozens of animated characters can be drawn efficiently with instanced rendering.
Dozens of animated characters can be drawn efficiently with instanced rendering. (Grass texture provided by Simon Murray of goodtextures.com)

Read moreInstanced Rendering in Metal

Cubic Environment Mapping in Metal: Reflection and Refraction

In this post, we’ll talk about some of the more advanced features of texturing in Metal. We’ll apply a cube map to a skybox to simulate a detailed environment surrounding the scene. We’ll also introduce a technique called cubic environment mapping to simulate reflection and refraction, to further enhance the realism of our virtual world.

The environment map reflected off a torus knot
The environment map reflected off a torus knot. Textures courtesy of Emil Persson (http://humus.name)

Read moreCubic Environment Mapping in Metal: Reflection and Refraction

Fundamentals of Image Processing in Metal

In this post, we will start exploring the world of image processing with the Metal shading language. We will create a framework capable of representing chains of image filters, then write a pair of image filters that will allow us to adjust the saturation and blur of an image. The end result will be an interactive app that allows you to control the image filter parameters in real-time.

This post is designed to be read after its companion article, Introduction to Compute Programming.

The sample application UI, allowing filter adjustment in real-time.
The sample application UI, allowing filter adjustment in real-time.

You can download the sample project here.

Read moreFundamentals of Image Processing in Metal

Introduction to Compute Programming in Metal

This post is an introduction to topics in compute (or data-parallel) programming. It is designed to be read before its companion article, Fundamentals of Image Processing. In this article, we will cover the basics of setting up the compute pipeline and executing kernel functions on large sets of data in parallel. The companion article covers the basics of image processing and builds on the foundation laid here.

Read moreIntroduction to Compute Programming in Metal

Feature Sets and Capabilities

One of the changes made to the Metal API in the iOS 8 GM was the addition of the supportsFeatureSet: method to the MTLDevice protocol. The purpose of this method is to differentiate between devices powered by the A7 and A8 processor, because these chips offer different capabilities to Metal.

Read moreFeature Sets and Capabilities