Fundamentals of Image Processing in Metal

In this post, we will start exploring the world of image processing with the Metal shading language. We will create a framework capable of representing chains of image filters, then write a pair of image filters that will allow us to adjust the saturation and blur of an image. The end result will be an interactive app that allows you to control the image filter parameters in real-time.

This post is designed to be read after its companion article, Introduction to Compute Programming.

The sample application UI, allowing filter adjustment in real-time.
The sample application UI, allowing filter adjustment in real-time.

You can download the sample project here.

Read more

Introduction to Compute Programming in Metal

This post is an introduction to topics in compute (or data-parallel) programming. It is designed to be read before its companion article, Fundamentals of Image Processing. In this article, we will cover the basics of setting up the compute pipeline and executing kernel functions on large sets of data in parallel. The companion article covers the basics of image processing and builds on the foundation laid here.

Read more

Textures and Samplers in Metal

Textures are a central topic in rendering. Although they have many uses, one of their primary purposes is to provide a greater level of detail to surfaces than can be achieved with vertex colors alone.

In this post, we’ll talk about texture mapping, which helps us bring virtual characters to life. We’ll also introduce samplers, which give us powerful control over how texture data is interpreted while drawing. Along the way, we will be assisted by a cartoon cow named Spot.

You can download the sample code for this post here.

Read more

Feature Sets and Capabilities

One of the changes made to the Metal API in the iOS 8 GM was the addition of the supportsFeatureSet: method to the MTLDevice protocol. The purpose of this method is to differentiate between devices powered by the A7 and A8 processor, because these chips offer different capabilities to Metal.

Read more

Up and Running with Metal, Part 3: Lighting and Rendering in 3D

In this post, we’ll finally start rendering in 3D. In order to get there, we’ll talk about how to load 3D model data from disk, how to tell Metal to draw from a vertex buffer using indices, and how to manipulate objects in real time.

This post assumes that you know a little linear algebra. I have written an incomplete introduction to the subject in this post. There are also many excellent resources around the Internet should you need more information on a particular topic.

You can download the sample project for this post here.

Read more

Linear Algebra for Graphics Programming

Introduction

This post will cover the essential mathematics for doing 3D graphics programming. I chose to split it out into a separate post because there is quite a lot of ground to cover, and attempting to wedge all of these concepts into a tutorial post would be overwhelming. If you already have a grasp of this material, this post is largely optional, but it does establish the notational and geometric conventions I use elsewhere.

Read more

The Whats and Wherefores of Metal

The previous two posts covered many of the nitty-gritty details of Metal, but I wanted to take a step back and put Metal in context. Not only are future posts going to require more work — more math, more figures, more code, deeper examples — I feel it may be necessary to explain the motivation behind this blog. This post is about what I think Metal is and can be, and why I’m so interested in helping others learn about it.

Read more

Up and Running with Metal, Part 2: Drawing Triangles

In our inaugural post of the series, we got a glimpse of many of the essential moving parts of the Metal framework: devices, textures, command buffers, and command queues. Although that post was long, it couldn’t possibly cover all these things in detail. This post will add a little more depth to the discussion of the parts of Metal that are used when rendering geometry. In particular, we’ll take a trip through the Metal rendering pipeline, introduce functions and libraries, and issue our first draw calls.

The end goal of this post is to show you how to start rendering actual geometry with Metal. The triangles we draw will only be in 2D. Future posts will introduce the math necessary to draw 3D shapes and eventually animate 3D models.

Download the sample code for this post here.

Read more

Up and Running with Metal, Part 1: Clearing the Screen

This post covers the bare minimum needed to clear the screen to a solid color in Metal. Even this simple operation requires many of the concepts exposed by the Metal framework. The next few posts in the Up and Running series will build on what we discuss here and take us through the basics of 3D rendering and image processing.

You can download the sample project for this post here.

Read more