Creating and Sampling Textures 创建和采样纹理

Creating and Sampling Textures

创建和采样纹理
【说明：】
本文将尽量采用英-中对照方式，来看看效果。

Load image data into a texture and apply it to a quadrangle.
将图像数据加载到纹理中并将其应用于四边形。

Overview 概述

You use textures to draw and process images in Metal.
A texture is a structured collection of texture elements, often called texels or pixels.
The exact configuration of these texture elements depends on the type of texture.
This sample uses a texture structured as a 2D array of elements, each of which contains color data, to hold an image.
您可以使用纹理在 Metal 中绘制和处理图像。
纹理是纹理元素的结构化集合，通常称为纹素或像素。这些纹理元素的确切配置取决于纹理的类型。此示例使用结构为 2D 元素数组的纹理来保存图像，每个元素都包含颜色数据。
The texture is drawn onto geometric primitives through a process called texture mapping. The fragment function generates colors for each fragment by sampling the texture.
纹理通过称为“纹理映射”的过程绘制到几何图元上。片段函数通过对纹理进行采样来为每个片段生成颜色。
Textures are managed by MTLTexture objects.
A MTLTexture object defines the texture’s format, including the size and layout of elements, the number of elements in the texture, and how those elements are organized.
Once created, a texture’s format and organization never change.
However, you can change the contents of the texture, either by rendering to it or copying data into it.
纹理由MTLTexture对象管理。
一个MTLTexture对象定义了纹理的格式，包括元素的大小和布局、纹理中元素的数量以及这些元素的组织方式。
一旦创建，纹理的格式和组织永远不会改变。但是，您可以通过渲染或将数据复制到其中来更改纹理的内容。
The Metal framework doesn’t provide an API to directly load image data from a file to a texture.
Metal itself only allocates texture resources and provides methods that copy data to and from the texture.
Metal apps rely on custom code or other frameworks, like MetalKit, Image I/O, UIKit, or AppKit, to handle image files. For example, you can use MTKTextureLoader to perform simple texture loading. This sample shows how to write a custom texture loader.
Metal 框架不提供 API 来直接将图像数据从文件加载到纹理。
Metal 本身只分配纹理资源，并提供将数据复制到纹理和从纹理复制数据的方法。
Metal 应用程序依赖自定义代码或其他框架（如 MetalKit、Image I/O、UIKit 或 AppKit）来处理图像文件。例如，您可以使用MTKTextureLoader来执行简单的纹理加载。此示例展示了如何编写自定义纹理加载器。

• Note: The Xcode project contains schemes for running the sample on macOS, iOS, or tvOS devices. The default scheme is macOS, which runs the sample as is on your Mac.
  笔记: Xcode 项目包含在 macOS、iOS 或 tvOS 设备上运行示例的方案。默认方案是 macOS，它按原样在 Mac 上运行示例。

Load and Format Image Data 加载和格式化图像数据

You can create a texture or update its contents manually, a process that’s covered in the next few sections.
You might do this for multiple reasons:
您可以手动创建纹理或更新其内容，此过程将在接下来的几节中介绍。您可能出于多种原因这样做：

• You have image data stored in a custom format.您以自定义格式存储了图像数据。
• You have textures whose contents need to be generated at runtime.您有其内容需要在运行时生成的纹理。
• You are streaming texture data from a server or otherwise need to dynamically update a texture’s contents.您正在从服务器流式传输纹理数据，或者需要动态更新纹理的内容。

In the sample, the AAPLImage class loads and parses image data from TGA files.
The class converts pixel data from the TGA file into a pixel format that Metal understands.
The sample uses the image’s metadata to create a new Metal texture and copies the pixel data into the texture.
在示例中，AAPLImage该类从 TGA 文件加载和解析图像数据。该类将 TGA 文件中的像素数据转换为 Metal 能够理解的像素格式。该示例使用图像的元数据创建新的 Metal 纹理并将像素数据复制到纹理中。

• Note: The AAPLImage class isn’t the focal point of this sample, so it isn’t discussed in detail. The class demonstrates basic image loading operations but doesn’t use or depend on the Metal framework. Its sole purpose唯一目的 is to facilitate loading image data and converting it into a Metal pixel format. You might create a similar class if you need to load an image that’s in a custom format.
  笔记: AAPLImage类并不是本示例的重点，故不详细讨论。该类演示了基本的图像加载操作，但不使用或依赖于 Metal 框架。它的唯一目的是方便加载图像数据并将其转换为 Metal 像素格式。如果您需要加载自定义格式的图像，您可以创建一个类似的类。
  Metal requires all textures to be formatted with a specific MTLPixelFormat value.
  The pixel format describes the layout of pixel data in the texture.
  This sample uses the MTLPixelFormatBGRA8Unorm pixel format, which uses 32 bits per pixel, arranged into 8 bits per component, in blue, green, red, and alpha order:
  Metal 要求所有纹理都使用特定的MTLPixelFormat格式。这个像素格式描述了纹理中像素数据的布局。此示例使用MTLPixelFormatBGRA8Unorm像素格式，每像素使用 32 位，按每个组件 8 位排列，按蓝色、绿色、红色和 alpha 顺序排列：MTLPixelFormatMTLPixelFormatBGRA8Unorm
  
  Before you can populate a Metal texture, you must format the image data into the texture’s pixel format.
  TGA files can provide pixel data either in a 32-bit-per-pixel format or a 24-bit-per-pixel format. TGA files that use 32 bits per pixel are already arranged in this format, so you just copy the pixel data.
  To convert a 24-bit-per-pixel BGR image, copy the red, green, and blue channels and set the alpha channel to 255, indicating a fully opaque pixel.
  在填充 Metal 纹理之前，您必须将图像数据格式化为纹理的像素格式。
  TGA 文件可以提供每像素 32 位格式或每像素 24 位格式的像素数据。使用每像素 32 位的 TGA 文件已经以这种格式排列，因此您只需复制像素数据。
  要转换每像素 24 位的 BGR 图像，请复制红色、绿色和蓝色通道并将 alpha 通道设置为 255，表示完全不透明的像素。

// Initialize a source pointer with the source image data that's in BGR form
uint8_t *srcImageData = ((uint8_t*)fileData.bytes +
                         sizeof(TGAHeader) +
                         tgaInfo->IDSize);

// Initialize a destination pointer to which you'll store the converted BGRA
// image data
uint8_t *dstImageData = mutableData.mutableBytes;

// For every row of the image
for(NSUInteger y = 0; y < _height; y++)
{
    // If bit 5 of the descriptor is not set, flip vertically
    // to transform the data to Metal's top-left texture origin
    NSUInteger srcRow = (tgaInfo->topOrigin) ? y : _height - 1 - y;

    // For every column of the current row
    for(NSUInteger x = 0; x < _width; x++)
    {
        // If bit 4 of the descriptor is set, flip horizontally
        // to transform the data to Metal's top-left texture origin
        NSUInteger srcColumn = (tgaInfo->rightOrigin) ? _width - 1 - x : x;

        // Calculate the index for the first byte of the pixel you're
        // converting in both the source and destination images
        NSUInteger srcPixelIndex = srcBytesPerPixel * (srcRow * _width + srcColumn);
        NSUInteger dstPixelIndex = 4 * (y * _width + x);

        // Copy BGR channels from the source to the destination
        // Set the alpha channel of the destination pixel to 255
        dstImageData[dstPixelIndex + 0] = srcImageData[srcPixelIndex + 0];
        dstImageData[dstPixelIndex + 1] = srcImageData[srcPixelIndex + 1];
        dstImageData[dstPixelIndex + 2] = srcImageData[srcPixelIndex + 2];

        if(tgaInfo->bitsPerPixel == 32)
        {
            dstImageData[dstPixelIndex + 3] =  srcImageData[srcPixelIndex + 3];
        }
        else
        {
            dstImageData[dstPixelIndex + 3] = 255;
        }
    }
}
_data = mutableData;

Create a Texture from a Texture Descriptor 从纹理描述符创建纹理

Use a MTLTextureDescriptor object to configure properties like texture dimensions and pixel format for a MTLTexture object.
Then call the newTextureWithDescriptor: method to create a texture.
使用一个MTLTextureDescriptor对象来配置MTLTexture对象的纹理尺寸和像素格式等属性。然后调用newTextureWithDescriptor方法创建纹理。

MTLTextureDescriptor *textureDescriptor = [[MTLTextureDescriptor alloc] init];

// Indicate that each pixel has a blue, green, red, and alpha channel, where each channel is
// an 8-bit unsigned normalized value (i.e. 0 maps to 0.0 and 255 maps to 1.0)
textureDescriptor.pixelFormat = MTLPixelFormatBGRA8Unorm;

// Set the pixel dimensions of the texture
textureDescriptor.width = image.width;
textureDescriptor.height = image.height;

// Create the texture from the device by using the descriptor
id<MTLTexture> texture = [_device newTextureWithDescriptor:textureDescriptor];

Metal creates a MTLTexture object and allocates memory for the texture data. This memory is uninitialized when the texture is created, so the next step is to copy your data into the texture.
Metal 创建一个MTLTexture对象并为纹理数据分配内存。创建纹理时，此内存未初始化，因此下一步是将数据复制到纹理中。

Copy the Image Data into the Texture 将图像数据复制到纹理中

Metal manages memory for textures and doesn’t provide you direct access to it. So you can’t get a pointer to the texture data in memory and copy the pixels yourself. Instead, you call methods on a MTLTexture object to copy data from memory you can access into the texture and vice versa.
Metal 管理纹理的内存,但是并不提供对它的直接访问。因此，您无法获得指向内存中纹理数据的指针并自己复制像素。取而代之，您可以调用MTLTexture对象上的方法来将数据从可以访问的内存复制到纹理中，反之亦然。
In this sample, the AAPLImage object allocated memory for the image data, so you’ll tell the texture object to copy this data.
在此示例中，AAPLImage对象为图像数据分配了内存，因此您要告诉纹理对象复制此数据。
Use a MTLRegion structure to identify which part of the texture you want to update.
This sample populates the entire texture with image data; so create a region that covers the entire texture.
使用MTLRegion结构来确定要更新纹理的哪个部分。此示例使用图像数据填充整个纹理；所以创建一个覆盖整个纹理的区域。

MTLRegion region = {
    { 0, 0, 0 },                   // MTLOrigin
    {image.width, image.height, 1} // MTLSize
};

Image data is typically organized in rows, and you need to tell Metal the offset between rows in the source image.
The image loading code creates image data in a tightly packed format, so the data of subsequent pixel rows immediately follows the previous row.
Calculate the offset between rows to be the exact length (in bytes) of a row — the number of bytes per pixel multiplied by the image width.
图像数据通常按行组织，您需要告诉 Metal 源图像中行之间的偏移量。
图像加载代码以“tightly packed紧密打包”的格式创建图像数据，因此后续像素行的数据紧跟在前一行之后。将行之间的偏移量计算为一行的确切长度（以字节为单位）——每像素的字节数乘以图像宽度。

NSUInteger bytesPerRow = 4 * image.width;

Call the replaceRegion:mipmapLevel:withBytes:bytesPerRow: method on the texture to copy pixel data from the AAPLImage object into the texture.
调用纹理上的方法 将像素数据从AAPLImage对象复制到纹理中。

[texture replaceRegion:region
            mipmapLevel:0
              withBytes:image.data.bytes
            bytesPerRow:bytesPerRow];

Map the Texture Onto a Geometric Primitive 将纹理映射到几何图元

You can’t render a texture on its own; you must map it onto geometric primitives (in this example, a pair of triangles) that are output by the vertex stage and turned into fragments by the rasterizer.
Each fragment needs to know which part of the texture should be applied to it.
You define this mapping with texture coordinates: floating-point positions that map locations on a texture image to locations on the geometric surface.
纹理不能自己渲染纹理；您必须将其映射到 由顶点阶段输出并由光栅化器转换为片段的几何图元（在本例中为一对​​三角形）。
每个片段都需要知道应该将纹理的哪个部分应用于它。
使用纹理坐标定义此映射：将纹理图像上的位置映射到几何表面上的位置的浮点位置。
For 2D textures, normalized texture coordinates are values from 0.0 to 1.0 in both x and y directions.
A value of (0.0, 0.0) specifies the texel at the first byte of the texture data (the top-left corner of the image).
A value of (1.0, 1.0) specifies the texel at the last byte of the texture data (the bottom-right corner of the image).
对于 2D 纹理，归一化纹理坐标是 x 和 y 方向上从 0.0 到 1.0 的值。
值 (0.0, 0.0) 指定纹理数据的第一个字节（图像的左上角）处的纹素。
值 (1.0, 1.0) 指定纹理数据最后一个字节（图像的右下角）的纹素。

Add a field to the vertex format to hold texture coordinates:
向顶点格式添加一个字段以保存纹理坐标：

typedef struct
{
    // Positions in pixel space. A value of 100 indicates 100 pixels from the origin/center.
    vector_float2 position;

    // 2D texture coordinate
    vector_float2 textureCoordinate;
} AAPLVertex;

In the vertex data, map the quad’s corners to the texture’s corners:
在顶点数据中，将四边形的角映射到纹理的角：

static const AAPLVertex quadVertices[] =
{
    // Pixel positions, Texture coordinates
    { {  250,  -250 },  { 1.f, 1.f } },
    { { -250,  -250 },  { 0.f, 1.f } },
    { { -250,   250 },  { 0.f, 0.f } },

    { {  250,  -250 },  { 1.f, 1.f } },
    { { -250,   250 },  { 0.f, 0.f } },
    { {  250,   250 },  { 1.f, 0.f } },
};

To send the texture coordinates to the fragment shader, add a textureCoordinate value to the RasterizerData data structure:
要将纹理坐标发送到片段着色器，请向RasterizerData数据结构添加一个textureCoordinate值：

struct RasterizerData
{
    // The [[position]] attribute qualifier of this member indicates this value is
    // the clip space position of the vertex when this structure is returned from
    // the vertex shader
    float4 position [[position]];

    // Since this member does not have a special attribute qualifier, the rasterizer
    // will interpolate its value with values of other vertices making up the triangle
    // and pass that interpolated value to the fragment shader for each fragment in
    // that triangle.
    float2 textureCoordinate;

};

In the vertex shader, pass the texture coordinates to the rasterizer stage by writing them into the textureCoordinate field.
The rasterizer stage interpolates插入 these coordinates across the quad’s triangle fragments.
在顶点着色器中，传送将纹理坐标到光栅化阶段（通过将纹理坐标写入textureCoordinate字段）。
光栅化阶段在四边形的三角形片段上插入这些坐标。

out.textureCoordinate = vertexArray[vertexID].textureCoordinate;

Calculate a Color from a Location in the Texture 从纹理中的位置计算颜色

You sample a texture to calculate a color from a location in the texture.
To sample the texture data, the fragment function needs the texture coordinates and a reference to the texture to sample. In addition to the arguments passed in from the rasterizer stage, pass in a colorTexture argument with a texture2d type and the [[texture(index)]] attribute qualifier.
This argument is a reference to a MTLTexture object to be sampled.
您对纹理进行采样以从纹理中的某个位置计算颜色。为了对纹理数据进行采样，片段函数需要纹理坐标和对要采样的纹理的引用。除了从光栅化阶段传入的参数之外，还要传入一个带有texture2d类型和[[texture(index)]]属性限定符的colorTexture参数。此参数是对要采样的MTLTexture对象的引用。

fragment float4
samplingShader(RasterizerData in [[stage_in]],
               texture2d<half> colorTexture [[ texture(AAPLTextureIndexBaseColor) ]])

Use the built-in texture sample() function to sample texel data.
The sample() function takes two arguments: a sampler (textureSampler) that describes how you want to sample the texture, and texture coordinates (in.textureCoordinate) that describe the position in the texture to sample.
The sample() function fetches one or more pixels from the texture and returns a color calculated from those pixels.
使用内置的纹理sample()函数对纹素数据进行采样。
该sample()函数采用两个参数：一个采样器 (textureSampler) 描述您希望如何对纹理进行采样，以及描述纹理中要采样的位置的纹理坐标 (in.textureCoordinate)。
该sanple()函数从纹理中获取一个或多个像素，并返回根据这些像素计算出的颜色。
When the area being rendered to isn’t the same size as the texture, the sampler can use different algorithms to calculate exactly what texel color the sample() function should return.
Set the mag_filter mode to specify how the sampler should calculate the returned color when the area is larger than the size of the texture, and the min_filter mode to specify how the sampler should calculate the returned color when the area is smaller than the size of the texture.
Setting a linear mode for both filters makes the sampler average the color of pixels surrounding the given texture coordinate, resulting in a smoother output image.
当渲染到的区域与纹理的大小不同时，采样器可以使用不同的算法来准确计算sample()函数应返回的纹素颜色。
设置mag_filter模式来 当指定当区域大于纹理大小时 采样器如何计算返回颜色，设置min_filter模式指定 当区域小于纹理大小时 采样器如何计算返回颜色。为两个过滤器设置一个linear模式会使采样器平均给定纹理坐标周围像素的颜色，从而产生更平滑的输出图像。

constexpr sampler textureSampler (mag_filter::linear,
                                  min_filter::linear);

// Sample the texture to obtain a color
const half4 colorSample = colorTexture.sample(textureSampler, in.textureCoordinate);

• Note: Try increasing or decreasing the size of the quad to see how filtering works.
  笔记: 尝试增加或减少四边形的大小以查看过滤的工作原理。

Encode the Draw Parameters 编码绘图参数

The process for encoding and submitting drawing commands is the same as that shown in Using a Render Pipeline to Render Primitives, so the complete code is not shown below.
The difference in this sample is that the fragment shader has an additional parameter. When you encode the command’s arguments, set the fragment function’s texture argument.
编码和提交绘图命令的过程与使用渲染管道渲染基元中所示的过程相同，因此下面不显示完整代码。此示例中的不同之处在于片段着色器有一个附加参数。对命令的参数进行编码时，请设置片段函数的纹理参数。
This sample uses the AAPLTextureIndexBaseColor index to identify the texture in both Objective-C and Metal Shading Language code.
此示例使用AAPLTextureIndexBaseColor索引来标识 Objective-C 和 Metal Shading Language 代码中的纹理。AAPLTextureIndexBaseColor

[renderEncoder setFragmentTexture:_texture
                          atIndex:AAPLTextureIndexBaseColor];

【2021-10-4】