FFmpeg 4.3 H265八，AVFrame使用使用SDLVSQT显示yuv文件，使用ffmpeg的AVFrame

原创

已于 2025-04-09 10:53:23 修改 · 739 阅读

标签

#ffmpeg #音视频 #c++

于 2024-10-31 18:20:41 首次发布

学习目的

使用ffmpeg 解码视频文件，然后通过SDL渲染显示。

通过前面的学习，我们每次从 aaa.yuv 文件中读取一张图片的大小,读取的字节存储到videodata_中，大小为 width_ * height_ * 1.5，这个是没有问题。

readfile_.read((char*)videodata_, width_ * height_ * 1.5);

但是在实际开发中，我们读取的一般是 mp4文件，这个mp4文件通过 ffmpeg的方法解析，解析成avpacket，然后将avpacket 解析成avframe，因此我们还需要学习，如何将aaa.yuv文件读取到avframe中。

为了达到这个学习点，首先我们要弄清楚，AVFrame 是个啥？了解了内部结构，才知道如何将aaa.yuv文件，读取到 avframe中。

一 AVFrame 是什么？

AVFrame 存储的是解码后的数据，（包括音频和视频）例如：yuv数据，或者pcm数据，参考AVFrame结构体的第一句话。

二 AVFrame 核心变量

uint8_t *data[AV_NUM_DATA_POINTERS]

AV_NUM_DATA_POINTERS = 8；

一张图片，如果是YUV420，data[0]存放的就是所有的Y，data[1]存放的就是所有的U，data[2]存放的就是所有的V。

data[0]的大小是多少呢？在没有字节对齐的情况下： width * height *1

这是因为YUV420p中每一个Y都占用一个字节

data[1]的大小是多少呢？在没有字节对齐的情况下： width * height /4

这是因为YUV420p中每一个U都占用0.25字节

data[2]的大小是多少呢？在没有字节对齐的情况下： width * height /4

这是因为YUV420p中每一个V都占用0.25字节

一张图片，如果是RGB8888的，data[0]就放置了所有的数据，data[1]后面都是空的

data[0]的大小是多少呢？在没有字节对齐的情况下： width * height *4

在这里要搞清楚YUV420P 的 data[1], 和 data[2] 的值为什么是的排列

【秒懂音视频开发】18_详解YUV - M了个J - 博客园

data -->xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
           ^ ^ ^
          |                          |                        |
        data[0]                data[1]             data[2]

比如说，当pix_fmt=AV_PIX_FMT_YUV420P时，data中的数据是按照YUV的格式存储的，也就是：

data -->YYYYYYYYYYYYYYYYYYYYYYYYUUUUUUUUUUUVVVVVVVVVVVV
            ^ ^ ^
            | | |
       data[0] data[1] data[2]

int linesize[AV_NUM_DATA_POINTERS];

linesize是指对应于每一行的大小，为什么需要这个变量，是因为在YUV格式和RGB格式时，每行的大小不一定等于图像的宽度。

对于RGB8888 ， linesize[0] 的值为 width * 4

对于YUV420P：

在不考虑对齐的情况下：

linesize[0] 表示的Y 一行的大小，由于一个Y占用1字节，那么一行的Y的的大小为 width * 1

linesize[1] 表示的U 一行的大小，由于Y 和UV 在垂直方向和水平方向都是 2:1，因此一行U的大小为 width /2

linesize[2] 表示的V 一行的大小，由于Y 和UV 在垂直方向和水平方向都是 2:1，因此一行U的大小为 width /2

AVFrame 结构体

/**
 * This structure describes decoded (raw) audio or video data.
 *
 * AVFrame must be allocated using av_frame_alloc(). Note that this only
 * allocates the AVFrame itself, the buffers for the data must be managed
 * through other means (see below).
 * AVFrame must be freed with av_frame_free().
 *
 * AVFrame is typically allocated once and then reused multiple times to hold
 * different data (e.g. a single AVFrame to hold frames received from a
 * decoder). In such a case, av_frame_unref() will free any references held by
 * the frame and reset it to its original clean state before it
 * is reused again.
 *
 * The data described by an AVFrame is usually reference counted through the
 * AVBuffer API. The underlying buffer references are stored in AVFrame.buf /
 * AVFrame.extended_buf. An AVFrame is considered to be reference counted if at
 * least one reference is set, i.e. if AVFrame.buf[0] != NULL. In such a case,
 * every single data plane must be contained in one of the buffers in
 * AVFrame.buf or AVFrame.extended_buf.
 * There may be a single buffer for all the data, or one separate buffer for
 * each plane, or anything in between.
 *
 * sizeof(AVFrame) is not a part of the public ABI, so new fields may be added
 * to the end with a minor bump.
 *
 * Fields can be accessed through AVOptions, the name string used, matches the
 * C structure field name for fields accessible through AVOptions. The AVClass
 * for AVFrame can be obtained from avcodec_get_frame_class()
 */

typedef struct AVFrame {
#define AV_NUM_DATA_POINTERS 8
    /**
     * pointer to the picture/channel planes.
     * This might be different from the first allocated byte. For video,
     * it could even point to the end of the image data.
     *
     * All pointers in data and extended_data must point into one of the
     * AVBufferRef in buf or extended_buf.
     *
     * Some decoders access areas outside 0,0 - width,height, please
     * see avcodec_align_dimensions2(). Some filters and swscale can read
     * up to 16 bytes beyond the planes, if these filters are to be used,
     * then 16 extra bytes must be allocated.
     *
     * NOTE: Pointers not needed by the format MUST be set to NULL.
     *
     * @attention In case of video, the data[] pointers can point to the
     * end of image data in order to reverse line order, when used in
     * combination with negative values in the linesize[] array.
     */
    uint8_t *data[AV_NUM_DATA_POINTERS];

    /**
     * For video, a positive or negative value, which is typically indicating
     * the size in bytes of each picture line, but it can also be:
     * - the negative byte size of lines for vertical flipping
     *   (with data[n] pointing to the end of the data
     * - a positive or negative multiple of the byte size as for accessing
     *   even and odd fields of a frame (possibly flipped)
     *
     * For audio, only linesize[0] may be set. For planar audio, each channel
     * plane must be the same size.
     *
     * For video the linesizes should be multiples of the CPUs alignment
     * preference, this is 16 or 32 for modern desktop CPUs.
     * Some code requires such alignment other code can be slower without
     * correct alignment, for yet other it makes no difference.
     *
     * @note The linesize may be larger than the size of usable data -- there
     * may be extra padding present for performance reasons.
     *
     * @attention In case of video, line size values can be negative to achieve
     * a vertically inverted iteration over image lines.
     */
    int linesize[AV_NUM_DATA_POINTERS];

    /**
     * pointers to the data planes/channels.
     *
     * For video, this should simply point to data[].
     *
     * For planar audio, each channel has a separate data pointer, and
     * linesize[0] contains the size of each channel buffer.
     * For packed audio, there is just one data pointer, and linesize[0]
     * contains the total size of the buffer for all channels.
     *
     * Note: Both data and extended_data should always be set in a valid frame,
     * but for planar audio with more channels that can fit in data,
     * extended_data must be used in order to access all channels.
     */
    uint8_t **extended_data;

    /**
     * @name Video dimensions
     * Video frames only. The coded dimensions (in pixels) of the video frame,
     * i.e. the size of the rectangle that contains some well-defined values.
     *
     * @note The part of the frame intended for display/presentation is further
     * restricted by the @ref cropping "Cropping rectangle".
     * @{
     */
    int width, height;
    /**
     * @}
     */

    /**
     * number of audio samples (per channel) described by this frame
     */
    int nb_samples;

    /**
     * format of the frame, -1 if unknown or unset
     * Values correspond to enum AVPixelFormat for video frames,
     * enum AVSampleFormat for audio)
     */
    int format;

    /**
     * 1 -> keyframe, 0-> not
     */
    int key_frame;

    /**
     * Picture type of the frame.
     */
    enum AVPictureType pict_type;

    /**
     * Sample aspect ratio for the video frame, 0/1 if unknown/unspecified.
     */
    AVRational sample_aspect_ratio;

    /**
     * Presentation timestamp in time_base units (time when frame should be shown to user).
     */
    int64_t pts;

    /**
     * DTS copied from the AVPacket that triggered returning this frame. (if frame threading isn't used)
     * This is also the Presentation time of this AVFrame calculated from
     * only AVPacket.dts values without pts values.
     */
    int64_t pkt_dts;

    /**
     * Time base for the timestamps in this frame.
     * In the future, this field may be set on frames output by decoders or
     * filters, but its value will be by default ignored on input to encoders
     * or filters.
     */
    AVRational time_base;

#if FF_API_FRAME_PICTURE_NUMBER
    /**
     * picture number in bitstream order
     */
    attribute_deprecated
    int coded_picture_number;
    /**
     * picture number in display order
     */
    attribute_deprecated
    int display_picture_number;
#endif

    /**
     * quality (between 1 (good) and FF_LAMBDA_MAX (bad))
     */
    int quality;

    /**
     * for some private data of the user
     */
    void *opaque;

    /**
     * When decoding, this signals how much the picture must be delayed.
     * extra_delay = repeat_pict / (2*fps)
     */
    int repeat_pict;

    /**
     * The content of the picture is interlaced.
     */
    int interlaced_frame;

    /**
     * If the content is interlaced, is top field displayed first.
     */
    int top_field_first;

    /**
     * Tell user application that palette has changed from previous frame.
     */
    int palette_has_changed;

#if FF_API_REORDERED_OPAQUE
    /**
     * reordered opaque 64 bits (generally an integer or a double precision float
     * PTS but can be anything).
     * The user sets AVCodecContext.reordered_opaque to represent the input at
     * that time,
     * the decoder reorders values