Skip to content

Enable local builds with FFmpeg8 #935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 9, 2025
Merged

Enable local builds with FFmpeg8 #935

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a132fe5
enable local build
Oct 2, 2025
9e31132
handle deprecated filter ctx function
Oct 6, 2025
fddf741
rebase
Oct 6, 2025
c4d0df8
use av_opt_set_array to avoid av_get_pix_fmt_name
Oct 8, 2025
4d41abb
replace nullptrs with torch check
Oct 8, 2025
c5fa1e1
incorporate suggestions
Oct 9, 2025
7f96f98
add comment explaining nb_elements
Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/torchcodec/_core/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,8 @@ else()
set(ffmpeg_major_version "6")
elseif (${libavcodec_major_version} STREQUAL "61")
set(ffmpeg_major_version "7")
elseif (${libavcodec_major_version} STREQUAL "62")
set(ffmpeg_major_version "8")
else()
message(
FATAL_ERROR
Expand Down
66 changes: 66 additions & 0 deletions src/torchcodec/_core/FFMPEGCommon.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@

#include <c10/util/Exception.h>

extern "C" {
#include <libavfilter/avfilter.h>
#include <libavfilter/buffersink.h>
}

namespace facebook::torchcodec {

AutoAVPacket::AutoAVPacket() : avPacket_(av_packet_alloc()) {
Expand Down Expand Up @@ -374,6 +379,67 @@ SwrContext* createSwrContext(
return swrContext;
}

AVFilterContext* createBuffersinkFilter(
AVFilterGraph* filterGraph,
const char* name,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're always passing "out" here, so let's just remove this parameter and hard-code "out" below. If we ever need another name, we can add the parameter then.

enum AVPixelFormat outputFormat) {
const AVFilter* buffersink = avfilter_get_by_name("buffersink");
TORCH_CHECK(buffersink != nullptr, "Failed to get buffersink filter.");

AVFilterContext* sinkContext = nullptr;
int status;

// av_opt_set_int_list was replaced by av_opt_set() in FFmpeg 8.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// av_opt_set_int_list was replaced by av_opt_set() in FFmpeg 8.
// av_opt_set_int_list was replaced by av_opt_set_array() in FFmpeg 8.

#if LIBAVUTIL_VERSION_MAJOR >= 60 // FFmpeg >= 8
// Output options like pixel_formats must be set before filter init
sinkContext = avfilter_graph_alloc_filter(filterGraph, buffersink, name);
TORCH_CHECK(
sinkContext != nullptr, "Failed to allocate buffersink filter context.");

status = av_opt_set_array(
sinkContext,
"pixel_formats",
AV_OPT_SEARCH_CHILDREN,
0, // start_elem
1, // nb_elems
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be set to 2 now

Suggested change
1, // nb_elems
2, // nb_elems

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually ok at 1, we do not want to set AV_PIX_FMT_NONE, only the first real pixel format. It seems AV_PIX_FMT_NONE is used to indicate the end of the array, as per claude:

The difference is in how the two APIs handle the pixel format list:

FFmpeg <= 7: av_opt_set_int_list()

enum AVPixelFormat pix_fmts[] = {outputFormat, AV_PIX_FMT_NONE};
av_opt_set_int_list(..., pix_fmts, AV_PIX_FMT_NONE, ...);

  • Uses sentinel-terminated arrays
  • AV_PIX_FMT_NONE (-1) is the terminator, not a value to set
  • Only sets 1 pixel format (stops when it hits the terminator)

FFmpeg 8: av_opt_set_array()

av_opt_set_array(..., nb_elems=2, ..., &outputFormat);

  • Uses count-based arrays (no sentinel)
  • Reads exactly nb_elems values from memory
  • With nb_elems=2, it reads:
    a. Element 0: outputFormat ✓
    b. Element 1: Memory after outputFormat (could be -1 or garbage)

If element 1 is -1 (AV_PIX_FMT_NONE), FFmpeg 8's validation rejects it as out of range [0 - 2.14748e+09], because -1 is now treated as an invalid value, not a
terminator.

Why the Error

FFmpeg 8 validates array elements more strictly. Negative values like AV_PIX_FMT_NONE (-1) are rejected, causing:
Cannot set array element 1 for parameter 'pixel_formats': value -1.000000 out of range

Solution

Set nb_elems=1 to only pass the single valid pixel format, not the terminator.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK that sounds good, let's just add a comment to that effect

AV_OPT_TYPE_PIXEL_FMT,
&outputFormat);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we pass enum AVPixelFormat pix_fmts[] = {outputFormat, AV_PIX_FMT_NONE}; like we were doing it below instead of just passing outputFormat?

Unless there's a good reason not to, in doubt, I think we should do the same.

TORCH_CHECK(
status >= 0,
"Failed to set pixel format for buffersink filter: ",
getFFMPEGErrorStringFromErrorCode(status));

status = avfilter_init_str(sinkContext, nullptr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? In #935 (review) the only thing we had to change was to replace av_opt_set_int_list with av_opt_set_array, but I think we can still rely on avfilter_graph_create_filter for FFmpeg 8 ? Same above with avfilter_graph_alloc_filter which may not be needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the new flow of allocation -> set flags -> init is required to set the pixel_formats flag. If we try to retain the previous pattern, this error appears:
Option 'pixel_formats' is not a runtime option and so cannot be set after the object has been initialized

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own ref, claude is telling me that the way you've done it above is the actual new recommended way:

● Based on my research, here's the answer to your question about FFmpeg 8 changes and simpler alternatives:

  Why avfilter_init_str() is Required in FFmpeg 8

  The use of avfilter_init_str() is unavoidable and actually represents the correct, simplified approach for FFmpeg
   8. Here's why:

  The API Change

  FFmpeg 8 deprecated av_opt_set_int_list() and replaced it with array-type options that require a two-step 
  initialization process:

  1. Allocate filter context: avfilter_graph_alloc_filter()
  2. Set array options: av_opt_set_array()
  3. Initialize filter: avfilter_init_str() or avfilter_init_dict()

  Why This is Actually Simpler

  Your current implementation is the simplified approach. The alternative would be more complex:

  Alternative 1: String-based (less flexible)
  // Would require converting enum to string manually
  char pix_fmt_str[32];
  av_get_pix_fmt_string(pix_fmt_str, sizeof(pix_fmt_str), outputFormat);
  avfilter_graph_create_filter(&sinkContext, buffersink, name,
      "pixel_formats=yuv420p", nullptr, filterGraph);

  Alternative 2: Dictionary-based (more verbose)
  AVDictionary* opts = nullptr;
  char pix_fmt_str[32];
  av_get_pix_fmt_string(pix_fmt_str, sizeof(pix_fmt_str), outputFormat);
  av_dict_set(&opts, "pixel_formats", pix_fmt_str, 0);
  sinkContext = avfilter_graph_alloc_filter(filterGraph, buffersink, name);
  avfilter_init_dict(sinkContext, &opts);
  av_dict_free(&opts);

  Your Current Implementation is Optimal

  The pattern you have is the recommended FFmpeg 8 approach:

  sinkContext = avfilter_graph_alloc_filter(filterGraph, buffersink, name);
  av_opt_set_array(sinkContext, "pixel_formats", AV_OPT_SEARCH_CHILDREN,
                   0, 1, AV_OPT_TYPE_PIXEL_FMT, &outputFormat);
  avfilter_init_str(sinkContext, nullptr);

  This is simpler because:
  - Type-safe: Uses AV_OPT_TYPE_PIXEL_FMT directly
  - No string conversion: Avoids enum-to-string conversion
  - Clean separation: Options set before initialization
  - Minimal code: Fewer lines than alternatives

  Recommendation

  Keep your current implementation. The avfilter_init_str() call is not a workaround—it's the intended FFmpeg 8 API
   design. The separation of allocation, option-setting, and initialization provides better error handling and
  flexibility than the older combined approach.

  The FFmpeg 8 changes actually made the API more consistent, even if it requires one additional function call.

TORCH_CHECK(
status >= 0,
"Failed to initialize buffersink filter: ",
getFFMPEGErrorStringFromErrorCode(status));
#else // FFmpeg <= 7
// For older FFmpeg versions, create filter and then set options
status = avfilter_graph_create_filter(
&sinkContext, buffersink, name, nullptr, nullptr, filterGraph);
TORCH_CHECK(
status >= 0,
"Failed to create buffersink filter: ",
getFFMPEGErrorStringFromErrorCode(status));

enum AVPixelFormat pix_fmts[] = {outputFormat, AV_PIX_FMT_NONE};

status = av_opt_set_int_list(
sinkContext,
"pix_fmts",
pix_fmts,
AV_PIX_FMT_NONE,
AV_OPT_SEARCH_CHILDREN);
TORCH_CHECK(
status >= 0,
"Failed to set pixel formats for buffersink filter: ",
getFFMPEGErrorStringFromErrorCode(status));
#endif

return sinkContext;
}

UniqueAVFrame convertAudioAVFrameSamples(
const UniqueSwrContext& swrContext,
const UniqueAVFrame& srcAVFrame,
Expand Down
5 changes: 5 additions & 0 deletions src/torchcodec/_core/FFMPEGCommon.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,4 +237,9 @@ int64_t computeSafeDuration(
const AVRational& frameRate,
const AVRational& timeBase);

AVFilterContext* createBuffersinkFilter(
AVFilterGraph* filterGraph,
const char* name,
enum AVPixelFormat outputFormat);

} // namespace facebook::torchcodec
24 changes: 4 additions & 20 deletions src/torchcodec/_core/FilterGraph.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
// LICENSE file in the root directory of this source tree.

#include "src/torchcodec/_core/FilterGraph.h"
#include "src/torchcodec/_core/FFMPEGCommon.h"

extern "C" {
#include <libavfilter/buffersink.h>
Expand Down Expand Up @@ -63,7 +64,6 @@ FilterGraph::FilterGraph(
}

const AVFilter* buffersrc = avfilter_get_by_name("buffer");
const AVFilter* buffersink = avfilter_get_by_name("buffersink");

UniqueAVBufferSrcParameters srcParams(av_buffersrc_parameters_alloc());
TORCH_CHECK(srcParams, "Failed to allocate buffersrc params");
Expand Down Expand Up @@ -93,26 +93,10 @@ FilterGraph::FilterGraph(
"Failed to create filter graph : ",
getFFMPEGErrorStringFromErrorCode(status));

status = avfilter_graph_create_filter(
&sinkContext_, buffersink, "out", nullptr, nullptr, filterGraph_.get());
sinkContext_ = createBuffersinkFilter(
filterGraph_.get(), "out", filtersContext.outputFormat);
TORCH_CHECK(
status >= 0,
"Failed to create filter graph: ",
getFFMPEGErrorStringFromErrorCode(status));

enum AVPixelFormat pix_fmts[] = {
filtersContext.outputFormat, AV_PIX_FMT_NONE};

status = av_opt_set_int_list(
sinkContext_,
"pix_fmts",
pix_fmts,
AV_PIX_FMT_NONE,
AV_OPT_SEARCH_CHILDREN);
TORCH_CHECK(
status >= 0,
"Failed to set output pixel formats: ",
getFFMPEGErrorStringFromErrorCode(status));
sinkContext_ != nullptr, "Failed to create and configure buffersink");

UniqueAVFilterInOut outputs(avfilter_inout_alloc());
UniqueAVFilterInOut inputs(avfilter_inout_alloc());
Expand Down
2 changes: 1 addition & 1 deletion src/torchcodec/_core/ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def load_torchcodec_shared_libraries():
# libraries do not meet those conditions.

exceptions = []
for ffmpeg_major_version in (7, 6, 5, 4):
for ffmpeg_major_version in (8, 7, 6, 5, 4):
pybind_ops_module_name = _get_pybind_ops_module_name(ffmpeg_major_version)
decoder_library_name = f"libtorchcodec_core{ffmpeg_major_version}"
custom_ops_library_name = f"libtorchcodec_custom_ops{ffmpeg_major_version}"
Expand Down
Loading