View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000665Cinelerra-GG[All Projects] Bugpublic2024-09-03 21:132024-09-26 23:19
Reportertarantas Assigned ToPhyllisSmith  
PrioritynormalSeveritymajorReproducibilityalways
Status acknowledgedResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary0000665: JPEG color range rendering (to x264 file) is broken while using YUV 8-bit color model
DescriptionThe result of rendering always has an increased contrast when using JPEG color range + YUV 8-bit color model.
The rendered colors differ both from composite window and source video.
Color mismatch appears no matter if the source video was in full or limited color ranged (tested both). So the issue should be reproducible on any video (if I'm not wrong).
There are no way to compensate the issue using ColorSpace or F_colorspace (I've tried).
The setting in File -> Render -> Video -> Pixels doesn't help either.
Steps To Reproducerm -r ~/.bcast5
Settings -> Preferences -> Appearance -> YUV color space: bt709
Settings -> Preferences -> Appearance -> YUV color range: JPEG (default)
Settings -> Format -> Color model: YUV-8 bit
File -> Render (with all defaults).

Tested using CInelerra 2024-06 from git (sorry, I can't build the newest due to 0000650 bug)
Additional InformationThe issue could be easily avoided just by switching to RGB(A)-Float or RGB(A)-8 bit. The last one still causes a very slight color shift (compared to compose window) which I don't understand either.

I consider this issue to be important because it's a decent footgun for a user. Silently ruining the video at the very end of a time consuming process is the worst thing could happened after doing rm -rf /. By the way, I've heard from my friend about enjoying exact the same feature implemented in big commercial product DaVinci Resolve (I mean ruining the final render).

I'm using Xorg on Debian Bookworm.
TagsNo tags attached.

Activities

Balling

Balling

2024-09-26 23:19

reporter   ~0005718

All jpegs are full range BT.601 matrix. Those can be produced by all kinds of other libraries. https://github.com/mozilla/mozjpeg and https://github.com/libjpeg-turbo/libjpeg-turbo being two most important.

Davinci supports Superwhite and superblack, see project settings. It is used as reference tool in Hollywood since forever. Davinci supports full range YCbCr slog3 and V-Log (Sony and Panasonic cameras respectively). I dunno whether it supports xvYCC fully, but it probably does. Davinci 19 supports HDR PQ in Windows API.

Davinci also supports proile 5 of Dolby Vision that uses full range IPTPQc2 (instead of YCbCr).

Those are examples of what you can use for encoding and decoding full range content.
Andrew-R

Andrew-R

2024-09-26 23:09

reporter   ~0005717

@Balling, I think I mixed few things here, because I am not sure where I can find real full-range video NOT made by ffmpeg. So I tried to see if xvYCC file will show anything interesting on scopes.

For patch on ffmpeg-devel list ...well, we do yuv->yuv copies if timeline set to yuv(a)8 and range between timeline and encoder differs. So it *may* affect us, but I'll try to wait for v2 and see if I can get different test file.
Balling

Balling

2024-09-26 23:01

reporter   ~0005716

https://forum.videohelp.com/attachments/31711-1431757037/C0126.MP4

This file is tagged as transfer xvYCC, in mediainfo:


Color range : Limited
Color primaries : BT.709
Transfer characteristics : xvYCC
Matrix coefficients : BT.709


That is not yet supported by ffmpeg properly, though zscale should just work, I will admit I never tested it. I know of this file, though. I have it saved years ago.

Just some clarifications: xvYCC is only EVER tagged as limited range, as it uses actively colors outside limited range, that is more than 100% Y/CB/CR.

https://ffmpeg.org//pipermail/ffmpeg-devel/2024-September/333823.html

The patch is for output to limited range and it is rather cosmetic, also from YCbCr to YcbCr... Are you sure that help this bug?
Andrew-R

Andrew-R

2024-09-26 22:41

reporter   ~0005715

It seems ffmpeg/libavcodec actually has a bug in range conversion, first version of patch series was posted recently:

https://ffmpeg.org//pipermail/ffmpeg-devel/2024-September/333823.html

but v2 expected in the future.
Balling

Balling

2024-09-10 03:36

reporter   ~0005714

>Color mismatch appears no matter if the source video was in full or limited color ranged

For full range BT.709 color matrix this is a known bug. https://trac.ffmpeg.org/ticket/9693

Workaround is I am afraid to force actual properties of JPEG and using zscale, scale does not allow proper forcing of parameters.

ffmpeg -i "this one does it.mp4" -vf zscale=min=709:m=170m:cin=left:c=center -frames:v 1 -q:v 1 correct.jpeg
Andrew-R

Andrew-R

2024-09-07 07:34

reporter   ~0005713

I tested cingg compiled around 1 nov 2020 and bug was still there. Additionally, it seems that rendering to yuv444 from rgba8 was broken too, at least files do not match in YUVViewer

yuv-test.y4m (230,473 bytes)
2020_nov_1.mp4 (6,361 bytes)
Andrew-R

Andrew-R

2024-09-06 18:27

reporter   ~0005711

Another patch.

It seems that some rgb based formats like bpm in ffmpeg still broken in yuv and some other format settings.

But for mp4 in yuv and few other codecs there is progress.

not commintable, just for testing

0001-Avoid-setcolorspacedetails-if-any-yuv-colormodels-bu.patch (1,083 bytes) 
From e25c6e6ba794a357c48fbdb87320a7ebd66c77d4 Mon Sep 17 00:00:00 2001
From: Andrew Randrianasulu <[email protected]>
Date: Fri, 6 Sep 2024 21:24:51 +0300
Subject: [PATCH] Avoid setcolorspacedetails if any yuv colormodels but model
 37

---
 cinelerra-5.1/cinelerra/ffmpeg.C | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cinelerra-5.1/cinelerra/ffmpeg.C b/cinelerra-5.1/cinelerra/ffmpeg.C
index 9b8832dd..5bbfebf8 100644
--- a/cinelerra-5.1/cinelerra/ffmpeg.C
+++ b/cinelerra-5.1/cinelerra/ffmpeg.C
@@ -1808,7 +1808,7 @@ int FFVideoConvert::convert_vframe_picture(VFrame *frame, AVFrame *op, AVFrame *
 	if( !sws_getColorspaceDetails(convert_ctx,
 			&inv_table, &src_range, &table, &dst_range,
 			&brightness, &contrast, &saturation) ) {
-		if( dst_range != color_range || table != color_table )
+		if(( dst_range != color_range || table != color_table) && (!BC_CModels::is_yuv(cmodel) && cmodel != 37))
 			sws_setColorspaceDetails(convert_ctx,
 					inv_table, src_range, color_table, color_range,
 					brightness, contrast, saturation);
-- 
2.39.4
tarantas

tarantas

2024-09-05 21:05

reporter   ~0005710

I'm standing by as well.
PhyllisSmith

PhyllisSmith

2024-09-05 20:15

manager   ~0005709

@Andrew-R
When you have another patch ready, I will test it. But I have not had a chance to duplicate the problem yet.
Andrew-R

Andrew-R

2024-09-05 19:11

reporter   ~0005708

oh, no, it breaks ffmpeg/bmp! do not use, sorry. Probably right place but wrong if.
Andrew-R

Andrew-R

2024-09-05 18:54

reporter   ~0005707

Can you try attached patch ?

0001-Experimental-skip-setting-colorspace-coeffecients-on.patch (1,091 bytes) 
From dcfed3d6cf6e101ebeadb7eeb6ba7a6a0806a9ce Mon Sep 17 00:00:00 2001
From: Andrew Randrianasulu <[email protected]>
Date: Thu, 5 Sep 2024 21:54:19 +0300
Subject: [PATCH] Experimental: skip setting colorspace coeffecients on
 ffmpeg-based enc in yuv modes

---
 cinelerra-5.1/cinelerra/ffmpeg.C | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cinelerra-5.1/cinelerra/ffmpeg.C b/cinelerra-5.1/cinelerra/ffmpeg.C
index e89d1fa1..d9e86a4f 100644
--- a/cinelerra-5.1/cinelerra/ffmpeg.C
+++ b/cinelerra-5.1/cinelerra/ffmpeg.C
@@ -1810,7 +1810,7 @@ int FFVideoConvert::convert_vframe_picture(VFrame *frame, AVFrame *op, AVFrame *
 	if( !sws_getColorspaceDetails(convert_ctx,
 			&inv_table, &src_range, &table, &dst_range,
 			&brightness, &contrast, &saturation) ) {
-		if( dst_range != color_range || table != color_table )
+		if(( dst_range != color_range || table != color_table ) && (cmodel !=  27 && cmodel != 17))
 			sws_setColorspaceDetails(convert_ctx,
 					inv_table, src_range, color_table, color_range,
 					brightness, contrast, saturation);
-- 
2.39.4
Andrew-R

Andrew-R

2024-09-05 07:28

reporter   ~0005706

it seems that BT601/* works k even for YUV format YET
if I select BT709/BT2020 and YUV format it displays wrong.

Interestingly, if I render yuv test file to next track in rgb8 mode (correct), and then switch format to yuva8 -. it displays errors on subtract!

So ... error somewhere in somewhere in guicast/bccolors.h , bccolors.C - yuv <-> yuv functions ....
Andrew-R

Andrew-R

2024-09-04 18:07

reporter   ~0005705

There is specific YUV viewer (qt5 based)

https://github.com/IENT/YUView?tab=readme-ov-file

Compiled on Slackware 15.0 i586

You can open both yuv test file and file reencoded by cingg (in yuv / y4m format, so no compression) and add "Difference sequence", drag both files there, select View -. split view -> split view -> side by side, then enable Zoom Box.

so yeah, not really the same :(

I suspect yuv444 8 bit full range from timeline converts wrongly to very same color format by swscale ? May be due to our wrong setup ...

image.png (281,924 bytes)
Andrew-R

Andrew-R

2024-09-04 13:46

reporter   ~0005703

Playing with this command:


ffmpeg -f lavfi -i yuvtestsrc -frames 1 -color_range 2 /dev/shm/yuv-test.y4m

ffmpeg version 4.4.4 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11.2.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --docdir=/usr/doc/ffmpeg-4.4.4/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --enable-avresample --arch=i586 --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libass --enable-libbluray --enable-libcdio --enable-frei0r --enable-libgsm --enable-openal --enable-libopus --enable-librtmp --enable-libsnappy --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libmp3lame --enable-opencl --enable-opengl --enable-libopenjpeg --disable-libpulse --enable-libsmbclient --enable-libsvtav1 --enable-libxml2 --enable-librsvg --enable-libdrm --enable-libaom --enable-libdav1d --enable-libsoxr --enable-libzimg --enable-vapoursynth
  libavutil 56. 70.100 / 56. 70.100
  libavcodec 58.134.100 / 58.134.100
  libavformat 58. 76.100 / 58. 76.100
  libavdevice 58. 13.100 / 58. 13.100
  libavfilter 7.110.100 / 7.110.100
  libavresample 4. 0. 0 / 4. 0. 0
  libswscale 5. 9.100 / 5. 9.100
  libswresample 3. 9.100 / 3. 9.100
  libpostproc 55. 9.100 / 55. 9.100
Input #0, lavfi, from 'yuvtestsrc':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (444P / 0x50343434), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
File '/dev/shm/yuv-test.y4m' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, yuv4mpegpipe, to '/dev/shm/yuv-test.y4m':
  Metadata:
    encoder : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv444p(pc, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn
    Metadata:
      encoder : Lavc58.134.100 wrapped_avframe
frame= 1 fps=0.0 q=-0.0 Lsize= 225kB time=00:00:00.04 bitrate=46094.6kbits/s speed=36.2x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 58100.253906%

src for command:
https://stackoverflow.com/questions/75260266/why-is-ffmpeg-biased-when-converting-to-16-bit-integer-or-32-bit-float-rgb-outpu

references ffmpeg bug
https://trac.ffmpeg.org/ticket/7978
"Summary of the bug: inaccurate conversion from YCbCr Narrow range to RGB Full range "


But I use cingg for comparing ..

bash-5.1$ cin /dev/shm/yuv-test.y4m
Cinelerra Infinity - built: Aug 26 2024 06:26:42
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
2003-2017 mods for Cinelerra-CV by CinelerraCV team
2015-2024 mods for Cinelerra-GG by Cinelerra-GG team

Libav version: Lavc61.3.100

Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

RenderFarmClient::main_loop: client started
Render::render_single: Session finished.
** rendered 1 frames in 0.087 secs, 11.494 fps
Total excess of backups: -50
Session time: 0:04:27
Cpu time: user: 0:00:05.329 sys: 0:00:01.709

I render another yuv file (ffmpeg/y4m, yuv444p pix format, full range)
and then put them on same timeline and subtract.

In rgb8/rgb-float I see ends of color bars, in yuv8 I see green-ish picture.

So, it seems that for now we better to use rgba8 for working with full range data, as long as we output to ffmpeg's yuv-based codecs ?

I am not strictly sure if our compositing (subtract) in yuv-8 wrong, or something else.

To Be continued ...
Andrew-R

Andrew-R

2024-09-04 12:12

reporter   ~0005702

so, I made two pngs

av4.png is rgba rendering of first frame from mp4. bt709/jpeg - rgba, x11
av5.png is yuv8 rendering of first frame from same mp4, bt709/jpeg, yuva, x11

Loading them on new tracks in order 4,5 and setting av4.png to subtract:

1) creates clear videoscope in rgba8
2) creates some black areas on videoscope in yuva8

so it seems that rgba- > yuva8 conversion even by itself is not error free ...

av4.png (3,225,664 bytes)
av5.png (3,340,709 bytes)
Andrew-R

Andrew-R

2024-09-04 04:26

reporter   ~0005701

strange.

I tried to render same mp4 file, using same settings (bt709/jpeg) but using different format settings9default rgba8 and yuva-8), into two files.
Then I loaded those two files on separate tracks in freshly loaded cingg.
Subtract -> there still noise in all channels (even if visually compositor stays black, but colorpicker pick something if you move it around)

It happens even if I set cingg to bt709/mpeg for new test round (overwriting test files).

I do not think you can compare out of the box (without filters) full and limited color range clips ... thus I compared two full color clips just rendered.

So, there surely some difference, but I am not sure if it just manifestation of expected rounding errors (yuvj420p full -> yuv444 full -> yuvj420 full vs same but with rgb8 in the middle) or something else ?

I guess we need test signals and some math.
Andrew-R

Andrew-R

2024-09-04 03:09

reporter   ~0005700

so

I render TGA (rgb uncompressed) via project YUVa-8, bt709/jpeg to mp4/h264 (default)

It comes out as yuvj420p pix format and if I subtract it - I see much smaller signal on vectoroscope (output set to X11) then when I try to set bt709/mpeg on same project, so our just rendered full range h264 reinterpreted as limited range.

Probably I need to sleep some more over this. There was very long 11 pp forum discussion on videohelp about how to convert full range to limited range yuv.
So, it might be not as easy as we all just hoped ...

https://forum.videohelp.com/threads/395939-ffmpeg-Color-Range

probably setting another player/screenshotter is unavoidable. this thread recommended ffplay with some filters ... or vsedit (windows?).

I think at least on my machine I have OpenGL bug with yuv8 subtraction mode, but I can avoid it by using another output mode.
Andrew-R

Andrew-R

2024-09-04 02:38

reporter   ~0005699

well, it seems that on my machine both TGA files (on rendered with yuv8 proj setting and one rendered with rgba8 project setting) actually subtracts and renders to new TGA ok, it just OpenGL display that having problems ..??? I set Output to x11 or x11-xv and green image in compositor on subtraction is gone ....

If bug really affect rendering to any ffmpeg-based video format then I probably discovered something else :/
Andrew-R

Andrew-R

2024-09-04 02:12

reporter   ~0005698

even mre interesting, rendering two TGA (rgb uncompressed) files from same mp4, with project set to rgba-float, and then subtracting those TGAs only also result in solid green color when we switch project to yuv8.... but much smaller on videoscope, than it was for EXR case.
Andrew-R

Andrew-R

2024-09-04 02:03

reporter   ~0005697

Hm, apparently even two EXR files rendered from mp4 at rgba-float project setting still differs, even if I use non-ffmpeg reader (try ffmpeg last setting) ....

so, this points at some bug in our rgb-.yuv or yuv->rgb routine .... (because first we need to convert floating point rgba to yuv8 (timeline format), and then yuv8 for rgba8 for display at OpenGL-X11. )
Andrew-R

Andrew-R

2024-09-04 01:43

reporter   ~0005696

so, I also tried to render ONE first frame as EXR. and if I set subtract mode on upper video track it also shows some green-ish error, more if I set jpeg, less if I set mpeg, but error still here.

Interesting, so it seems that ffmpeg-based reading to yuv timeline already botched?
Andrew-R

Andrew-R

2024-09-04 01:29

reporter  

cingg-yuv-x264bug.png (449,364 bytes)
Andrew-R

Andrew-R

2024-09-04 01:25

reporter   ~0005695

To be honest I think it even exist in MPEG color range/yuv, at least for this clip?

https://forum.videohelp.com/attachments/31711-1431757037/C0126.MP4

Load clip, set project format to yuv(a), render to x264 withdefault profile but put result on new tracks.
Set upper track to subtract mode, put on Compositor's videoscope.

There will be visible green signal in both bt709/mpeg and bt709/jpeg.
No such signal if you set project to rgba/rgba-float ...

Issue History

Date Modified Username Field Change
2024-09-03 21:13 tarantas New Issue
2024-09-04 01:25 Andrew-R Note Added: 0005695
2024-09-04 01:29 Andrew-R File Added: cingg-yuv-x264bug.png
2024-09-04 01:43 Andrew-R Note Added: 0005696
2024-09-04 02:03 Andrew-R Note Added: 0005697
2024-09-04 02:12 Andrew-R Note Added: 0005698
2024-09-04 02:38 Andrew-R Note Added: 0005699
2024-09-04 03:09 Andrew-R Note Added: 0005700
2024-09-04 04:26 Andrew-R Note Added: 0005701
2024-09-04 12:12 Andrew-R File Added: av4.png
2024-09-04 12:12 Andrew-R File Added: av5.png
2024-09-04 12:12 Andrew-R Note Added: 0005702
2024-09-04 13:46 Andrew-R Note Added: 0005703
2024-09-04 18:07 Andrew-R File Added: image.png
2024-09-04 18:07 Andrew-R Note Added: 0005705
2024-09-05 07:28 Andrew-R Note Added: 0005706
2024-09-05 16:33 PhyllisSmith Assigned To => PhyllisSmith
2024-09-05 16:33 PhyllisSmith Status new => acknowledged
2024-09-05 18:54 Andrew-R File Added: 0001-Experimental-skip-setting-colorspace-coeffecients-on.patch
2024-09-05 18:54 Andrew-R Note Added: 0005707
2024-09-05 19:11 Andrew-R Note Added: 0005708
2024-09-05 20:15 PhyllisSmith Note Added: 0005709
2024-09-05 21:05 tarantas Note Added: 0005710
2024-09-06 18:27 Andrew-R File Added: 0001-Avoid-setcolorspacedetails-if-any-yuv-colormodels-bu.patch
2024-09-06 18:27 Andrew-R Note Added: 0005711
2024-09-07 07:34 Andrew-R File Added: yuv-test.y4m
2024-09-07 07:34 Andrew-R File Added: 2020_nov_1.mp4
2024-09-07 07:34 Andrew-R Note Added: 0005713
2024-09-10 03:36 Balling Note Added: 0005714
2024-09-26 22:41 Andrew-R Note Added: 0005715
2024-09-26 23:01 Balling Note Added: 0005716
2024-09-26 23:09 Andrew-R Note Added: 0005717
2024-09-26 23:19 Balling Note Added: 0005718