 |
I found two other Linux tools for dealing with timecode (not personally tested by me) https://github.com/nu774/mp4fpsmod - "Tiny mp4 time code editor. You can use this for changing fps, delaying audio tracks, executing DTS compression, extracting time codes of mp4." https://github.com/x42/ltc-tools - "Commandline tools to deal with linear-timecode (LTC). requires libltc: https://github.com/x42/libltc" where libltc readme says "libLTC - Linear (or Longitudinal) Timecode (LTC) is an encoding of SMPTE timecode data as a Manchester-Biphase encoded audio signal. The audio signal is commonly recorded on a VTR track or other storage media. libltc provides functionality to encode and decode LTC audio from/to SMPTE or EBU timecode, including SMPTE date support." |
 |
I have tested the 'align timecodes' function in the new version and am very happy to say that is working wonderful! Along with the 'To clip' function I can now make synced up subclips. I believe that to my knowledge Cinelerra GG is now the first FLOSS video editing program to support timecode syncing with subclip functionality: that's a big premiere! Thanks to everyone involved and especially PhyllisSmith. Few notes for further improvements: - A batch function would be great, especially to already sync up the clips from the 'Resources' window. - When I use the 'align timecodes' function the timecode of the whole sequence changes to the timecode of the clip I'm syncing. I have not seen this in other editing programs and don't think this is a useful feature, I think it would be better if the timecode of the sequence stays untouched and the clips stays in place. |
 |
Update for others: 2 sets of good test media was sent and a test of the first one, worked great with simply loading, and using the option in the pulldown of "Align Timecodes". The second test did not go well and it looked like the timing was off by 10.5 seconds making it not working to get aligned. Still looking at this to see if there is any way to salvage it. Also need to "Syncing in the timeline with timecode would be great but the next step is making a subclip of the synced up audio and video for editing." and I suspect some way to batch this up is needed. More later. |
|
|
@Katacuts No problem, we thought that eventually someone would want this capability enough to provide good test media. However you find a way to email or privately share a drive to [email protected], we will keep them private and only use to be able to set up a test for programming purposes. We are not at all sure that we can make this work yet. Thanks for the additional url as the previous one you initially sent was very helpful. |
|
|
I am very sorry for not checking this post sooner. It’s been a couple of months and I see there has been allot of thought and effort put into this problem. I am aware there is a distance between the programming world and the A/V world but did not expect that you would not have testing media for syncing. @ PhyllisSmith My apologies for halting the development by not supplying you test material. I am going to transfer you some files now, please indeed keep these private and only use these for testing purposes. Like I stated in the opening post, I have looked around the libre video editing world and have not found a program offering the timecode syncing possibility. Of course there are workflows possible using (proprietary) DaVinci Resolve to sync and make proxies, you can import these proxies and edit these with your favorite NLE. But it’s an extra program and export step that would better be done in the same NLE you are editing with. Besides that making proxies is time consuming. The only two NLE’s that can sync with timecode (that I know of) and work on Linux are proprietary at the moment: Lighworks & DaVinci Resolve. I hope we can change that! For clarity again: I am not talking about wavevorm syncing like in the multicam examples. It’s about using camera clips that have an external audio recorder. The camera and audio recorder are sync-locked with a genlock device on the set often several times a day. This article might give some clarity: https://www.bhphotovideo.com/explora/video/tips-and-solutions/timecode-versus-sync-how-they-differ-and-why-it-matters . |
 |
@PhyllisSmith I tried the new "Mix masters" option. It works; it creates a mixer for each track already present in the timeline. What seems strange to me is that it creates mixers for video tracks, but also for audio tracks. Is it useful to create a mixer screen for a sound track? At the moment I'm not sure how to use this. |
 |
To the question @Andrea_Paz asked about other Linux software... I couldn't possibly give a definitive answer, but I'm not personally aware of any other free/libre NLEs that make use of embedded timecode metadata. (I have no idea what's out there in the commercial space, nor do I care.) Shotcut would seem the most likely candidate, and there doesn't APPEAR to be any support, at least from a quick survey of the interface and available documentation. There's no mention at https://www.shotcut.org/features/ which seems like the place that info would be. And in the UI, I can see where it's able to READ the timecode metadata out of the C0001.mp4 file, as an example: 1. Load C0001.mp4 as the active Source video 2. Open the Properties panel by clicking the main toolbar icon 3. Open that panel's hamburger menu and choose "More information" 4. One of the sections of the resulting data dump (which, as the first line notes, is just the results of `ffprobe`, the same software we've been using to examine the metadata) will show: ~~~~~~~~ [streams.stream.2.tags] creation_time=2017-02-12T01:31:28.000000Z language=und handler_name=Timed Metadata Media Handler timecode=83:01:01;02 ~~~~~~~~ However (and these are the critical bits), the embedded timecode doesn't show up in the Metadata tab of the Properties panel itself, nor does 83:01:01;02 ever appear anywhere in the editing interface — unlike the Creation Data metadata, which is consumed by Shotcut, and can be modified in the UI. So, if there is any support for metadata-based synchronization, it's terrifically well hidden. I suspect there just isn't. Cinelerra-GG may indeed be one of the first/only, at least in the Linux open-source world, which I agree makes it a differentiator worth promoting. I'd only caution against jumping the gun on any — to borrow a turn of phrase from the posterior-covering branch of Madison Avenue — "forward-looking statements" regarding what features _may_ be present in the future. (No matter how near that future appears to be.) Holding off on "advertising" the timecode-sync tools until they're completed, tested, and known to be working/useful would seem prudent. � |
|
|
The "Align Timecodes" was moved from under Window to under Tracks pulldown per Sam suggestion. Plus a minor fix was added to align_timecodes where a clip was overlapping moves. I am still working on understanding and fixing the document in respect to a lot of the questions/answers from Andrea/ferdnyc. @Pierre Thanks for more feedback. I did check the "Clowns" you had sent awhile back but they had no timecodes that I could see. There is a Mint 18 version with the latest changes at: https://cinelerra-gg.org/download/testing/cinelerra-5.1-mint18-x86_64-static.txz In this version note that under the Window pulldown, the Mixers option, there is now a "Mix masters" option. This has not been documented yet, but it makes it easy to load up several tracks with video and then do a bunch of edits, and then you can use the option "Mix masters" to make mixer tracks out of the edited material. You will then have to add another Video Track on the top to create the desired video like as if you had "Open Mixers" on the assets in the Resources window. There are other ways to make use of "Mix masters" too but I have not figured them out yet. |
|
|
(And if the "Align Timecodes" option can do some or all of that automatically, then so much the better. But the concept is the same whether it's a manual process or an automatic one.) |
|
|
@Andrea_Paz Here, a theoretical example may help. Imagine you have three videos, recordings of an event from three different angles, simultaneously shot on three cameras. The cameras all started and stopped recording at different times, but they were all synced to the same master clock, so each recording is timestamped with the synchronized "world time" of the recording — all three cameras using the same times relative to each other. So, the camera files each have an embedded timestamp for the first frame of video: Camera 1: 00:10:13;20 Camera 2: 00:18:03;05 Camera 3: 00:06:24;07 When you load in the three files to different tracks, they'll be placed on a timeline that starts from 00:00:00:00 as usual. But, by middle-clicking on any of the video tracks, you can view the timecode embedded at that point in that file. To make it easier to align the videos onto the timeline, it makes sense to set the "origin point" (the leftmost time index) of the timeline ruler to the timecode of the earliest video, which would be camera 3. So, by clicking the clapboard, you can set 00:06:24;07 as the start timecode for the timeline. Then, positioning the camera 3 video at the start of the timeline will line its frames up with the times on the ruler. Having done that, you can position the other two recordings so their timecodes line up, and all three tracks will be in sync. |
 |
I do not really understand how timecodes work; I would like to ask a few questions: What's the point of using the timecode option in the timebar? How do I choose the track with the minor timecode ("The first step is to locate the earliest timecode")? How do I use RMB --> Timecode? What are clock times value? With the clapperboard button and Info --> timecode you can only see if the media has an embedded timecode or can you also edit an existing one or even create it if it is not there? How do I scan the asset using MMB? Generic question for those who know other NLEs for linux: Is CinGG the only one who has sync via timecode or are there others? (if it is the only one you should find a way to advertise it). |
|
|
I understand Phyllis your problem in getting multi-camera video footage with timecode correctly adjusted for use in synchronization . This requires that the timecode generators of all the cameras involved on a shoot (in multi-camera mode) have been connected by a suitable connector and precisely synchronized with each other before shooting. Their timecode generators must also have been configured identically, either in "drop frame" or "non-drop frame" mode, as well as in "free run" mode (where the timecode flows even when cameras are not shooting). Professional studios apply these procedures and have compatible cameras that allow this. In less industrial productions, one often has to deal with more disparate cameras and equipment, not all of which offer a reference timecode and even less often the possibility of being precisely synchronized with another device. Personally, all I have left is my old HDV camera (HVR-Z1U) which is now very outdated compared to the current cameras. From the "pro-user" camera range it allows to record timecode but does not include the necessary connections for synchronization with another device. All the video sequences I've sent you in the past probably have some timecode in the signal, but I doubt that this is enough to make really useful tests. |
|
|
@Andrea_Paz @Andrew-R @ferdnyc This is the code gg finally came up with and checked into GIT. As many statements have been made, what to use for the timecode and what it means is ill-defined. We never came up with any good "real" test for synchronizing with an auxiliary generated audio file so until we have that kind of material, this is all that can be done. It does work for multiple cameras that have video timecodes to line up. Attached is a quick document that will be added eventually to the manual after converted to latex. As already mentioned in email, a demo is at: https://streamable.com/nwec0t Align_Timecodes.pdf (388,992 bytes) |
|
|
@Andrew-R I have downloaded some of your suggested examples as I am sure that I will be doing a lot of testing. @ferdnyc Actually that last article is the kind GG likes and will make sense to him (not me!) |
|
|
@Andrew-R *nod* Same from an unmodified ffprobe 4.2.3, I guess that patch has long since landed upstream. One interesting wrinkle here is that we've already seen, just in two files, two completely different timecode formats. The C0001.MP4 file contained a first-frame timecode of '83:01:01;02'. The Sony sample file, it's '00:06:06:19'. Since that value is just a _string_ value, its interpretation is left up to the reading application, and there's a subtle but potentially important in how the value should be interpreted, when the last value is separated by a semicolon vs. a colon. A semicolon signifies _drop-frame_ timecode. Drop-frame timecode has to do with the standard video frame rates for NTSC being 29.97 fps, and the slight timing difference that introduces compared to SMPTE's standard clock rates. (Of which there are apparently only four: 24, 25, and 30 frames/second, plus 30 fps drop-frame, which is really 29.97 fps. I'll let Wikipedia take it from here... ====From: https://en.wikipedia.org/wiki/SMPTE_timecode#Drop-frame_timecode================ The altered frame rate meant that an "hour of timecode" at a nominal frame rate of 30 frame/s, when played back at 29.97 frame/s was longer than an hour of wall-clock time by 3.6 seconds, leading to an error of almost a minute and a half over a day. To correct this, drop-frame SMPTE timecode was invented. In spite of what the name implies, no video frames are dropped or skipped when using drop-frame timecode. Rather, some of the timecodes are dropped. In order to make an hour of timecode match an hour on the clock, drop-frame timecode skips frame numbers 0 and 1 of the first second of every minute, except when the number of minutes is divisible by ten.[a] This causes timecode to skip 18 frames each ten minutes (18,000 frames @ 30 frame/s) and almost perfectly compensates for the difference in rate. For example, the sequence when frame counts are dropped: 01:08:59:28 01:08:59:29 01:09:00:02 01:09:00:03 For each tenth minute: 01:09:59:28 01:09:59:29 01:10:00:00 01:10:00:01 While non-drop timecode is displayed with colons separating the digit pairs—"HH:MM:SS:FF"—drop-frame is usually represented with a semicolon (;) or period (.) as the divider between all the digit pairs—"HH;MM;SS;FF", "HH.MM.SS.FF"—or just between the seconds and frames—"HH:MM:SS;FF" or "HH:MM:SS.FF". =================== An article used as a source by that Wikipedia article, extremely informative but it gets very deep in the weeds (there are timing graphs and everything): http://www.philrees.co.uk/articles/timecode.htm |
|
|
https://forum.shotcut.org/t/xavc-s-format/19211 has link to this file as ffmpeg ticket sample: https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket5645/sony_a7s_XAVC_S_1080_25p_10s.MP4 I patched ffmpeg 2.8.17 with slightly modified patch: /home/guest/botva/src/ffmpeg/ffmpeg-2.8.11/ffmpeg_rtmd.patch 1589/1589 CP1251 100% --- libavformat/mov.c.orig 2020-07-07 21:49:08.000000000 +0300 +++ libavformat/mov.c 2020-07-11 00:38:30.086124634 +0300 @@ -4312,6 +4312,31 @@ return 0; } +static int mov_read_rtmd_track(AVFormatContext *s, AVStream *st) +{ + MOVStreamContext *sc = st->priv_data; + char buf[AV_TIMECODE_STR_SIZE]; + int64_t cur_pos = avio_tell(sc->pb); + int hh, mm, ss, ff, drop; + + if (!st->nb_index_entries) + return -1; + + avio_seek(sc->pb, st->index_entries->pos, SEEK_SET); + avio_skip(s->pb, 13); + hh = avio_r8(s->pb); + mm = avio_r8(s->pb); + ss = avio_r8(s->pb); + drop = avio_r8(s->pb); + ff = avio_r8(s->pb); + snprintf(buf, AV_TIMECODE_STR_SIZE, "%02d:%02d:%02d%c%02d", + hh, mm, ss, drop ? ';' : ':', ff); + av_dict_set(&st->metadata, "timecode", buf, 0); + + avio_seek(sc->pb, cur_pos, SEEK_SET); + return 0; +} + static int mov_read_timecode_track(AVFormatContext *s, AVStream *st) { MOVStreamContext *sc = st->priv_data; @@ -4578,8 +4603,11 @@ if (mov->chapter_track > 0) mov_read_chapters(s); for (i = 0; i < s->nb_streams; i++) - if (s->streams[i]->codec->codec_tag == AV_RL32("tmcd")) + if (s->streams[i]->codec->codec_tag == AV_RL32("tmcd")) { mov_read_timecode_track(s, s->streams[i]); + } else if (s->streams[i]->codec->codec_tag == AV_RL32("rtmd")) { + mov_read_rtmd_track(s, s->streams[i]); + } } /* copy timecode metadata from tmcd tracks to the related video streams */ and then output was like this: ffprobe sony_a7s_XAVC_S_1080_25p_10s.MP4 ffprobe version 2.8.17 Copyright (c) 2007-2020 the FFmpeg developers built with gcc 5.5.0 (GCC) configuration: --arch=i486 --target-os=linux --prefix=/usr --libdir=/usr/lib --mandir=/usr/man --docdir=/usr/doc/ffmpeg-2.8.17 --enable-gpl --enable-version3 --disable-static --enable-shared --enable-runtime-cpudetect --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-ffserver --enable-doc --enable-avdevice --enable-avcodec --enable-avformat --enable-avutil --enable-swresample --enable-swscale --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-ladspa --enable-libass --enable-libbluray --disable-libbs2b --disable-libcaca --disable-libcelt --enable-libcdio --disable-libdc1394 --disable-libflite --enable-libfreetype --enable-libfribidi --disable-libgme --enable-libgsm --enable-libiec61883 --disable-libilbc --disable-libkvazaar --disable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --disable-libpulse --disable-libquvi --enable-librtmp --enable-libschroedinger --enable-libsmbclient --enable-libsnappy --disable-libsoxr --enable-libspeex --disable-libssh --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-libzvbi --enable-lzma --enable-openal --enable-opengl --enable-sdl --enable-x11grab --enable-zlib --disable-debug libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sony_a7s_XAVC_S_1080_25p_10s.MP4': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 creation_time : 2016-03-05 14:17:25 Duration: 00:00:10.08, start: 0.000000, bitrate: 49084 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 47339 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default) Metadata: creation_time : 2016-03-05 14:17:25 handler_name : Video Media Handler encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2016-03-05 14:17:25 handler_name : Sound Media Handler Stream #0:2(und): Data: none (rtmd / 0x646D7472), 204 kb/s (default) Metadata: creation_time : 2016-03-05 14:17:25 handler_name : Non-Real Time Metadata timecode : 00:06:06:19 Unsupported codec with id 0 for input stream 2 |
|
|
@ferdnyc: I think even my old ffmpeg 2.8.15 can write 'artificial' timecode into MOV container, like this: ffmpeg -i C0126.MP4 -timecode 02:02:03:03 -c:a copy output.mov ffmpeg version 2.8.15 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.5.0 (GCC) configuration: --arch=i486 --target-os=linux --prefix=/usr --libdir=/usr/lib --mandir=/usr/man --docdir=/usr/doc/ffmpeg-2.8.15 --enable-gpl --enable-version3 --disable-static --enable-shared --enable-runtime-cpudetect --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-ffserver --enable-doc --enable-avdevice --enable-avcodec --enable-avformat --enable-avutil --enable-swresample --enable-swscale --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-ladspa --enable-libass --enable-libbluray --disable-libbs2b --disable-libcaca --disable-libcelt --enable-libcdio --disable-libdc1394 --disable-libflite --enable-libfreetype --enable-libfribidi --disable-libgme --enable-libgsm --enable-libiec61883 --disable-libilbc --disable-libkvazaar --disable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --disable-libpulse --disable-libquvi --enable-librtmp --enable-libschroedinger --enable-libsmbclient --enable-libsnappy --disable-libsoxr --enable-libspeex --disable-libssh --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-libzvbi --enable-lzma --enable-openal --enable-opengl --enable-sdl --enable-x11grab --enable-zlib --disable-debug libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 Guessed Channel Layout for Input Stream #0.1 : stereo Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C0126.MP4': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 creation_time : 2015-05-16 05:52:01 Duration: 00:00:04.80, start: 0.000000, bitrate: 48822 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 1920x1080 [SAR 1:1 DAR 16:9], 46870 kb/s, 50 fps, 50 tbr, 50k tbn, 100 tbc (default) Metadata: creation_time : 2015-05-16 05:52:01 handler_name : Video Media Handler encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2015-05-16 05:52:01 handler_name : Sound Media Handler Stream #0:2(und): Data: none (rtmd / 0x646D7472), 409 kb/s (default) Metadata: creation_time : 2015-05-16 05:52:01 handler_name : Non-Real Time Metadata [libx264 @ 0x8b4a8e0] using SAR=1/1 [libx264 @ 0x8b4a8e0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA4 FMA3 LZCNT BMI1 [libx264 @ 0x8b4a8e0] profile High, level 4.2 [libx264 @ 0x8b4a8e0] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 [mov @ 0x8b49ac0] Codec for stream 1 does not use global headers but container format requires global headers Output #0, mov, to 'output.mov': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 timecode : 02:02:03:03 encoder : Lavf56.40.101 Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 50 fps, 12800 tbn, 50 tbc (default) Metadata: creation_time : 2015-05-16 05:52:01 handler_name : Video Media Handler encoder : Lavc56.60.100 libx264 Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, stereo, 1536 kb/s (default) Metadata: creation_time : 2015-05-16 05:52:01 handler_name : Sound Media Handler Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help frame= 240 fps= 12 q=-1.0 Lsize= 7864kB time=00:00:04.80 bitrate=13421.7kbits/s video:6957kB audio:900kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.088144% [libx264 @ 0x8b4a8e0] frame I:1 Avg QP:25.63 size:364332 [libx264 @ 0x8b4a8e0] frame P:60 Avg QP:25.98 size: 96772 [libx264 @ 0x8b4a8e0] frame B:179 Avg QP:32.04 size: 5324 [libx264 @ 0x8b4a8e0] consecutive B-frames: 0.4% 0.0% 1.2% 98.3% [libx264 @ 0x8b4a8e0] mb I I16..4: 4.5% 18.4% 77.1% [libx264 @ 0x8b4a8e0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 46.8% 28.3% 22.3% 0.0% 0.0% skip: 2.6% [libx264 @ 0x8b4a8e0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 54.2% 1.1% 0.1% direct: 0.2% skip:44.4% L0:65.0% L1:28.3% BI: 6.7% [libx264 @ 0x8b4a8e0] 8x8 transform intra:18.6% inter:49.1% [libx264 @ 0x8b4a8e0] coded y,uvDC,uvAC intra: 99.7% 87.8% 44.3% inter: 19.6% 9.4% 0.0% [libx264 @ 0x8b4a8e0] i16 v,h,dc,p: 3% 2% 72% 24% [libx264 @ 0x8b4a8e0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 7% 14% 16% 7% 7% 5% 20% 12% [libx264 @ 0x8b4a8e0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 10% 16% 11% 7% 7% 6% 15% 9% [libx264 @ 0x8b4a8e0] i8c dc,h,v,p: 56% 20% 18% 7% [libx264 @ 0x8b4a8e0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x8b4a8e0] ref P L0: 51.1% 15.8% 27.9% 5.2% [libx264 @ 0x8b4a8e0] ref B L0: 94.1% 5.7% 0.2% [libx264 @ 0x8b4a8e0] ref B L1: 98.5% 1.5% [libx264 @ 0x8b4a8e0] kb/s:11872.72 guest@slax:/dev/shm$ ffprobe output.mov ffprobe version 2.8.15 Copyright (c) 2007-2018 the FFmpeg developers built with gcc 5.5.0 (GCC) configuration: --arch=i486 --target-os=linux --prefix=/usr --libdir=/usr/lib --mandir=/usr/man --docdir=/usr/doc/ffmpeg-2.8.15 --enable-gpl --enable-version3 --disable-static --enable-shared --enable-runtime-cpudetect --enable-ffmpeg --enable-ffplay --enable-ffprobe --enable-ffserver --enable-doc --enable-avdevice --enable-avcodec --enable-avformat --enable-avutil --enable-swresample --enable-swscale --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-ladspa --enable-libass --enable-libbluray --disable-libbs2b --disable-libcaca --disable-libcelt --enable-libcdio --disable-libdc1394 --disable-libflite --enable-libfreetype --enable-libfribidi --disable-libgme --enable-libgsm --enable-libiec61883 --disable-libilbc --disable-libkvazaar --disable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --disable-libpulse --disable-libquvi --enable-librtmp --enable-libschroedinger --enable-libsmbclient --enable-libsnappy --disable-libsoxr --enable-libspeex --disable-libssh --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-libzvbi --enable-lzma --enable-openal --enable-opengl --enable-sdl --enable-x11grab --enable-zlib --disable-debug libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output.mov': Metadata: major_brand : qt minor_version : 512 compatible_brands: qt encoder : Lavf56.40.101 Duration: 00:00:04.80, start: 0.000000, bitrate: 13421 kb/s Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 11873 kb/s, 50 fps, 50 tbr, 12800 tbn, 100 tbc (default) Metadata: handler_name : DataHandler encoder : Lavc56.60.100 libx264 timecode : 02:02:03:03 Stream #0:1(eng): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, stereo, s16, 1536 kb/s (default) Metadata: handler_name : DataHandler Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s Metadata: handler_name : DataHandler timecode : 02:02:03:03 Unsupported codec with id 0 for input stream 2 guest@slax:/dev/shm$ mediainfo output.mov General Complete name : output.mov Format : MPEG-4 Format profile : QuickTime Codec ID : qt 0000.02 (qt ) File size : 7.68 MiB Duration : 4 s 800 ms Overall bit rate : 13.4 Mb/s Writing application : Lavf56.40.101 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : [email protected] Format settings : CABAC / 4 Ref Frames Format settings, CABAC : Yes Format settings, Reference frames : 4 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 4 s 800 ms Bit rate : 11.9 Mb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 50.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.115 Stream size : 6.79 MiB (88%) Writing library : x264 core 148 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Language : English Codec configuration box : avcC Audio ID : 2 Format : PCM Format settings : Big / Signed Codec ID : twos Duration : 4 s 800 ms Bit rate mode : Constant Bit rate : 1 536 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Bit depth : 16 bits Stream size : 900 KiB (11%) Language : English Default : Yes Alternate group : 1 Other ID : 3 Type : Time code Format : QuickTime TC Duration : 4 s 800 ms Time code of first frame : 02:02:03:03 Time code, striped : Yes Language : English Default : No In theory, new-ish ffmpeg should support writing those in mp4, too: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/b4dcd351ec50caaa484bc5c66b4a8d5557a0f1ea committer Michael Niedermayer <[email protected]> Sun, 28 Feb 2016 05:47:23 +0300 (03:47 +0100) commit b4dcd351ec50caaa484bc5c66b4a8d5557a0f1ea movenc: Timecode in MP4 Although MP4 does not have a concrete specification to store timecode information, the following technical note from Apple describes a way to achieve this via timecode track, similar to how it is done for MOV files. https://developer.apple.com/library/mac/technotes/tn2174/_index.html - Enabled creation of timecode tracks for MP4 in the same way as MOV. - Used nmhd as media information header of timecode track of MP4 instead of gmhd used in MOV, thus avoiding tcmi also, as recommended above. - Bypassed adding source reference field for MP4, as suggested above. Issue: https://trac.ffmpeg.org/ticket/4704 ---- but not tested by me personally, due to old system ffmpeg (may be I'll patch it) Additionally, I found rtmd track timecode was supported in newer ffmpeg: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/d693392886b8454c818e384c816b9ede53c570d8 Mon, 27 Jun 2016 11:27:25 +0300 (10:27 +0200) commit d693392886b8454c818e384c816b9ede53c570d8 avformat/mov: parse rtmd track timecode so, there is no need to mess with mp4v2 utilites ..... Hopefully this will provide some way to test changes in CinGG without hunting all those sample files, initially? But I'll continue to look around for real ones .... |
|
|
@Andrew-R There's no question that GOPs are time_stamped_ when encoded, among other things it's how the decoder ensures that it's displaying the correct frames in the correct order. But like I said, normally those just start at 0 and work up from there — which is exactly what your mp4file dump shows. To obtain timecode useful for synchronizing separate files, there would need to be some sort of field that contains, as that one user said in the discussion about which clock to use, "any field which is different from 0:0:0:0". (Or something monotonically increasing from 0 based on the playback rate, like all of the timecodes displayed in that dump.) Ah! Here we go, yes. That C0001.MP4 file does contain useful timecode data, in a "Timed Metadata" RTMD (real-time metadata) stream. Though, the last line of output being an error message about the "unsupported codec" for the RTMD stream makes me nervous about ffmpeg's ability to usefully _decode_ that stream. Still, it can clearly at least extract the per-*stream* metadata, including the stored external-clock reference timecode for the start of recording: $ ffprobe -i C0001.MP4 ffprobe version 4.2.3 Copyright (c) 2007-2020 the FFmpeg developers built with gcc 10 (GCC) configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' ' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librsvg --enable-libsrt --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzimg --enable-libzvbi --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-libmfx --enable-runtime-cpudetect libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561ea444a440] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1001 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x561ea444a440] st: 0 edit list 1 Cannot find an index entry before timestamp: 1001. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C0001.MP4': Metadata: major_brand : XAVC minor_version : 16785407 compatible_brands: XAVCmp42iso2 creation_time : 2017-02-12T01:31:28.000000Z Duration: 00:00:49.05, start: 0.000000, bitrate: 97074 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 3840x2160 [SAR 1:1 DAR 16:9], 95289 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default) Metadata: creation_time : 2017-02-12T01:31:28.000000Z handler_name : Video Media Handler encoder : AVC Coding Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2017-02-12T01:31:28.000000Z handler_name : Sound Media Handler Stream #0:2(und): Data: none (rtmd / 0x646D7472), 245 kb/s (default) Metadata: creation_time : 2017-02-12T01:31:28.000000Z handler_name : Timed Metadata Media Handler timecode : 83:01:01;02 Unsupported codec with id 0 for input stream 2 |
|
|
Update on mp4v2 utils, I entered wrong command, now after installing new Slackware package (https://slackbuilds.org/repository/14.2/libraries/libmp4v2/ - this one lists https://github.com/sergiomb2/libmp4v2 as source) and typing mp4trackdump -t 3 C0126.MP4 I see list of timecodes (?): mp4trackdump -t 3 C0126.MP4 mp4trackdump version -r mp4file C0126.MP4, track 3, samples 240, timescale 50000 sampleId 1, size 1024 duration 1000 time 0 00:00:00.000 S sampleId 2, size 1024 duration 1000 time 1000 00:00:00.020 S sampleId 3, size 1024 duration 1000 time 2000 00:00:00.040 S sampleId 4, size 1024 duration 1000 time 3000 00:00:00.060 S sampleId 5, size 1024 duration 1000 time 4000 00:00:00.080 S sampleId 6, size 1024 duration 1000 time 5000 00:00:00.100 S sampleId 7, size 1024 duration 1000 time 6000 00:00:00.120 S sampleId 8, size 1024 duration 1000 time 7000 00:00:00.140 S sampleId 9, size 1024 duration 1000 time 8000 00:00:00.160 S sampleId 10, size 1024 duration 1000 time 9000 00:00:00.180 S |
|
|
More XAVC-S findings: https://github.com/SK-Hardwired/xavc_rtmd2srt XAVC S video real-time meta-data parser and GPS data extractor Extract realtime meta-data (i.e. per-frame) from Sony XAVC video and render to srt subtitle file. Written in python 3.7 More timecode woes from Lightswork user forum: https://www.lwks.com/index.php?option=com_kunena&func=view&catid=12&id=131106&Itemid=81#ftop ----quote---- David Rasberry Pro User OFFLINE Posts: 2658 3 years, 2 months ago #145410 hugly wrote: My personal favorite for amateur usage is still MPEG2 I-Frame codec. High transcoding speeds, reasonable file sizes and bitrates, good quality, fluently editable, support of almost any resolution and frame rate, no licensing problems, playable in any media player. What more do I (amateur) need? By the way, jpaulbiss footage (and other AVCHD/PCM.MOV camera recorded footage I tested) has a valid timecode track containing SMPTE compliant start T/C in Quicktime format. With ProRes/PCM in a MOV container, Lightworks will read the Quicktime timecode track properly, with AVCHD present it doesn't (T/C zero after import). I assume there are two different demuxers at work for MOV containers. From my point of view this is a major problem because AVCHD in MOV container is a commonly used recording format. All this isn’t new, but from my point of view this issue should be fixed soon. I looked into this sometime back comparing a Prores LT.mov compressed file with AVCHD i-frame compressed file in .mov format. Both .mov files have the same set of TC fields in the file header, which is where the various clock field values for different start frame clocks are registered. There is a specific TC field that is populated in the ProRes file that translates the relevant clock code to SMPTE format. That field is not populated in any H.264 AVC variant from most low end cameras. There are separate clock fields for shot start 00:00:00:00, time of day clock if present, and free run clock from camera startup. Any of these can be used for the SMPTE TC field depending on camera settings and whether the camera firmware sets the SMPTE field or not. Lightworks only reads the SMPTE reference field. QT timecode is not automatically SMPTE compliant, but can be. If you ask Lightworks to read something other than the SMPTE field, which of the 5 or so clock fields present is it to pick from? ---quote end-- On this user suggested: >which of the 5 or so clock fields present is it to pick from? I'd say any field which is different from 0:0:0:0 would be better than none ,-) ---quote end---- So, may be CinGG can reuse same startegy - search for first non-all-zeros timecode, and display/use it, at least initially ? |
|
|
some more files in XAVC-S format (?) found at forum: https://forum.videohelp.com/threads/371906-XAVC-S-How-to-export-time-date-metadata-to-subtitles-(srt) Not sure, my mp4extract fails to extract metadata track ... mp4v2 utility apparently moved to https://github.com/TechSmith/mp4v2 ? format of this track posted as: ---quote--- This is the format of the RTMD stream for those that don't want to DL it mp4file C0126.MP4, track 3, samples 240, timescale 50000 sampleId 1, size 1024 duration 1000 time 0 00:00:00.000 S sampleId 2, size 1024 duration 1000 time 1000 00:00:00.020 S sampleId 3, size 1024 duration 1000 time 2000 00:00:00.040 S sampleId 4, size 1024 duration 1000 time 3000 00:00:00.060 S sampleId 5, size 1024 duration 1000 time 4000 00:00:00.080 S sampleId 6, size 1024 duration 1000 time 5000 00:00:00.100 S sampleId 7, size 1024 duration 1000 time 6000 00:00:00.120 S sampleId 8, size 1024 duration 1000 time 7000 00:00:00.140 S sampleId 9, size 1024 duration 1000 time 8000 00:00:00.160 S ---quote end---- There is also ffmpeg ticket about wrong timecode extraction, currently downloading bigger file https://trac.ffmpeg.org/ticket/6793 wget http://50.19.4.114/bugs/ffmpeg/C0001.MP4 a bit slow ... |
|
|
More timecode discussion (in relation to proprietary editors): https://www.vegascreativesoftware.info/us/forum/date-created-to-timecode-in-or-date-time-stamp--112646/ ----quote---- doubleJ wrote on 9/1/2018, 3:55 PM So, on an even wilder front... I downloaded a bunch of sample footage (R3D, XAVC, XDCAM, DNxHD, DV, ProRes, maybe others). I imported 17 video files into Vegas Pro 16 and Premiere Pro CC 2018. Of all the files, only 2 showed timecode in both Vegas and Premiere (and that was an old DV AVI). There were 8 files with timecode in Vegas and 5 files with timecode in Premiere. It's likely that some of the other files simply didn't have timecode, but that's a pretty narrow margin. How is a person supposed to make "standards-compliant" files? Hehehe... Going back to my initial topic, though... If I was going to transcode video into one of these formats that does work, would I be able to insert timecode during that process? Maybe if I would use FFMPEG, TMPGEnc, or VirtualDubMod? JJ --- doubleJ wrote on 9/2/2018, 6:43 PM Ok, I'm making some head-way... This just adds some metadata and it shows the timecode in both NLEs. ffmpeg.exe -i originalfile.mp4 -metadata timecode="14:36:08:29" -metadata creation_time="2018-08-05 14:36:08" -c copy newfile.mp4 input file timecode key and value in metadata creation date/time metadata (this only seems to show in File Explorer properties details) don’t do any encoding (just copy the video/audio) output file This does the same thing, but transcodes to the Avid DNxHD intermediate format (mxf didn’t open in Vegas, so I used mov). ffmpeg.exe -i originalfile.mp4 -metadata timecode="14:36:08:29" -metadata creation_time="2018-08-05 14:36:08" -c:v dnxhd -b:v 145M -s 1920x1080 -r 30 -c:a pcm_s16le newfile.mov I also tried ProRes, but Vegas Pro didn’t see the timecode. I haven't been able to find ffmpeg information on encoding to CineForm. So, now the work is to create a batch file that will read the Date Created file attribute and use it for timecode and creation_time. I downloaded a batch script that reads a directory and echoes Date Created and Time Created, so I'm hopeful. JJ ----quote end---- I don't know where exactly you can find mentioned source files (probably some camera sample file repo?) pdf remotely related to this problem: https://mediaarea.net/Events/2018-10-26_NoTimeToWait3/presentations/19. Ben Turkus, Kelly Haydon - If we could turn back timecode/nttw3_timecode_final.pdf (with spaces in URL) Both files here should contain some 'timecode breaks' , or at least something Kino can recognize, so one file shows up as bunch of clips): https://yadi.sk/d/I0jFMoL9Tg5V0w |
|
|
I think you can try some old DV samples - old Cin codebase apparently used libdv's interface to such data, but for ffmpeg's APIs testing/prototyping you _probably_ can reuse some of those files (AVI container apparently lack timecode track possibility, but codec-level timecode from camcoder still around). Let me know if you need some HUGE (2Gb+) samples, via cloud.mail.ru or Yandex disk. I think some of my 'research' still lives at https://www.cinelerra-gg.org/bugtracker/view.php?id=332 and ..somewhere else (still searching for it) |
|
|
@Andrea - your reference is helpful to me as I think this is what GG has been trying to explain to me for years. This article makes it a lot clearer than he does. @ferdnyc Thanks for the hints again. He is now talking about time since 1970, so that may be the current approach as you suggested in: "That's defined as the "Start time of the stream in real world time, in microseconds since the Unix epoch (00:00 1st January 1970).", so if there were going to be an externally-referenced timecode value, that's likely where it would be stored (or it would be relative to that field)." His plan is to write code to do the work with the idea that what he actually uses for the "timecode" synch may have to change if he ever gets example media. |
|
|
I don't know if this will help: http://dranger.com/ffmpeg/tutorial05.html |
|
|
gg writes: I have identified several sources of timecode data that can be considered for use. There is still quite a lot of discovery needed before a good implementation may be started. The design can be broken into two program sets. 1) get_initial_timecode(edit) 2) align armed tracks on mwindow timeline get_initial_timecode should access the edit asset media, determine the program set that contains that edit's video layer or audio channel, determine the (video?) master layer, and read the media for pts (presentation timestamp) and timecode data. Once both (or equivalent) data are available, the apparent timecode of the initial frame/sample should be returned. This should make it possible to add a new edit popup menuitem to set the main window timebar (in timecode hh:mm:ss:ff), by using the session frame rate as a time standard. A new menubar menuitem, align armed tracks on mwindow timeline, can then iterate over a set of selected edits, and similarly determine the edit timecode offset, then adjust the edit startproject or startsource position to align the edit with the main window timeline timecodes. Each file codec can potentially provide timecode information. So far, I have looked into fileffmpeg and filempeg, but I suspect that there is more to follow. ffmpeg has at least 5 sources of timecode data that are present in various formulation. 1) AVFormatContext metadata "timecode" 2) AVStream metadata "timecode" 3) AVFrame metadata "timecode" 4) AVFrameSideData type AV_FRAME_DATA_GOP_TIMECODE 5) AVFrameSideData type AV_FRAME_DATA_S12M_TIMECODE filempeg has GOP (group of pictures) smtpe timecodes that are only present in the video start_gop headers. What to do when there are no usable timecodes is still an open question. Audio tracks may not have any timecodes and may not belong to a program group. All of this said, I definitely will need an example set of media data to do testing. I do not have any such thing, and that is a show stopper. Any suggestions at this point are welcome, and any example media would be greatly appreciated. gg |
|
|
The patches referenced by @Andrew-R will add internal timecode information to the metadata of various codecs. But for the most part, FFmpeg's internal timecode is relative to the start of the data stream. So it's not clear to me whether it would have a format that's real-world synchronizable. The patchset's updated unit test for frame side data (something that already exists, even without those patches) uses 0-relative timecodes, which makes me suspect that it would not: http://ffmpeg.org/pipermail/ffmpeg-devel/2020-June/265036.html There's also the fact that timecode_frame_start in the codec context is a deprecated attribute: https://www.ffmpeg.org/doxygen/4.1/structAVCodecContext.html#a98fefd16c6f7eaaaca1aadf80b8a9453 That makes the whole thing sort of weird, TBH. One *potential* for accessing external timecode data is the AVFormatContext attribute start_time_realtime: https://www.ffmpeg.org/doxygen/4.1/structAVFormatContext.html#aa5ddb5cee1df28f21739133f2e37f1c5 That's defined as the "Start time of the stream in real world time, in microseconds since the Unix epoch (00:00 1st January 1970).", so if there were going to be an externally-referenced timecode value, that's likely where it would be stored (or it would be relative to that field). But I'm not sure under what conditions, or if ever, start_time_realtime will be set for streams decoded from files. It sounds like it's primarily intended as an attribute for realtime streamed data (like internet livestreams, for example). It's also entirely possible that timecode_frame_start WILL be set relative to an external timecode, if one is stored in the file's metadata. I agree that sample files would be extremely useful, since few of us ever have occasion to work with files that use an external clock reference. |
|
|
@Katacuts GG started an investigation on "timecodes" today. It is not at all clear which timecode in the file is the one to use -- there are 4 different things that are referred to as a timecode. Quite confusing. In order to further investigate, it would be helpful if you could send a set of at least 3 files that belong to a set to be synched up so he can figure out which timecode is relevant. Any additional information for clarification would be highly appreciated as we are computer people and not A/V technicians. Unfortunately without more technical information and a set of valid test media, we are kind of stuck on whether or not this can even be implemented by us. If you can put the files in a dropbox or shared driver and email [email protected] we will keep them private. @Andrew-R THANKS for the ffmpeg patch hints as it provided a starting place for GG to look. |
|
|
More timecode patches about to hit (hopefully) ffmpeg 4.4: http://ffmpeg.org/pipermail/ffmpeg-devel/2020-July/265560.html [[FFmpeg-devel] [PATCH 1/4] avcodec/hevcdec: add timecode metadata] and others ... http://ffmpeg.org/pipermail/ffmpeg-devel/2020-July/265563.html [FFmpeg-devel] [PATCH 3/4] avcodec/h264_slice: add timecode metadata with commit message saying: "Please test with below command: ./ffplay -vf drawtext="fontfile=/Library/Fonts/Arial.ttf:text=\\'%{metadata\\:timecode}\\'" \ ../fate-suite/h264/crew_cif_timecode-2.h264" I guess developer was working from Mac ? also as patch series: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=1664 |
|
|
Not sure if 'timecode' here and timecode in those patches talk about same thing, but I spotted those two patches on ffmpeg-devel: http://ffmpeg.org/pipermail/ffmpeg-devel/2020-June/264622.html [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_sei: support HEVC timecode decode [FFmpeg-devel] [PATCH 2/2] avcodec/hevcdec: create AVFrame side data from HEVC timecodes like H.264 |
|
|
@Andrea_Paz Will you be adding your video to the YouTube channel? It is a good one! |
|
|
@Katacuts Just some feedback. First of all, there is nothing preventing the inclusion of this capability, EXCEPT development time which is in very short supply. About "productions with hundreds of hours of footage" -- I am not certain that using Cinelerra for this much is really do-able. About "workflow described here" -- thank you for that reference as it gives me a good idea of expectations. About "if it is hard to program timecode synchronization? ...Has it been attempted before" -- a lot of work was done initially (like 10 years ago) to synchronize the video and audio, and I believe at the time trying to synchronize via timecode was fraught with varying amounts of missing timestamps. So synchronizing via waveform might be easier. However, there are actually 2 things that prevent going forward with an effort to provide this feature. As already mentioned lack of development time pretty much stops the whole thing. The second thing is BT 0000433, part 2, for treating stereo (or multiple audio tracks) as a single entity. There needs to be a whole new container designed to fit and then implemented somehow or other, perhaps using program streams. We have been stuck on coming up with a method to do this for a couple of months now although we discuss it almost daily. Maybe if we can get past 0000433, it will be possible to work on this, but I do not see this happening any time soon. |
|
|
Try watching this video too, which shows the multicam workflow from start to finish: https://drive.google.com/file/d/1Gzf84dcjkFqGtoLbCsePyNakKdxK8lu8/view?usp=sharing It is not all automated, some steps remain in manual. In the end, we have to manually synchronize the external track to the one created with OpenMixers and replace it with this one. I understand that this is not suitable for a Pro environment. However I agree with you: Timecode support, and being able to synchronize audio/video via timecode, would be a nice addition. Let's see if it's possible for Good Guy (maybe the SMTPE timecode which is STD). |
|
|
Unfortunately I don't see this method workable for productions with hundreds of hours of footage. Also it happens quite often that the camera doesn't record any audio. This looks like a decent way to sync multicam footage but manually marking every in point of the clapperboard or sync points is very time consuming. Time is always a factor so usually the first step is to use the timecode or waveforms to batch sync everything up roughly. Then you go by every clip and check if there was a drift in the timecode or the wavesync didn't work and then adjust manually if needed. But the first batch sync does 90% of the work. The workflow described here is quite close to what I do : https://blog.frame.io/2017/05/15/sync-clips-in-davinci-resolve/ . I was wondering if it is hard to program timecode synchronization? Has it been attempted before or is there some proprietary component needed? I can find no program in the open source video editing world who has it. |
|
|
@Katacuts As Andrea stated, maybe this will work. An old demo that was created at the time the "align audio" was added in March 2019 is at: https://streamable.com/ln2e5h (Andrea can make a better one if needed since I am only a tester). The "align audio" implementation as not as elegant as it could be due to being a late add on instead of designed in. Each mixer is like a program stream and mixers is the method that creates that container. Let us know if this can work for you. |
|
|
If the video footage has its own audio track then you can synchronize the audio taken externally. See manual 14.1 and 14.1.1 (if you need it I can provide an example video). If instead you only have video tracks without audio plus the external audio track, then the only possibility is to do it manually (better with a clapperboard), since CinGG does not support audio/video sync via timecode. |
- Anonymous
