 |
Some times strange things just happen for no reason. @Andrew-R Thanks for the bugzilla url reference to libvpx bug. We will wait until a newer release to upgrade since it was only a point upgrade from 1.8.1 to 1.8.2. We were only attempting to get all libraries upgraded at once. |
 |
Since the 2020-03 release it builds fine for me. I don't think there were any relevant system updates apart from GCC, but I don't think that was the problem. I'm at a loss to explain why it did not build after dav1d was rolled back. Anyway, all fine now. |
 |
something like this apparently works, at least build continues into plugins .....
c11_configure.ac_from_oxr_1.patch (1,858 bytes) |
|
|
oh, this still doesn't work yet, but may be because in cinelerra/Makefile I see CFLAGS := \ -I/usr/local/include \ -I$(GUICAST) \ -I$(LIBZMPEG3) \ $(static_incs) \ $(CFLAGS) [...] $(shell echo $(CFLAGS) > $(OBJDIR)/c_flags) and $(OBJDIR)/c_flags used for building essentially C++ files, and one of them (fileexr) references c++ library, compiled with different set of CXXFLAGS .... even if updated configure patch (attached, with hack, currently compiling) set CXXFLAGS correctly, or at least in same way as OpenEXR?
c11_configure.ac_from_oxr.patch (1,618 bytes) |
|
|
from pkg description - just updated my autoconf-archive into 2019 version (from late 2015) PACKAGE DESCRIPTION: # autoconf-archive (GNU Autoconf Macro Archive) # # The Autoconf Macro Archive aims to provide a collection of reusable # Autoconf macros as free software. The archive currently features more # than 500 macros which perform portability tests ranging from compiler # support for weird language extensions to automatic generation of # sophisticated Automake rules. # # http://www.gnu.org/software/autoconf-archive/ still, apparently some Ubuntu failed to provide this archive .. then ppl complained about autotools being broken, and switched to cmake :} joke ...... https://github.com/SFTtech/nyan/issues/6 ...not sure if I should smile or not. probably not. Also, libvpx was giving some ..fun to Mozilla, too! https://bugzilla.mozilla.org/show_bug.cgi?id=1525393 |
|
|
Ah, there is some code in OpenEXR's configure.ac: dnl Enable and choose c++ standard AC_ARG_ENABLE(cxxstd, AC_HELP_STRING([--enable-cxxstd=14], [enable ISO c++ standard 11/14 [[default=auto]]]), [cxxstd="${enableval}"], [cxxstd=14]) if test "${cxxstd}" = 17 ; then AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory]) <------>CXXFLAGS="$CXXFLAGS -std=c++17" else if test "${cxxstd}" = 14 ; then AX_CXX_COMPILE_STDCXX([14], [noext], [mandatory]) <---->CXXFLAGS="$CXXFLAGS -std=c++14" else if test "${cxxstd}" = 11 ; then AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory]) CXXFLAGS="$CXXFLAGS -std=c++11" else if test "${cxxstd}" = 03 ; then CXXFLAGS="$CXXFLAGS -std=c++03" else dnl automatically determine... AX_CXX_COMPILE_STDCXX([11], [noext], [optional]) AX_CXX_COMPILE_STDCXX([14], [noext], [optional]) AX_CXX_COMPILE_STDCXX([17], [noext], [optional]) if test "$HAVE_CXX14" = 1 ; then <----> CXXFLAGS="$CXXFLAGS -std=c++14" cxxstd = 14 else if test "$HAVE_CXX11" = 1 ; then <----> CXXFLAGS="$CXXFLAGS -std=c++11" cxxstd = 11 fi fi fi fi fi fi :} still, it detects c++14 on my gcc 5.5.0 (and this ver. of gcc basically forced on me due to all those kernel vulnerabilities from 2018) and thus build lib in c++11 mode at very least ? I tried to add some elementary c++ level checking in Cin's configure.ac, see patch, it works with m4 1.4.17 (from 2013), currently upgraded to latest m4 1.4.18 (for some reason gnu site lists latest as m4-latest.tar.bz2 2013-09-22 03:08 1.4M while in reality latest is [ ] m4-1.4.18.tar.bz2 2016-12-31 17:39 1.4M - see at https://ftp.gnu.org/gnu/m4/ )
c11_configure.ac.patch (394 bytes) |
|
|
Updated version, still it must be tweaked, for example lame also has dependency on nasm (I currently uninstalled nasm completely)
nasm_thirdparty-2.patch (3,941 bytes) |
|
|
something like this (completely untested)
nasm_thirdparty.patch (3,719 bytes) |
|
|
ok, libvpx version + 0.0.1 doesn't sounds very exiting :} but missing asm sounds as potential slowdown on all machines without nasm > 2.13 (because x264/x265/ffmpeg all use it ?) I tried to add this to configure.ac, in addition to other lines: echo "export PATH=`realpath $(echo $PWD)`/thirdparty/nasm-2.14.02:$PATH" now I have line in global_config [a lot of line skipped] export thirdparty_libraries libraries export CFLAGS_ CXXFLAGS_ LDFLAGS_ unexport CFLAGS CXXFLAGS LDFLAGS export PATH=/dev/shm/tmp/cinelerra-goodguy-20200321/cinelerra-5.1/thirdparty/nasm-2.14.02:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/X11R7/bin:/usr/games:/usr/lib/java/bin:/bin: so, will it work or not ..... sorry, currently my thirdparty hacks a bit of mess, because they try to deal with few different issues at one go ..:/ |
|
|
I completely forgot to mention that last month when upgrading the libraries that are listed int the file cinelerra_path/thirdparty/downloads.txt, libvpx currently at version1.8.1 had a problem at 1.8.2 so it was not upgraded. No analysis was done on that at all - it did compile on Fedora but when I loaded a test file, it did not correctly load so we just dropped it. |
|
|
I tried to add nasm to thirdparty buildsystem, in thirdparty/Makefile diff --git a/cinelerra-5.1/thirdparty/Makefile b/cinelerra-5.1/thirdparty/Makefile index 0c595ee..f2b89ac 100644 --- a/cinelerra-5.1/thirdparty/Makefile +++ b/cinelerra-5.1/thirdparty/Makefile @@ -234,6 +234,7 @@ libtheora.ldflags?="$(call ld_path,libvorbis,lib/.libs) $(call ld_path,libogg,sr libtheora.cfg_params?= --disable-examples --disable-spec --enable-shared=no libuuid.cfg_params?=--enable-shared=no libvorbis.cfg_params?= --disable-oggtest --enable-shared=no +nasm.cfg_params?= --disable-doc openjpeg.cfg_params?= -DBUILD_SHARED_LIBS:BOOL=OFF openjpeg.cfg_vars?=$(call cmake_config,.) openjpeg.mak_params?= ; cd $(call bld_path,openjpeg,src/lib/openjp2); ln -sf . openjpeg-2.1 @@ -248,13 +249,14 @@ tiff.cfg_params+= --enable-shared=no --disable-zstd $(call if_pkg,libwebp,\ --with-webp-lib-dir=$(call pkg_libs,libwebp)) twolame.cfg_params?=--enable-shared=no x264.cfg_params?= --enable-static --enable-pic -x265.cfg_vars?=$(call cmake_config,source) +#x265.cfg_vars?=$(call cmake_config,source) x265.cfg_params?= -DENABLE_SHARED=no libvpx.cfg_params?= --enable-pic DS:=$$$$$$$$ pkg_cfg=$(call bld_path,$(1),usr/local/lib/pkgconfig): LV2_PKGCFG:=$(subst : ,:,$(foreach p,lv2 serd sord sratom lilv suil,$(call pkg_cfg,$(p)))) +LV2_PKGCFG:=+ $(PKG_CONFIG_PATH) # lv2 define waf-setup @@ -313,13 +315,13 @@ rules=$(info $(1)) else rules=$(eval $(1)) endif - +$(call rules,$(call std-build,nasm)) $(call rules,$(call std-build,a52dec,djbfft)) $(call rules,$(call std-build,djbfft)) $(call rules,$(call std-build,audiofile)) $(call rules,$(call std-build,encore)) $(call rules,$(call std-build,esound,audiofile)) in configure.ac diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac index f61ef27..edc569a 100644 --- a/cinelerra-5.1/configure.ac +++ b/cinelerra-5.1/configure.ac @@ -363,6 +363,11 @@ PKG_3RD([twolame],[auto], [ libtwolame/.libs/libtwolame.a ], [ libtwolame ]) +PKG_3RD([nasm],[yes], + [nasm-2.14.02], + [ ], + [ . ]) + PKG_3RD([x264],[auto], [x264-snapshot-20191217-2245-stable], [ libx264.a ], @@ -373,7 +378,7 @@ PKG_3RD([x265],[auto], [ libx265.a ], [ . source ]) -PKG_3RD([libvpx],[auto], +PKG_3RD([libvpx],[yes], [libvpx-1.8.1], [ libvpx.a ], [ . ]) @@ -908,6 +913,7 @@ PKG_PROVIDE([libtheora]) PKG_PROVIDE([libuuid]) PKG_PROVIDE([libvorbis]) PKG_PROVIDE([mjpegtools]) +PKG_PROVIDE([nasm]) PKG_PROVIDE([libaom]) PKG_PROVIDE([dav1d]) PKG_PROVIDE([libwebp]) sorry this libvpx thing sneaked in again .... and nasm-2.14.02.tar.xz in thirdparty/src but how to add newly-build nasm executable to PATH ? |
|
|
Sorry about failing builds. May be at some point in the future at least some libraries will re-add this backward compatibility. In the meantime re-using already existing download mechanism (used for ffmpeg/OpenCV) will be helpful in my case .... (but this will require some conditionals and two or more set of patches). |
|
|
So today I am doing a production test build. Looks like the latest version of openexr is broken, and dav1d is also so do not make the cut. It is important to realize that upgrading is a double edge sword. It does bring fixes and adds vitality to the contemporary state of the library series. Frequently, the thirdparty build exceeds the vendor offerings, Particularly with high use visibility packages like ffmpeg. But it also bring risks. When a build fails, it may involve restarting the entire series, after the mod freeze, and with almost no time to test anything. This is not a great idea when you are looking a bug in a console log, and the rest of the builds are still chugging along. What do you do? Almost without exception, an emergency fix is added which removes the cause of the problem, usually by dropping a mod. This consequence is almost always an untested result. What to do, what to do... When a package "does not play well with others", I usually freeze it. OpenCV has been frozen ever since they dropped sift and surf in a sort of bait and switch source bases. dav1d is also in peril since they make no effort to provide usable source on the systems available in the community. The failing instruction seems to be only used a s "shift". There are no other ways to do this which work for the mortal man? say like divide, or what about a regular shift? Almost all code that has historical relevance has some sort of conditioning to provide historical context. I am sort of expecting that dav1d is also about to be a "bait and switch", probably with the encoder. This makes it much less likely to be carried, and is against the "open source" policy that they use as bait. |
|
|
> IMHO it would be very bad to make it dependent on some advanced tools (like the latest and greatest version of nasm) that are not supported on the platforms. well, this is true for GCC/libstdc++ itself, because they have huge, system-wide impact. But nasm itself small enough for updating. > Just being able to build it one one site and just distributes the binaries is not what we should aim for I think, quite the contrary. I agree, just I usually find some possible solution AFTER I speak out some idea .... see https://www.mail-archive.com/[email protected]/msg01170.html (" AVX512 asm instructions - idea for configure.ac") and https://www.mail-archive.com/[email protected]/msg01172.html ("stdc++11 detecion via autotools") - damn, typo! Should be detection! >> a .vdi size of between 10 and 15G per distro .. mmm ... sorry, I should buy external hdd again :} LANG=C df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 5.8G 1.1M 5.8G 1% /dev /dev/sda2 9.8G 9.4G 0 100% / /dev/sda3 269G 269G 70M 100% /home /dev/loop0 177M 177M 0 100% /opt/libreoffice-5 none 5.8G 5.8G 25M 100% /dev/shm you see, not a whole lot of space ... but, this is silly excuse. Will fix it later ... hopefully this month? > If you find a fix for a bug or new feature, build CinGG with that on at least one CinGG supported other platform, to make sure you catch build problems. Yeah, and from experience I better to try on OLD distro, because new ones usually have more eyes on it .... |
|
|
@Andrew-R, I applaud you enthusiasm to improve CinGG. However, I feel there are some important things to keep in mind. - CinGG is very stable, and it should be a goal not to lower it. It is better to have less features in a stable program than nice bells-and-whistles in one that crashes. CinGG has a reputation to lose! - CinGG is available for many platforms, and can be built on many. IMHO it would be very bad to make it dependent on some advanced tools (like the latest and greatest version of nasm) that are not supported on the platforms. Just being able to build it one one site and just distributes the binaries is not what we should aim for I think, quite the contrary. - I keep a set of VirtualBox VMs with different Linux distros just for testing. I don't know how much space you have, but using dynamic disks and not installing more than is needed, then after installing CinGG build tools and compiling it, I end up having a .vdi size of between 10 and 15G per distro (with Mint XFCE so far the smallest). Maybe you can set one up for a CinGG supported customer friendly 64 bit distribution, like Ubuntu, or Mint? I normally use Mint XFCE. And I do not upgrade my VMs with levels of software beyond what the distro has, otherwise it isn't a good representation of what is in the field, I only use the distro's package manager. - If you find a fix for a bug or new feature, build CinGG with that on at least one CinGG supported other platform, to make sure you catch build problems. |
|
|
I think configure (in CinGG) option is "--enable-dav1d", so for disabling it you need "--disable-dav1d". You can go dirty way and install locally-built nasm in /usr/local/bin (or even in your home dir, just make sure PATH variable is updated). I think because main speed of libdav1d comes from assembler language fragments - their reliance on new nasm understandable .... |
|
|
I see .. sorry, it was my idea to upgrade dav1d.. On Slackware it was easy to upgrade nasm. It should be relatively safe to upgrade nasm alone on buildfarm systems (unlike, say, gcc/libstdc++ itself ..). May eb I really should try and do some 'admin' work there? (only thing I remotely good at is building and rebuilding stuff ..until it, hopefully, works). But I don't have GPG key or some known good reputation.... and anyway, updating build systems probably best to try AFTER monthly rebuild finished successfully. |
|
|
Looks like you found the CDEF_FILTER_4x4 problem. We will back out 0.6 in the next 20 minutes in GIT. GG writes below: The first complaint from nasm seems to be: src/x86/cdef.asm:1902: error: parser: instruction expected src/x86/cdef.asm:1894: ... from macro `CDEF_FILTER_4x4_PRI' defined here which is: vgf2p8affineqb m9, m2, [r3+r6*8] {1to8}, 0 ; abs(diff) >> shift You have to admit, anybody who embeds the word "affine" in an instruction is probably already in trouble... yep, it seems to be an actual instruction, not a macro. The dav1d guys seem to have no regard for whether or not it may work in the field. I sent them a makefile that prevents the problems it already has with meson, and they rejected it. Apparently working on your machine is not a goal for them. It also seems that gitlab now require credentials to just access the current source. Seems like the whole idea of opensource is not important and is being factored out of this code base. I think this product just went static, and will not be upgraded. It has no effective encoder, and is only available on 4 of 18 platforms. Sorry to hear that. |
|
|
I think that the assembler complains about instruction "vpcmpub", the first one in macro CDEF_FILTER_4x4_PRI . That instruction is I think only available for AVX512 (Intel). Is there not a configuration option to not used AVX512? |
|
|
Attached is the NASM levels from our build computer using xen.
NASM_levels.txt (2,351 bytes) |
|
|
The 0.6 version of dav1d requires NASM 2.14. We are backing down to 0.5 and will stay at that version. It is ridiculous that the dav1d developers do not care if it works on Mint 19.3 which is the current Mint. The only distros that we build that are at 2.14 are the latest versions Arch, Debian, Gentoo, Tumbleweed, and Fedora. The rest are at 2.12 and 2.13 including the most widely used Ubuntu. |
|
|
Good to get this early warning. The problem appears to be the dav1d upgrade from 0.5 to 0.6 checked into GIT on March 12 with a macro. GG is looking at it now. As a temporary workaround I think here is a parameter in the build to script to build without dav1d (but I did not look it up). It is getting more and more difficult to keep some of these thirdparty libraries up to date as the developers switch from tried and true standard tools to newer, less standard tools. For example OpenEXR is the worse, OpenCV is not really "Open" and dav1d WITHOUT providing any new capabilities. We are going to re-evaluate and "if it is not broken and provides nothing new, we are not going to change it so we can concentrate on Cinelerra code instead. However, in the case of dav1d, the speed up gains in version 0.6 would be good to have. |
|
|
log (272,771 bytes) |
- Anonymous
