Concat already recorded WebM files

[sebavan]

Hello,

It is definitely more a question than an issue :-)

I am wondering if it would be possible to use the same kind of setup to stitch and trim webm files together ?

I was trying to look into ffmpeg alternatives.

Thanks in advance

[beaufortfrancois]

@surma Is that something we could do here?

[guest271314]

@sebavan See https://github.com/guest271314/native-messaging-mkvmerge/blob/master/README.md

[sebavan]

Thx, the idea was more to do it in the browser from a wasm build of libVPX, as I currently repurposed ts-ebml to do the job :-)

[guest271314]

@sebavan That is the ultimate idea with mkvmerge https://discourse.wicg.io/t/port-mkvmerge-to-javascript/3841, see merge() description at https://discourse.wicg.io/t/offlinemediacontext/3814. Currently using Native Messaging at Chromium due to Native File System being implemented to write/read files to storage device as mkvmerge currently requires the files to be on disk https://gitlab.com/mbunkus/mkvtoolnix/issues/2576, https://gitlab.com/mbunkus/mkvtoolnix/issues/2582, https://www.reddit.com/r/mkvtoolnix/comments/cdi824/mkvmerge_prints_error_when_merging_webm_files/. Had the same notion re ts-ebml legokichi/ts-ebml#26.

What found:

• Tracks must match AV:AV. If A or V do not exist in the original container they must be created to concatenate
• Width and height must be considered, when encoding VP8 or VP9 if the same width and height are expected to be encoded into the file https://bugs.chromium.org/p/chromium/issues/detail?id=972470, though see last comment at https://bugs.chromium.org/p/webm/issues/detail?id=1642, which have not tried, yet. Without the patches applied have not been able to concatenate or re-encode videos resulting in correct width and height using ffmpeg or vpxenc
• Native File System can in fact perform the read/write operations, generally consistently, though the first request for permissions usually is not successful, is set to "prompt"
• It is tedious to determine track order and create an --append-to map
• At Chromium the only way that have been able to preserve width and height for a live stream is to use OpenH264 and save the file as .webm resulting in a non-standard file https://bugs.chromium.org/p/chromium/issues/detail?id=980822 playable at Chromium/Chrome though which means the file cannot currently be played back at Mozilla Firefox or Nightly https://plnkr.co/edit/Axkb8s?p=preview

And probably some notes that are not recollecting right now.

mkvmerge has the capabilities to merge WebM and Matroska files. That what the code was created for. Not sure if libvpx was in fact created for that purpose.

The only issue with using the linked Native Messaging approach is setup time and not being able to use the code at any URL other than chrome://<extension>. A full port of mkvmerge to JavaScript would require a fair amount of effort emscripten-core/emsdk#260, though should be possible, if the read/write operations are included in the port. An alternative approach that have been cosidering is to watch a directory using bash, write the file which contains the mkvmerge command and execute the program, similar to an IIFE in the form of a shell script WICG/file-system-access#72 (comment).

Am willing to help with the procedure (libvpx) as it appears there is ultimately the same or similar requirement. Would suggest to look into mkvmerge versions 36.0.0 and greater.

[guest271314]

@sebavan If you are interested am relatively certain the appendTo algorithm implementation at https://github.com/guest271314/native-messaging-mkvmerge/blob/master/app/native-messaging-mkvmerge.js#L205 through https://github.com/guest271314/native-messaging-mkvmerge/blob/master/app/native-messaging-mkvmerge.js#L228 could be improved and tested to be certain the correct track order is set before executed as an option to the mkvmerge command.

What approach are you currently using to concatenate or merge WebM and Matroska files?

[guest271314]

@sebavan

as I currently repurposed ts-ebml to do the job :-)

Can you post a link to the code?

[sebavan]

Unfortunately it is not OSS at the moment but basically I simply change the timestamp of each chunk to add the previous concatenated duration.

I hope those kind of simple edition tools will make it into a richer js api.

[guest271314]

@sebavan

Had the same concept. Though am not sure how to achieve that goal.

For audio it is possible to use Web Audio API decodeAudioData() and OfflineAudioContext() startRendering() method to merge discrete files into one file.

For encoded images within a video AFAIK there is not an existing method in JavaScript to extract all images from a file without playing the media.

See also https://discourse.wicg.io/t/webcodecs-proposal/3662/15.

[guest271314]

@sebavan

Re

For encoded images within a video AFAIK there is not an existing method in JavaScript to extract all images from a file without playing the media.

see https://libwebpjs.appspot.com/vp8/webm-javascript-decoder/.

FWIW Improved the implementation of mkvmerge --append-to option algorithm in JavaScript. Where given a list of media files having arbitrary track order in the container or only a single audio track or video track

https://upload.wikimedia.org/wikipedia/commons/6/6e/Micronesia_National_Anthem.ogg#t=0,2
https://upload.wikimedia.org/wikipedia/commons/a/a4/Xacti-AC8EX-Sample_video-001.ogv#t=0,4
https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=10,20
https://mirrors.creativecommons.org/movingimages/webm/ASharedCulture_480p.webm#t=22,26
https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=55,60
https://raw.githubusercontent.com/w3c/web-platform-tests/master/media-source/mp4/test.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerJoyrides.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerMeltdowns.mp4#t=0,6

creating the missing adjacent track then piping through MediaRecorder to get WebM files (VP8 codec as VP9 does not preserve original resolution of the video)

[
  {
    "audio": 1,
    "video": 0
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 1,
    "video": 0
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  }
]

note the order of indexes 1 and 3 are opposite from the remainding indexes this is the corresponding map

1:0:0:1,1:1:0:0,2:1:1:0,2:0:1:1,3:0:2:1,3:1:2:0,4:0:3:0,4:1:3:1,5:0:4:0,5:1:4:1,6:0:5:0,6:1:5:1,7:0:6:0,7:1:6:1,8:0:7:0,8:1:7:1

the resulting file https://github.com/guest271314/native-messaging-mkvmerge/blob/master/native-messaging-mkvmerge-vp8.webm?raw=true, plnkr https://plnkr.co/edit/8J61Rw?p=preview.

[guest271314]

@sebavan https://discourse.wicg.io/t/webmediacontainers-proposal/3928

[rahulbansal16]

Unfortunately it is not OSS at the moment but basically I simply change the timestamp of each chunk to add the previous concatenated duration.

Hi @sebavan can you provide more details or a sample snippet. I am trying to trim the first few seconds of the video obtained from the Chrome Media API. I am trying to understand How the clusters element are structured and how will the resultant metadata looks like.