プログラミング

How to use FFmpeg.wasm. What’s the Cross Origin Isoration?

投稿日:

Nowadays, I got the information that FFmpeg can be used with Javascript and I tried it immediately. It is called FFmpeg.wasm. It seems that it was easy to use when it was first released, but now it seems to be a little difficult due to the influence of the usage limitation of “shared array buffer”. Though who work as a programmer understand the meaning of the information and respond to it, but this is not the case for those who are studying the program, including myself. I gathered information and managed to use FFmpeg.wasm, so I wrote this article.

SharedArrayBuffer repuires Cross-Origin-Isoration

I don’t know the details, but if I mention only that it is an obstacle to the use of FFmpeg.wasm…

“SharedArrayBuffer is used in FFmpeg.wasm, and in order to use SharedArrayBuffer, it must be an environment with cross-origin isolation.”

Also, this limitation seems to be from Chrome 91 for Chrome browsers.

https://developers-jp.googleblog.com/2021/03/chrome-91-sharedarraybuffer.html

So can I use it by reverting to an older version of Chrome? However, even if I can use it only by myself, I can not use it as a service, so I didn’t choose the way by lowering the Chrome version.

I don’t know the details about cross-origin isolation, but I understand that it is OK if I could make an isolated environment. And as Google said in the link above, it is necessary to send the header below on the web page.

Cross-Origin-Embedder-Policy: require-corp // COEP
Cross-Origin-Opener-Policy: same-origin   // COOP

Ahhh, I have no idea what it means. I don’t know where to write the above headers and where to set them. I just have to search web to find someone’s solution.

Seeking a simple solution

I found a lot of solutions by Google searching. However, many of them used Node.js. It seems that Node.js has a function that sends headers, so we can easily send the above two lines of headers.

But I don’t want to use Node.js. The reason is that I just want to try a little example, so I found it bothersome to use a service that can deploy apps using Node.js.

While searching, I found an article that PHP can set headers. I’ve dealt with PHP a little, so I think I can go with this. (It took me about an hour to remember that PHP is a server-side language …)

If I use the PHP method, I can try it only with my own domain, so it’s quite easy.

Write headers of COOP and COEP in PHP file:Source Code

If you search FFmpeg.wasm by Google, you will find many sample programs that worked before the sharedarraybuffer limitation was applied. Let me use the code in the article below.

https://zenn.dev/sugar/articles/ce971201435814b0350c

Since it says Chrome 86, this article was written before the limitaion.
The points I’ve improved are as follows.
1. Download ffmpeg.min.js, ffmpeg-core.js and ffmpeg-core.worker.js files and set at the same directiory of main program.
2. Reaname the sample program file index.html as index.php, then add the headers of COOP and COEP.

Here are my codes.

index.php

<?php
ini_set('mbstring.internal_encoding' , 'UTF-8');

header('Cross-Origin-Opener-Policy: same-origin');
header('Cross-Origin-Embedder-Policy: require-corp');
?>

<html>
<head>
  <script>
    if (crossOriginIsolated) {
      // Post SharedArrayBuffer
      console.log('crossOriginIsolated');
    } else {
      // Do something else
      console.log('NOT crossOriginIsolated');
    }
  </script>
  <script src="ffmpeg.min.js" ></script>
 </head>

  <body>
  <script src="app.js"></script>
  </body>
</html>

In the 11th line, check if cross-origin isolation is performed by crossOriginIsolated.
If you comment out the 4th and 5th lines, it will be NOT crossOriginIsolated.

app.js

(async () => {
  console.log('app.js');
  const { createFFmpeg } = FFmpeg

  function generateImages() {
    console.log('generateImages');
    const canvas = document.createElement('canvas')
    canvas.width = 320
    canvas.height = 240

    const ctx = canvas.getContext('2d')
    ctx.textBaseline = 'middle'
    ctx.textAlign = 'center'
    ctx.font = '64px serif'

    const arr = []

    for (let i = 0; i < 4; i++) {
      ctx.clearRect(0, 0, canvas.width, canvas.height)
      ctx.fillStyle = '#fff'
      ctx.fillRect(0, 0, canvas.width, canvas.height)
      ctx.fillStyle = '#000'
      ctx.fillText(i + 1, canvas.width / 2, canvas.height / 2)

      const dataUrl = canvas.toDataURL()
      arr.push(dataUrl)
    }

    return arr
  }

  async function generateVideo(images) {
    console.log('generateVideo');
    //const ffmpeg = createFFmpeg({ log: true })
    const ffmpeg = createFFmpeg({
      corePath: 'ffmpeg-core.js',
      log: true
    });

    await ffmpeg.load()

    images.forEach(async (image, i) => {
      await ffmpeg.write(`image${i}.png`, image)
    })

    await ffmpeg.run('-r 1 -i image%d.png -pix_fmt yuv420p output.mp4')
    const data = ffmpeg.read('output.mp4')
    return data
  }

  function createObjectUrl(array, options) {
    console.log('createObjectUrl');
    const blob = new Blob(array, options)
    const objectUrl = URL.createObjectURL(blob)
    return objectUrl
  }

  function insertVideo(src) {
    console.log('insertVideo');
    const video = document.createElement('video')
    video.controls = true

    video.onloadedmetadata = () => {
      document.body.appendChild(video)
    }

    video.src = src
  }

  const div = document.createElement('div')
  div.innerText = '動画生成中'
  document.body.appendChild(div)

  const images = generateImages()
  const video = await generateVideo(images)
  const objectUrl = createObjectUrl(, { type: 'video/mp4' })
  insertVideo(objectUrl)

  document.body.removeChild(div)
})()

Lines 35-38 are important. Since We used cross-origin isolation, we have to prepare FFmpeg only for our own domain. Thus, it cannot be used as downloading from the official page such as

”https://unpkg.com/@ffmpeg/ffmpeg@0.8.3/dist/ffmpeg.min.js”


Since it will go to refer to ffmpeg-core.js and ffmpeg-core.worker.js in “createFFmpeg”, we need to put all of files including ffmpeg.min.js in our own domain.

You can check the operation of the program here.

https://g-llc.co.jp/videoTest2.php

-プログラミング

執筆者:

関連記事

Android App: How to share video in external storage to MediaStore?

Code Android Studio Electric Eel | 2022.1.1 Patch 2[Java]String path = xxx; //path of the video in t …

Javascriptでまばたき検知 face-api.js landmarks

顔認識の技術の発展により、今日ではまばたきの検知まで容易に行えるようです。例えばPython、OpenCV、dlibを使用したものではhttps://www.pyimagesearch.com/201 …

no image

ショーモナイノ/ ソースコード(サーバーサイド)

ショーモナイノのコードを公開していないかとのお問い合わせを頂きました。GitHubでの公開を検討しましたが、書き散らかした粗末なコードをGitHubに置くべきではないと判断しました。代わりに自分のブロ …

Node.jsゲーム公開はHerokuが良い – 初めてのゲーム開発からデプロイまでの記録

ゲーム開発初心者が簡単なNode.jsオンラインゲームを作成して公開したい場合はHerokuで公開するのが良いです。Node.jsゲームの公開手段は様々ありますが、Herokuは無料プランで十分にテス …

2重振り子の数値シミュレーション、解析から結果描画まで[Octave / Matlab / Android iPhoneアプリもあるよ]

久々に数値解析をやってみたくなりました。題材としては多くの人がやっている二重振り子が面白そうです。では言語は何にしようかと迷うところですが、コードを書いて計算して結果を描画するまで可能な「Octave …

スポンサーリンク