Nowadays, I got the information that FFmpeg can be used with Javascript and I tried it immediately. It is called FFmpeg.wasm. It seems that it was easy to use when it was first released, but now it seems to be a little difficult due to the influence of the usage limitation of “shared array buffer”. Though who work as a programmer understand the meaning of the information and respond to it, but this is not the case for those who are studying the program, including myself. I gathered information and managed to use FFmpeg.wasm, so I wrote this article.
I don’t know the details, but if I mention only that it is an obstacle to the use of FFmpeg.wasm…
“SharedArrayBuffer is used in FFmpeg.wasm, and in order to use SharedArrayBuffer, it must be an environment with cross-origin isolation.”
Also, this limitation seems to be from Chrome 91 for Chrome browsers.
https://developers-jp.googleblog.com/2021/03/chrome-91-sharedarraybuffer.html
So can I use it by reverting to an older version of Chrome? However, even if I can use it only by myself, I can not use it as a service, so I didn’t choose the way by lowering the Chrome version.
I don’t know the details about cross-origin isolation, but I understand that it is OK if I could make an isolated environment. And as Google said in the link above, it is necessary to send the header below on the web page.
Cross-Origin-Embedder-Policy: require-corp // COEP
Cross-Origin-Opener-Policy: same-origin // COOP
Ahhh, I have no idea what it means. I don’t know where to write the above headers and where to set them. I just have to search web to find someone’s solution.
Seeking a simple solution
I found a lot of solutions by Google searching. However, many of them used Node.js. It seems that Node.js has a function that sends headers, so we can easily send the above two lines of headers.
But I don’t want to use Node.js. The reason is that I just want to try a little example, so I found it bothersome to use a service that can deploy apps using Node.js.
While searching, I found an article that PHP can set headers. I’ve dealt with PHP a little, so I think I can go with this. (It took me about an hour to remember that PHP is a server-side language …)
If I use the PHP method, I can try it only with my own domain, so it’s quite easy.
Write headers of COOP and COEP in PHP file：Source Code
If you search FFmpeg.wasm by Google, you will find many sample programs that worked before the sharedarraybuffer limitation was applied. Let me use the code in the article below.
https://zenn.dev/sugar/articles/ce971201435814b0350c
Since it says Chrome 86, this article was written before the limitaion.
The points I’ve improved are as follows.
1. Download ffmpeg.min.js, ffmpeg-core.js and ffmpeg-core.worker.js files and set at the same directiory of main program.
2. Reaname the sample program file index.html as index.php, then add the headers of COOP and COEP.
Here are my codes.
index.php
<?php
ini_set('mbstring.internal_encoding' , 'UTF-8');
header('Cross-Origin-Opener-Policy: same-origin');
header('Cross-Origin-Embedder-Policy: require-corp');
?>
<html>
<head>
<script>
if (crossOriginIsolated) {
// Post SharedArrayBuffer
console.log('crossOriginIsolated');
} else {
// Do something else
console.log('NOT crossOriginIsolated');
}
</script>
<script src="ffmpeg.min.js" ></script>
</head>
<body>
<script src="app.js"></script>
</body>
</html>In the 11th line, check if cross-origin isolation is performed by crossOriginIsolated.
If you comment out the 4th and 5th lines, it will be NOT crossOriginIsolated.
app.js
(async () => {
console.log('app.js');
const { createFFmpeg } = FFmpeg
function generateImages() {
console.log('generateImages');
const canvas = document.createElement('canvas')
canvas.width = 320
canvas.height = 240
const ctx = canvas.getContext('2d')
ctx.textBaseline = 'middle'
ctx.textAlign = 'center'
ctx.font = '64px serif'
const arr = []
for (let i = 0; i < 4; i++) {
ctx.clearRect(0, 0, canvas.width, canvas.height)
ctx.fillStyle = '#fff'
ctx.fillRect(0, 0, canvas.width, canvas.height)
ctx.fillStyle = '#000'
ctx.fillText(i + 1, canvas.width / 2, canvas.height / 2)
const dataUrl = canvas.toDataURL()
arr.push(dataUrl)
}
return arr
}
async function generateVideo(images) {
console.log('generateVideo');
//const ffmpeg = createFFmpeg({ log: true })
const ffmpeg = createFFmpeg({
corePath: 'ffmpeg-core.js',
log: true
});
await ffmpeg.load()
images.forEach(async (image, i) => {
await ffmpeg.write(`image${i}.png`, image)
})
await ffmpeg.run('-r 1 -i image%d.png -pix_fmt yuv420p output.mp4')
const data = ffmpeg.read('output.mp4')
return data
}
function createObjectUrl(array, options) {
console.log('createObjectUrl');
const blob = new Blob(array, options)
const objectUrl = URL.createObjectURL(blob)
return objectUrl
}
function insertVideo(src) {
console.log('insertVideo');
const video = document.createElement('video')
video.controls = true
video.onloadedmetadata = () => {
document.body.appendChild(video)
}
video.src = src
}
const div = document.createElement('div')
div.innerText = '動画生成中'
document.body.appendChild(div)
const images = generateImages()
const video = await generateVideo(images)
const objectUrl = createObjectUrl(, { type: 'video/mp4' })
insertVideo(objectUrl)
document.body.removeChild(div)
})()Lines 35-38 are important. Since We used cross-origin isolation, we have to prepare FFmpeg only for our own domain. Thus, it cannot be used as downloading from the official page such as
”https://unpkg.com/@ffmpeg/ffmpeg@0.8.3/dist/ffmpeg.min.js”
Since it will go to refer to ffmpeg-core.js and ffmpeg-core.worker.js in “createFFmpeg”, we need to put all of files including ffmpeg.min.js in our own domain.
You can check the operation of the program here.
