Detect scene changes

Find the moments where a video cuts to a new scene, and list each scene with its start, end and length.

Input

Choose a file or drop it here

Up to 100 MB.

The video to examine.
How big a change counts as a new scene, from 0.05 (small changes) to 0.95 (only hard cuts).
Shortest time between two scene changes, in seconds. Changes closer together are counted as one.
Also give the frame number of each change and the number of frames examined.

Result

Choose a file and press Run.

Files you upload and the results are stored for one hour, then deleted. Anyone who has the link can open them, so don't upload anything private.

Use it from code or an agent

# 1. Upload the file. The reply ends with a line "ref  <id>/f1": that is the file reference.
curl -T sample.mp4 https://setget.net/drop/

# 2. Run the tool with that reference. "wait" holds the reply for up to 30 seconds while the job runs.
curl -s https://setget.net/v1/tools/video-scene-detect \
  -H 'Content-Type: application/json' \
  -d '{"input":{"video":"DROP_ID/f1"},"wait":30}'

# 3. The reply is {"tool", "ms", "result", "files": [{"name", "size", "url", "ref"}], "job"}.
#    Download a result file from its "url", or pass its "ref" to another tool.
#    Still running after the wait? The reply is 202 with "job": follow it with
#    curl -s https://setget.net/v1/jobs/JOB_ID?wait=30
import json, urllib.parse, urllib.request

BASE = "https://setget.net"

def call(method, path, body=None, headers=None):
    request = urllib.request.Request(BASE + path, body, headers or {}, method=method)
    return json.load(urllib.request.urlopen(request))

def upload(path):
    """Step 1: upload a file and get its reference."""
    name = path.replace("\\", "/").rsplit("/", 1)[-1]
    with open(path, "rb") as handle:
        reply = call("PUT", "/drop/" + urllib.parse.quote(name), handle.read(), {"Accept": "application/json"})
    return reply["file"]["ref"]

# Step 2: run the tool with the reference.
body = {"input": {'video': upload("sample.mp4")}, "wait": 30}
reply = call("POST", "/v1/tools/video-scene-detect", json.dumps(body).encode(), {"Content-Type": "application/json"})

# Step 3: a job that is still running comes back as {"job": ...}. Wait for it.
while "result" not in reply and reply.get("job", reply).get("status") in ("queued", "running"):
    reply = call("GET", "/v1/jobs/" + reply.get("job", reply)["id"] + "?wait=30")
if "error" in reply:
    raise SystemExit(reply["error"]["message"])
print(reply["result"])
for file in reply["files"]:   # result files: download from "url", or pass "ref" to another tool
    print(file["name"], file["size"], file["url"], file["ref"])
const BASE = "https://setget.net";

// Step 1: upload a file and get its reference. file: a File or Blob
// (in Node: new Blob([fs.readFileSync(path)])).
async function upload(file, name) {
  const response = await fetch(`${BASE}/drop/${encodeURIComponent(name)}`, {
    method: "PUT", headers: {Accept: "application/json"}, body: file,
  });
  return (await response.json()).file.ref;
}

// Step 2: run the tool with the reference.
let reply = await (await fetch(`${BASE}/v1/tools/video-scene-detect`, {
  method: "POST",
  headers: {"Content-Type": "application/json"},
  body: JSON.stringify({input: {"video":await upload(file, "sample.mp4")}, wait: 30}),
})).json();

// Step 3: a job that is still running comes back as {job}. Wait for it.
while (!reply.result && ["queued", "running"].includes((reply.job || reply).status)) {
  reply = await (await fetch(`${BASE}/v1/jobs/${(reply.job || reply).id}?wait=30`)).json();
}
if (reply.error) throw new Error(reply.error.message);
console.log(reply.result);
for (const file of reply.files) console.log(file.name, file.size, file.url, file.ref);
1. Upload. Tool: drop
Arguments:
{
  "action": "create",
  "files": [
    {
      "name": "sample.mp4",
      "size": "<bytes>"
    }
  ]
}
PUT each file's bytes to its "upload.url" with the listed headers, and keep its "ref".
With a shell, one command does both: curl -T FILE https://setget.net/drop/

2. Run. Tool: run_tool
Arguments:
{
  "slug": "video-scene-detect",
  "input": {
    "video": "DROP_ID/f1"
  }
}

3. The answer holds "result" and "files" (each with "url" and "ref").
If it holds a "job" that is still running instead, follow it. Tool: job
Arguments:
{
  "action": "wait",
  "id": "JOB_ID"
}
{
  "input": {
    "video": "DROP_ID/f1"
  },
  "wait": 30
}

This is a file tool, so a call takes three steps: upload the file to get a reference, run the tool with that reference, then read the result and download its files. Uploads and results are public by link and expire after one hour.

Endpoint: POST /v1/tools/video-scene-detect. Over MCP: run_tool with slug: "video-scene-detect". Machine-readable: input schema, Markdown. Connect an agent.

Inputs

NameTypeRequiredWhat it does
video file yes The video to examine. In the API: a file reference such as DROP_ID/f1. Accepts: video.
threshold number no How big a change counts as a new scene, from 0.05 (small changes) to 0.95 (only hard cuts). Default: 0.3. From 0.05 to 0.95.
min_gap number no Shortest time between two scene changes, in seconds. Changes closer together are counted as one. Default: 0.5. From 0 to 60.
frame_numbers boolean no Also give the frame number of each change and the number of frames examined. Default: false.

Examples

Find the cuts in a clip

Input
{
  "video": "sample.mp4"
}
Result

Press the example button above to run it on the sample file.

Smaller changes too, with frame numbers

Input
{
  "video": "sample.mp4",
  "threshold": 0.1,
  "min_gap": 1,
  "frame_numbers": true
}
Result

Press the example button above to run it on the sample file.

Questions

What do I do with the times?
Use them as start and end in video-trim to cut a scene out, or let video-frames save one picture per scene.
It found too many or too few changes. What now?
Raise the threshold to ignore camera movement and flashes. Lower it to catch soft cuts. Slow fades from one scene to the next are hard to find at any setting.