Skip to content
filmstrip

Probe and frames

Probe

Section titled “Probe”

probe reads a source’s metadata without decoding it. An unreadable file comes back as a Failure, so there’s nothing to catch.

val info: MediaInfo? =
  when (val result = filmstrip.probe(MediaSource.of("/path/to/clip.mp4"))) {
    is ProbeResult.Success -> {
      println("${result.info.duration}, ${result.info.video?.displaySize}")
      result.info
    }
    is ProbeResult.Failure -> {
      println(result.error.message)
      null
    }
  }

MediaInfo has the duration, a video and an audio track, either of which is null when the source has none, and isExportable, which is false for DRM-protected files. Those can’t be exported or previewed, so check it first.

On a video track, size a target from displaySize, which has rotation and pixel aspect applied, rather than codedSize. bitDepth is often null, since only HEVC carries it somewhere every backend can read. hdrTransfer is null for SDR.

Android and Apple read a container with core alone. The JVM and the browser need a registered backend. Without one, both fail with BackendMissing, naming the artifact to add. A still needs no backend on any target.

On the JVM a probe that ffprobe hasn’t finished within its deadline is stopped and refused as TimedOut, which names the read and its deadline, rather than leaving the call hanging. See Limits.

Capabilities

Section titled “Capabilities”

capabilities asks what this device’s encoders can do. The answer is cached per process. On the JVM a test encode that runs past its deadline fails the call naming the encoder, and that failure isn’t cached, so asking again measures again.

val capabilities: DeviceCapabilities? =
  when (val result = filmstrip.capabilities()) {
    is CapabilitiesResult.Success -> {
      println("HEVC encoder: ${result.capabilities.encoderFor(VideoCodec.Hevc)}")
      println("HDR: ${result.capabilities.supportsHdrEncoding}")
      result.capabilities
    }
    is CapabilitiesResult.Failure -> {
      println(result.error.message)
      null
    }
  }

encoderFor answers one codec’s limits, such as maxSize, maxFrameRate and maxBitrate, or null when there’s no encoder for it. To find out what will happen to one particular edit, use plan, which already takes all of this into account.

One frame

Section titled “One frame”

frame renders the frame covering an instant of a composition, with its effects applied. You own the image and have to close it.

when (val result = filmstrip.frame(composition, at = 3.seconds, heightPx = 360)) {
  is FrameResult.Success -> {
    result.image.use { image ->
      println("${image.widthPx}x${image.heightPx} at ${result.presentationTime}")
    }
  }
  is FrameResult.Failure -> {
    println(result.error.message)
  }
}

Leave heightPx null to render at the composition’s own output height. A height at or above that height renders at that height too, since a frame is never scaled up. Zero or less fails with InvalidComposition.

For one source with no edit, pass the source directly.

val result = filmstrip.frame(MediaSource.of("/path/to/clip.mp4"), at = 1.seconds)

Many frames

Section titled “Many frames”

frames emits one result per position, each as it’s ready. It’s made for a timeline strip, so a frame may come from the nearest sync sample rather than the one covering its position. presentationTime says where each one actually landed. Use frame when a position has to be exact.

val positions = List(10) { index -> (index * 2).seconds }


filmstrip.frames(composition, positions, heightPx = 96).collect { result ->
  when (result) {
    is FrameResult.Success -> {
      result.image.use { image ->
        println("${image.widthPx}x${image.heightPx} at ${result.presentationTime}")
      }
    }
    is FrameResult.Failure -> {
      println(result.error.message)
    }
  }
}