Media helpers for Mob apps, through the platform's own decoders: what's in a media file, its audio as PCM for speech-to-text, and a video frame as a JPEG.

{:ok, %{duration_ms: 12_480, has_video: true, width: 1080, height: 1920}} =
  MobMedia.info(path)

{:ok, pcm} = MobMedia.audio_pcm(path)          # 16 kHz mono s16le
{:ok, text} = MobWhisper.transcribe(pcm)

{:ok, jpeg} = MobMedia.frame(path, 4_000, max_px: 1024)

Android decodes with MediaExtractor + MediaCodec and grabs frames with MediaMetadataRetriever; iOS uses AVAssetReader and AVAssetImageGenerator. Anything the platform's decoders handle works (verified on both: m4a/AAC, mp3, wav, ogg/opus, mp4/H.264). No ffmpeg, no permission: path is a local file the app can already read (from Mob.Files.pick/2, mob_photos, mob_camera, Mob.Audio, ...), not a content:// URI.

Sync, off the schedulers

Every function blocks the calling process until the answer is ready and returns it. The work itself runs on a native thread (a Kotlin thread pool on Android, an operation queue on iOS); the NIF only queues it and returns, and the native side sends the result back as a message. No BEAM scheduler, normal or dirty, is held while a file decodes.

Decoding a long recording takes a while (10-30% of its length on a mid-range Android phone), so don't call audio_pcm/2 from a screen's handle_event/handle_info directly: the screen would stop answering taps until it returns. Run it in a task and send yourself the result:

screen = self()
Task.start(fn -> send(screen, {:pcm_ready, MobMedia.audio_pcm(path)}) end)

Each call runs in its own short-lived process, so a call that times out (:timeout option) returns {:error, :timeout} at once while the native work finishes in the background and its temporary output is deleted. A caller that dies is handled the same way.

Memory

audio_pcm/2 never holds the whole decode in native memory: the platform side streams the PCM into a file as it decodes. Without :to, that file is temporary and is read into the binary audio_pcm/2 returns (16 kHz mono is 1.9 MB a minute, 115 MB an hour). For long recordings pass to: path: the PCM stays in that file ({:ok, path}), and you can read it a window at a time (whisper works in 30 s windows: 960 000 bytes at 16 kHz mono):

{:ok, pcm_path} = MobMedia.audio_pcm(video, to: Path.join(dir, "audio.pcm"))
{:ok, fd} = :file.open(pcm_path, [:read, :binary, :raw])
{:ok, window} = :file.pread(fd, 0, 960_000)

Errors

• :not_found — path doesn't exist.
• :unsupported — not a format the platform can open or decode.
• :no_audio / :no_video — the file has no such track.
• :decode_failed — the decoder failed part way (a damaged file).
• :io_error — the output couldn't be written or read back.
• :timeout — the :timeout option ran out.
• :unavailable — the NIF isn't linked into this build (or you're on the host without MobMedia.Engine.Fake).
• {:jni, reason} (Android) / :alloc (iOS) — the NIF couldn't queue the job; see MobMedia.Engine.Native.

Engines

The decoding is done by a MobMedia.Engine: MobMedia.Engine.Native when the NIF is linked, MobMedia.Engine.Unavailable otherwise. Tests and agents use MobMedia.Engine.Fake (real WAV decoding, scripted answers for anything else):

config :mob_media, engine: MobMedia.Engine.Fake