I'm currently using ReplayKit for background screen recording, but I can't determine whether the screen is in landscape mode from the CMSampleBuffer. All other APIs for detecting screen orientation are foreground-based. What should I do?
Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Hello,
I'm developing an app that displays a photo library using UICollectionView and PHCachingImageManager. I'd like to achieve a user experience similar to the native iOS Photos app, where low-quality images are shown quickly while scrolling, and higher-quality images are loaded for visible cells once scrolling stops.
I'm currently using the following approach:
While Scrolling: I'm using the UICollectionViewDataSourcePrefetching protocol. In the prefetchItemsAt method, I call startCachingImages with low-quality options to cache images in advance.
After Scrolling Stops: In the scrollViewDidEndDecelerating method, I intend to load high-quality images for the currently visible cells.
I have a few questions regarding this approach:
What is the best practice for managing both low-quality and high-quality images efficiently with PHCachingImageManager? Is it correct to call startCachingImages with fastFormat options and then call it again with highQualityFormat in scrollViewDidEndDecelerating?
How can I minimize the delay when a low-quality image is replaced by a high-quality one? Are there any additional strategies to help pre-load high-quality images more effectively?
Topic:
Media Technologies
SubTopic:
Photos & Camera
My app is a camera app that supports Picture-in-Picture (PiP) mode.
Normally, when the device rotates, I get the device orientation from iOS and use it to rotate the camera feed so that the preview stays correctly aligned.
However, when the app enters PiP mode, it is considered to be in the background, and I can no longer receive orientation updates from the system.
As a result, I can’t apply rotation corrections to the camera video in PiP mode.
Is there any way to retrieve device orientation while the app is in the background (specifically during PiP mode)?
Any guidance would be greatly appreciated.
Thank you!
Our Final Cut Pro workflow extension built with ProExtensionHost framework uses an advanced NSPasteboardItemDataProvider system with multi-version FCPXML support (1.9, 1.10, 1.13) and proper relative path
UIDs for Motion templates. We've implemented clip wrapper approach with placeholder assets and elements containing effects to enable direct timeline drag functionality. However, drag
and drop from our Final Cut Pro workflow extension directly to timeline is still not working despite proper element structure in our FCPXML. Our implementation creates valid clip elements with
effects applied, but Final Cut Pro timeline doesn't accept them during drag operations from our ProExtensionHost-based workflow extension.
Steps to Reproduce:
Create Final Cut Pro workflow extension using ProExtensionHost framework with NSPasteboardItemDataProvider implementation
Generate FCPXML with proper element structure:
Expected Result: Clip should be accepted by timeline and effect applied from workflow extension
Actual Result: Timeline rejects drag operation from ProExtensionHost-based workflow extension
Question: Are there additional requirements or ProExtensionHost API calls needed beyond standard NSPasteboardItemDataProvider for Final Cut Pro workflow extension timeline drag functionality?
I found that the aggregated device correctly obtains input channels in the standard microphone mode. However, in voice isolation mode, it only retrieves channels from the first sub-device in the aggregated device's list. If I want to properly obtain channel information in voice isolation mode, how should I do it?
I'm developing a Final Cut Pro X workflow extension that transcribes audio and creates a text output. I need to allow users to drag this text directly from my extension into FCPX's timeline as titles.
Current Implementation:
Using NSFilePromiseProvider as per Apple's guidelines for drag and drop
Generating valid FCPXML (v1.10) with proper structure:
Complete resources section with format and asset references
Event and project hierarchy
Asset clip with connected title elements
Proper timing and duration calculations
Supporting multiple pasteboard types:
com.apple.finalcutpro.xml.v1-10
com.apple.finalcutpro.xml.v1-9
com.apple.finalcutpro.xml
What's Working:
Drag operation initiates correctly
File promise provider is set up properly
FCPXML generation is successful (verified content)
All required pasteboard types are registered
Proper logging confirms data is being requested and provided
Current Pasteboard Types Offered:
com.apple.NSFilePromiseItemMetaData
com.apple.pasteboard.promised-file-name
com.apple.pasteboard.promised-suggested-file-name
com.apple.pasteboard.promised-file-content-type
Apple files promise pasteboard type
com.apple.pasteboard.NSFilePromiseID
com.apple.pasteboard.promised-file-url
com.apple.finalcutpro.xml.v1-10
com.apple.finalcutpro.xml.v1-9
com.apple.finalcutpro.xml
What additional requirements or considerations are needed to make FCPX accept the dragged FCPXML content? Are there specific requirements for workflow extensions regarding drag and drop operations with titles that aren't documented?
Any insights, especially from those who have implemented similar functionality in FCPX workflow extensions, would be greatly appreciated.
Technical Details:
macOS Version: 15.5 (24F74)
FCPX Version: 11.1.1
Extension built with SwiftUI and AppKit integration
Using NSFilePromiseProvider and NSPasteboardItemDataProvider
Full pasteboard type support for FCPXML versions
Hi all,
i can successfully match music using shazamkit on Apple using SwiftUI, a simple app that let user to load an audio file and exctracts the relative match, while i am unable to match music using shamzamkit on Android. I am trying to make the same simple app but i cannot match music as i get MATCH_ATTEMPT_FAILED every time i try to. I don't know what i am doing wrong but the shazam part in the kotlin Android code is in this method :
suspend fun processAudioFileInBackground(
filePath: String,
developerTokenProvider: DeveloperTokenProvider
) = withContext(Dispatchers.IO) {
val bufferSize = 1024 * 1024
val audioFile = FileInputStream(filePath)
val byteBuffer = ByteBuffer.allocate(bufferSize)
byteBuffer.order(ByteOrder.LITTLE_ENDIAN)
var bytesRead: Int
while (audioFile.read(byteBuffer.array()).also { bytesRead = it } != -1) {
val signatureGenerator = (ShazamKit.createSignatureGenerator(AudioSampleRateInHz.SAMPLE_RATE_44100) as ShazamKitResult.Success).data
signatureGenerator.append(byteBuffer.array(), bytesRead, System.currentTimeMillis())
val signature = signatureGenerator.generateSignature()
println("Signature: ${signature.durationInMs}")
val catalog = ShazamKit.createShazamCatalog(developerTokenProvider, Locale.ENGLISH)
val session = (ShazamKit.createSession(catalog) as ShazamKitResult.Success).data
val matchResult = session.match(signature)
println("MatchResult : $matchResult")
setMatchResult(matchResult)
byteBuffer.clear()
}
audioFile.close()
}
I noticed that changing Locale in catalog creation results in different result as i get NoMatch without exception. Can you please help me with this?
Topic:
Media Technologies
SubTopic:
Audio
I'm capturing video stream from GoPro camera (via UDP MPEG-TS packets) but unable to play in iOS app. can someone provide a source code to decode MPEG-TS data to CMSampleBuffer.
I am working on an app which plays audio - https://youtu.be/VbAfUk_eYl0?si=nJg5ayy2faWE78-g - and one of the features is, on restart, if you had paused playback of a file at the time the app was previously shut down (or were playing one at the time of shutdown), the paused state and position in the file is restored exactly as it was, on restart.
The functionality works. However, it seems impossible to get the "now playing" information in iOS into the right state to reflect that via the MediaPlayer API. On restart, handlers are attached to the play/pause/togglePlayPause actions on MPRemoteCommandCenter.shared(), and the map of media info is updated on MPNowPlayingInfoCenter.default().nowPlayingInfo.
What happens is that iOS's media view shows the audio as playing and offers a pause button - even though the play action is enabled and the pause action is disabled.
Once playback has been initiated (my workaround is to have the pause action toggle the play state, since otherwise you wouldn't be able to initiate playback from controls in a car without initiating it once from a device first).
I've created a simplified white-noise-player demo to illustrate the problem - simply build and deploy it, and then start the app, lock your device and look at the playback controls on the lock screen. It will show a pause button - same behavior I've described.
https://github.com/timboudreau/ios-play-pause-demo
I've tried a few things to narrow down the source of the issue - for example, thinking that not MPNowPlayingInfoPropertyPlaybackProgress and MPMediaItemPropertyPlaybackDuration might be the culprit (since the system interpolates elapsed time and it's recommended to update those properties infrequently) on startup might do the trick, but the result is the same, just without a duration or progress shown.
What governs this behavior, and is there some way to explicitly tell the media player API your current state is paused?
On Apple TV 4K 3rd generation, with tvOS 26 beta 2, when two HomePod 2 are paired to the device, music and movie sources with Dolby Atmos can only be listened to in stereo. dolby atmos not supported
Topic:
Media Technologies
SubTopic:
Audio
My app Balletrax is a music player for people to use while they teach ballet. Used to be you could silence notifications during use, but now the customer seems to have to know how to use Focus mode, remember to turn it on and off, and have to check the notifications one does and doesn't want to use. Is there no way to silence all notifications when the app is in use?
Hi everyone,
I’m trying to use AVAssetResourceLoaderDelegate to handle a live radio stream (e.g. Icecast/HTTP stream). My goal is to have access to the last 30 seconds of audio data during playback, so I can analyze it for specific audio patterns in near-real-time.
I’ve implemented a custom resource loader that works fine for podcasts and static files, where the file size and content length are known. However, for infinite live streams, my current implementation stops receiving new loading requests after the first one is served. As a result, the playback either stalls or fails to continue.
Has anyone successfully used AVAssetResourceLoaderDelegate with a continuous radio stream? Or maybe you can suggest betterapproach for buffering and analyzing live audio?
Any tips, examples, or advice would be appreciated. Thanks!
We are encountering an issue where AVPlayer throws the error:
Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" > Underlying Error Domain[(null)]:Code[0]:Desc[(null)]
This error seems to occur intermittently during video playback, especially after extended usage or when switching between different streams. We observe Error 11819 (AVFoundationErrorDomain) in Conviva platform that some of our users experience it but we couldn't reproduce it so far and we’re need support to determine the root cause and/or best practices to prevent it.
Some questions we have:
What typically triggers this error?
Could it be related to memory/resource constraints, network instability, or backgrounding?
Are there any recommended ways to handle or recover from this error gracefully?
Any insights or guidance would be greatly appreciated. Thanks!
Topic:
Media Technologies
SubTopic:
Streaming
Hi, I’ve developed a photo app that includes a photo deletion feature.
Some users have reported encountering PHPhotosError.operationInterrupted (3301) when attempting to delete photos.
Initially, I suspected that some of the assets might have a sourceType of typeiTunesSynced, since the documentation notes that iTunes-synced assets cannot be edited or deleted.
However, after checking the logs, all of the assets involved are of typeUserLibrary.
Additionally, the user mentioned that some photos in the iPhone Photos do not show a delete button.
I’m unsure whether the absence of the delete button is related to the 3301 error.
I’d like to confirm the following:
Under what conditions does PHPhotosError.operationInterrupted (3301) occur, and how should it be handled?
Why do some photos in the iPhone Photos not show a delete button?
The code for deleting photos is as follows:
PHPhotoLibrary *library = [PHPhotoLibrary sharedPhotoLibrary];
[library performChanges:^{
PHFetchResult *assetsToBeDeleted = [PHAsset fetchAssetsWithLocalIdentifiers:delUrls options:nil];
if (assetsToBeDeleted) {
[PHAssetChangeRequest deleteAssets:assetsToBeDeleted];
}
} completionHandler:^(BOOL success, NSError *error) {
When I play an m3u8 video using AVPlayer, it can play smoothly at 2x speed. However, when I set it to 3x speed, the playback is not smooth and there is no sound.
Topic:
Media Technologies
SubTopic:
Video
ApplicationMusicPlayer is not available on watchOS but all other platforms. Is there a technical reason for that like battery life? Same goes for SystemMusicPlayer and MPMusicPlayerController. I already filed feedbacks for that.
Are serialized parameters already available inside -pluginInstanceAddedToDocument via FxParameterRetrievalAPI or are they being read later?
Hello! I have been following the UsingAVFoundationToPlayAndPersistHTTPLiveStreams sample code in order to test persisting streams to disk. In addition to support for m3u8, I have noticed in testing that this also seems to work for MP3 Audio, simply by changing the plist entries to point to remote URLs with audio/mpeg content. Is this expected, or are there caveats that I should be aware of?
Thanks you!
Hello, I'm trying to subscribe to AVPlayerItem status updates using Combine and it's bridge to Swift Concurrency – .values.
This is my sample code.
struct ContentView: View {
@State var player: AVPlayer?
@State var loaded = false
var body: some View {
VStack {
if let player {
Text("loading status: \(loaded)")
Spacer()
VideoPlayer(player: player)
Button("Load") {
Task {
let item = AVPlayerItem(
url: URL(string: "https://sample-videos.com/video321/mp4/360/big_buck_bunny_360p_5mb.mp4")!
)
player.replaceCurrentItem(with: item)
let publisher = player.publisher(for: \.status)
for await status in publisher.values {
print(status.rawValue)
if status == .readyToPlay {
loaded = true
break
}
}
print("we are out")
}
}
}
else {
Text("No video selected")
}
}
.task {
player = AVPlayer()
}
}
}
After I click on the "load" button it prints out 0 (as the initial status of .unknown) and nothing after – even when the video is fully loaded.
At the same time this works as expected (loading status is set to true):
struct ContentView: View {
@State var player: AVPlayer?
@State var loaded = false
@State var cancellable: AnyCancellable?
var body: some View {
VStack {
if let player {
Text("loading status: \(loaded)")
Spacer()
VideoPlayer(player: player)
Button("Load") {
Task {
let item = AVPlayerItem(
url: URL(string: "https://sample-videos.com/video321/mp4/360/big_buck_bunny_360p_5mb.mp4")!
)
player.replaceCurrentItem(with: item)
let stream = AsyncStream { continuation in
cancellable = item.publisher(for: \.status)
.sink {
if $0 == .readyToPlay {
continuation.yield($0)
continuation.finish()
}
}
}
for await _ in stream {
loaded = true
cancellable?.cancel()
cancellable = nil
break
}
}
}
}
else {
Text("No video selected")
}
}
.task {
player = AVPlayer()
}
}
}
Is this a bug or something?
Hi Apple Team,
We have integrated FairPlay Streaming Server SDK v3 into our MDRM platform since 2017, the system works stable and stayed untouched. As you know, both Widevine and Playready have requirements to upgrade the Server SDK regularly. We want to know if Apple imposes similar requirements for upgrading the FPS SDK, or if we may continue using the old one without any updates.
Thanks for your support!