(WIP) Audio core module with UniFFI bindings

[ladvoc]

No description provided.

[pblazej]

soxr fails to build on sims, working fix (build.rs)

    // Disable stack checking for iOS/tvOS/visionOS targets to avoid ___chkstk_darwin dependency
    // which is only available on macOS, not iOS
    if target.contains("apple-ios") || target.contains("apple-tvos") || target.contains("apple-visionos") {
        build.flag("-fno-stack-check");
    }

[pblazej]

Swift → Rust experiment

Starting point: UnsafeMutablePointer<Float>

• https://developer.apple.com/documentation/avfaudio/avaudiopcmbuffer/floatchanneldata
• https://github.com/livekit/client-sdk-swift/blob/main/Sources/LiveKit/Types/AudioBuffer.swift#L45

Rust APIs:

• fn push(&self, input: &[i16]) → [Int16] - requires copy on the Swift side already (Array owns its data buffer) 🔴
• fn push(&self, input: AudioBufferBytes) → Data 🟠
  • does not require copy on the Swift side due to https://developer.apple.com/documentation/foundation/data/init(bytesnocopy:count:deallocator:)
  • does copy over FFI boundary because of the actual lower impl

pub struct AudioBufferBytes {
    pub data: Vec<u8>,
}

var writer = createWriter()
write(value, into: &writer)
return RustBuffer(bytes: writer) // here comes the copy

• brute-force fn push_from_ptr(&self, data_ptr: u64, len: u32) - works, no copy on either side, no ownership, no safety 🟢

let frameCount = audioBuffer.frames
let channelBuffer = audioBuffer.rawBuffer(forChannel: 0)

// Arithmetics...
try resampler.pushFromPtr(dataPtr: UInt64(bitPattern: Int64(Int(bitPattern: channelBuffer))), len: UInt32(frameCount * MemoryLayout<Float>.size))

Conclusion: it boils down to using u64 under the hood, where the conversion really happens (bindgen or Swift) probably is not that important

#[derive(uniffi::Record)]
pub struct AudioBuffer {
    /// Pointer value as u64 (points to Float32 data in Swift memory)
    pub pointer: u64,
    /// Length in bytes
    pub len: u32,
}

impl AudioBuffer {
    /// Get the pointer as *const u8
    pub fn as_ptr(&self) -> *const u8 {
        self.pointer as *const u8
    }
    
    /// Get length in bytes
    pub fn byte_len(&self) -> usize {
        self.len as usize
    }
}

[pblazej]

Rust → Swift experiment

I think the assumption is that we leak() the Rust thing into Swift, then expose sth like:

#[uniffi::export]
pub fn free_rust_audio_buffer(buffer: RustAudioBuffer) {
    unsafe {
        if buffer.pointer != 0 {
            let capacity = buffer.sample_capacity();
            let len = buffer.sample_len();
            let ptr = buffer.pointer as *mut i16;
            let _ = Vec::from_raw_parts(ptr, len, capacity);
        }
    }
}