How I Found 8 Microphones Android Said Didn't Exist

The question

Some of our classroom panels have an “AI camera”: a 4K camera and a microphone array that frame whoever is speaking. The feature came from the hardware vendor, as a black box. In August 2026 the question was simple. Could we build it ourselves, on the panel’s own Android, and control it?

Speaker framing needs two things: who is in the room, and which one is talking. The first is computer vision. The second, I assumed, was the microphone array, since direction-of-arrival is what microphone arrays are for.

My initial mental model

8 microphones ──► time difference of arrival ──► angle of the voice
                                                      │
4K camera ──► detect people ──► pick the person at that angle ──► crop

I expected the hard part to be the signal processing. I expected Android to give me the channels. Both expectations were wrong, and the order they failed in is the interesting part.

What actually happens

First, physics decides the architecture

Two findings from the first day of research reshaped the plan. Published results put GCC-PHAT, the standard time-delay method, at roughly 10° of error in a reverberant room (RT60 ≈ 1 s), which is a typical classroom. At 6 metres that is about a metre sideways, enough to pick the wrong student.

The camera changed the picture too. The module has a fixed 48 MP sensor with 4K output and no motors, so “pan and zoom” is digital cropping and the whole room is in every frame. Together those facts invert the design. Vision draws the box. Audio only has to choose which visible person to crop to. A direction accurate to ±10° is plenty for that.

Then Android reports channels that don’t exist

I wrote a probe app that requests an 8-channel AudioRecord and checks what comes back. My desk dev panel was another model, with no camera module and a USB webcam plugged in. On it, the probe reported “8 channels verified”. That panel’s audio policy caps input at stereo.

val record = AudioRecord.Builder()
    .setAudioFormat(AudioFormat.Builder()
        .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
        .setSampleRate(16_000)
        .setChannelIndexMask(0xFF)          // ask for 8 channels
        .build())
    .build()
record.startRecording()                     // succeeds
record.channelCount                         // 8
record.read(buffer, 0, buffer.size)         // returns 8 channels' worth of bytes

Everything succeeds. AudioFlinger satisfies an 8-channel request from a 2-channel source by duplicating channels. getChannelCount() tells you what you asked for, not what the hardware has.

The test that works is to de-interleave the PCM and count channels that are bit-distinct. Two real microphones never match sample for sample, because their noise floors differ. Duplicated channels match exactly. Two caveats make the test lie if you skip them:

Digital silence: all-zero channels are trivially identical, so a dead mic reads the same as an upmixed one. Report per-channel RMS and an allSilent flag separately.
Warm-up zeros: capture streams deliver silence for the first few hundred milliseconds. Discard several buffers, and capture for more than a second.

The USB stack hides a device entirely

The same panel had a second trick. dumpsys usb said the webcam had audio input. AudioManager.getDevices() never listed it, with any audio source, whether or not the camera was streaming.

The reason: that panel’s USB audio HAL surfaces one USB audio device, and the built-in soundbar had already claimed it. The only way past that is to read the device’s isochronous endpoints directly. The Java USB API can’t do isochronous transfers, so that means libusb over the UsbDeviceConnection file descriptor. That was worth knowing before buying any USB array: Android may never show it to you.

Let’s look at the hardware

The AI-camera panel itself, a TeachmintX 75 Pro dev unit on a rooted userdebug build, has a discrete USB microphone module built around a Rockchip RK3308. I dumped its raw descriptors from sysfs and decoded them by hand:

Audio streaming interface 1
  alt 0   no endpoints                      (off)
  alt 1   bNrChannels = 1, 16-bit, 16 kHz   isochronous IN, 34-byte packets
  (there is no alt 2)
Input terminal: bNrChannels = 1

tinypcminfo agreed, and so did the kernel’s /proc/asound view: one channel, 16 kHz. That made four independent layers saying mono. The module beamforms its eight microphones in its own DSP and sends one channel over USB.

I wrote it down as settled: building our own sound localization from raw channels was impossible on this hardware. The descriptors left no room for anything else. The mono stream was still useful for voice-activity detection, so the plan became vision plus “someone is talking right now”.

The surprising part

The module had more interfaces than it needed for audio: a vendor-specific bulk pair, a CDC-ACM serial port, and a HID interface. If direction-of-arrival existed anywhere, it would be on one of those.

Passive reads returned nothing. The serial node gave zero bytes, idle and during capture. The HID interface was a boot keyboard, the hardware mute button. Decompiling the vendor’s apps found no direction, azimuth or beamforming anywhere in their code or native libraries. The camera tracked speakers inside its own firmware, and the panel never learned an angle.

The native library that talked to the module was more forthcoming. Its strings and disassembly showed it driving the RK3308 over HID, and one routine stood out: it wrote a short fixed command that opened the module’s UART. The serial port was silent because it was switched off, not because it had nothing to say.

With the UART opened, the port spoke a small request/response protocol: a literal tag, a length, then JSON. One command type asked the module to record for N seconds and dump the result. I asked for nine seconds and eight channels.

The declared payload was 3,456,010 bytes. That is 9 s × 16,000 Hz × 2 bytes × 12 channels, interleaved, and period-24 autocorrelation confirmed the 24-byte frame stride.

channels 0–7    the eight microphones
channels 8–9    near-silent (likely echo-cancellation reference)
channels 10–11  zero padding

Proving they were really eight microphones

The bit-distinct test said the channels were different. It can’t say they are eight positions. The first captures were all noise floor, because the array was muted upstream of both paths. Unmuted, with a video playing in the room, each microphone’s lag against channel 0 was stable across independent windows:

lag vs ch0 in samples at 16 kHz (1 sample ≈ 2.1 cm of path)
t = 2.50 s    0  0 +1  0 +2 +2 +3 +2
t = 4.38 s    0  0 +1  0 +2 +2 +3 +3
t = 7.50 s    0  0 +1  0 +2 +2 +2 +2      same geometry every window

Peak cross-correlation against channel 0 was 0.44–0.73. That is high enough to be the same sound and far from the 1.0 an upmix would give. One window came back with garbage lags at correlation ~0.28, which taught the rule for any TDOA code: gate the estimate on correlation strength. Every take also starts with a large DC transient, so skip the first ~1,000 samples or the peak-picker locks onto it. It fooled me twice.

The USB-descriptor reasoning had been right and incomplete at the same time. The raw channels never crossed the audio endpoints. They crossed the serial link, as a file transfer.

Practical consequences

The conclusion became “possible, but not like this”:

The UART path is batch-only. Every cycle is request → record → dump, with about ten seconds of fixed overhead and roughly 30 seconds per round trip. Short requests don’t shorten it. No other command streams.
Bandwidth isn’t the problem. The dump bursts at 1.1 MB/s, about four times what eight channels of 16 kHz audio need. The link is a USB gadget, not a slow physical serial line. The limit is the firmware’s protocol.
So the channel is a ground-truth dataset generator, and a concrete request for the module’s firmware owner: expose an 8-channel alternate setting on the audio interface. The panel can already flash that firmware. We just don’t have an image.

Meanwhile, I built the product the physics allows. The prototype frames faces with ML Kit, using the camera’s 4K output since it has no hardware zoom. When the mono stream says someone is speaking, a motion localizer picks the person moving at that moment and steers the crop. I tried three algorithms behind one interface. At 640×360 they cost 52–102 µs per frame on the panel, which leaves the CPU for everything else. 103 unit tests pin the behaviour, including one that documents exactly where my own quadtree idea loses, rather than quietly fixing it.

A few platform facts from the probes, for anyone doing similar work:

4K camera output is fine as YUV and unusable as JPEG. The camera ran at 12.6–16.1 fps in YUV_420_888 at 4K and fell to 0.5 fps as JPEG.
“Loaded” doesn’t mean “has an NPU”. A bundled RKNN runtime loads happily on a panel with no NPU device node. Check the hardware, not the dlopen result.
Static library lists lie. public.libraries.txt listed an OpenCL library that then failed to load.

What I would remember

An API echoing your request isn’t evidence. channelCount == 8 meant “you asked for 8”. Measure the data: distinct bits, RMS, routed device.
“Impossible at this layer” isn’t “impossible”. Four layers agreed the array was mono, and all of them described the audio path. The answer was on a different interface.
Silence has a cause. The serial port wasn’t empty. It was closed. The array wasn’t dead. It was muted.
Prove geometry with stability. Lags that hold across independent windows separate real microphones from duplicates. Correlation strength says when to trust a window.
Let physics pick the architecture. ±10° of audio error was never going to draw a box. It only had to choose one.

Postscript: the repo that came back from the logs

The original working copy was never pushed anywhere and didn’t survive a move to a new laptop. I rebuilt it in October from two sources. One was the AI pair-programming session history, which held every file write and edit from those three days, replayed in timestamp order. The other was a decompile of the APK still installed on the panel, which filled the gaps and checked the replay class by class. All 103 tests pass on the reconstruction. The research notebook didn’t make it, which is the best argument I know for pushing on day one.