Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tapping the internal microphones #259

Merged
merged 20 commits into from
Nov 4, 2022
Merged

Tapping the internal microphones #259

merged 20 commits into from
Nov 4, 2022

Conversation

povik
Copy link
Member

@povik povik commented Oct 23, 2022

This has extension of experiments/aop.py, addition of AOP tracer and some other tweaks.

To cite the key commit:

experiments/aop.py: Extend AOP experiment, add tracer

The AOP uses an 'EPIC' protocol similar to the one other coprocessor
firmware is using but not in the exact same version. Add code for
tracing the AOP calls and extend the aop.py experiment with the client
side of it. Include description of audio calls and some other calls
related to sensor discovery.

Furthermore, in experiments/aop.py, do some AOP audio setup. Once that
is done we can start streaming samples from the internal microphones by
making what AOP considers power state adjustment calls. That is, we
adjust the power state of a 'hpai' device, first to a 'pw1 ' stage,
then to 'pwrd' stage.

So, to see microphone samples, enter the AOP experiment shell first:

  $ M1N1DEVICE=/dev/ttyACM0 experiments/aop.py

Within the shell, adjust the power state of 'hpai':

  >>> aop_set_audio_pstate('hpai', 'pw1 ')

At that point /arm-io/admac-aop-audio powers up. In parallel to the AOP
shell, we can start tools/admac_stream.py on the just powered-up ADMAC
instance:

  $ M1N1HEAP=0x10010000000 M1N1DEVICE=/dev/ttyACM1 tools/admac_stream.py \
        --node admac-aop-audio --channel 1 -v | xxd -g 4 -c 12 -e

Returning back to the AOP shell, we can then set 'hpai' to 'pwrd' state
to kick off the streaming:

  >>> aop_set_audio_pstate('hpai', 'pwrd')

By that point, we should see samples coming out on the ADMAC end. The
samples are 32-bit floats packed in groups of three in a frame, e.g.

00000000: ba7ac6a7 ba32d3c3 baa17ae2  ..z...2..z..
0000000c: 38ccea5f b99c1a37 ba0c4bb1  _..87....K..
00000018: 39d2354f 3964b5ff 39b209fb  O5.9..d9...9
00000024: b96a1d1f 39c8503f 3958fc4f  ..j.?P.9O.X9
00000030: b6b1f5ff 39c72b8f 39bbe017  .....+.9...9
0000003c: 3a912de5 36dd4f7f 37f1147f  .-.:.O.6...7

This has been tested and will to some degree be specific to 2021 Macbook
Pro (t6000). Differences on other models TBD (at the very least the
number of microphones can be presumed different).

Signed-off-by: Martin Povišer <povik@protonmail.com>

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
The new registers contain SRAM limits allocated to each channel.

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
The Reloadable parent doesn't have an __init__ method, so calling the
super may call at unexpected places in richer inheritance hierarchies.

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
Let user override the proxyutils heap base for concurrent running of two
proxyclient sessions.

Signed-off-by: Martin Povišer <povik@protonmail.com>
We need to adapt the prop-parsing logic to give precedence to path
matching over name matching, since otherwise the audio-* rule (which is
already in the list) catches (...)/aop-audio/audio-pdm2. We want that
to fall under *aop-audio* rule instead.

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
In t6000 ADT, the AOP SRAM base is specified *including* the bus offset
where the bus offset isn't expected, so our decoding of it includes the
bus offset twice. Patch it.

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
We need to allocate a buffer for the AOP on the OSLog endpoint for it to
fully boot. Copy in a modified version of the general OSLog endpoint
driver to do that.

Signed-off-by: Martin Povišer <povik@protonmail.com>
The new class can possibly be shared between the m1n1 AOP tracer and
AOP client.

Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
Signed-off-by: Martin Povišer <povik@protonmail.com>
The AOP uses an 'EPIC' protocol similar to the one other coprocessor
firmware is using but not in the exact same version. Add code for
tracing the AOP calls and extend the aop.py experiment with the client
side of it. Include description of audio calls and some other calls
related to sensor discovery.

Furthermore, in experiments/aop.py, do some AOP audio setup. Once that
is done we can start streaming samples from the internal microphones by
making what AOP considers power state adjustment calls. That is, we
adjust the power state of a 'hpai' device, first to a 'pw1 ' stage,
then to 'pwrd' stage.

So, to see microphone samples, enter the AOP experiment shell first:

  $ M1N1DEVICE=/dev/ttyACM0 experiments/aop.py

Within the shell, adjust the power state of 'hpai':

  >>> aop_set_audio_pstate('hpai', 'pw1 ')

At that point /arm-io/admac-aop-audio powers up. In parallel to the AOP
shell, we can start tools/admac_stream.py on the just powered-up ADMAC
instance:

  $ M1N1HEAP=0x10010000000 M1N1DEVICE=/dev/ttyACM1 tools/admac_stream.py \
        --node admac-aop-audio --channel 1 -v | xxd -g 4 -c 12 -e

Returning back to the AOP shell, we can then set 'hpai' to 'pwrd' state
to kick off the streaming:

  >>> aop_set_audio_pstate('hpai', 'pwrd')

By that point, we should see samples coming out on the ADMAC end. The
samples are 32-bit floats packed in groups of three in a frame, e.g.

00000000: ba7ac6a7 ba32d3c3 baa17ae2  ..z...2..z..
0000000c: 38ccea5f b99c1a37 ba0c4bb1  _..87....K..
00000018: 39d2354f 3964b5ff 39b209fb  O5.9..d9...9
00000024: b96a1d1f 39c8503f 3958fc4f  ..j.?P.9O.X9
00000030: b6b1f5ff 39c72b8f 39bbe017  .....+.9...9
0000003c: 3a912de5 36dd4f7f 37f1147f  .-.:.O.6...7

This has been tested and will to some degree be specific to 2021 Macbook
Pro (t6000). Differences on other models TBD (at the very least the
number of microphones can be presumed different).

Signed-off-by: Martin Povišer <povik@protonmail.com>
The ADT lacks some of the info on AOP power domains on t6000, so fill
that in manually.

Signed-off-by: Martin Povišer <povik@protonmail.com>
@marcan marcan merged commit de0c032 into AsahiLinux:main Nov 4, 2022
@marcan
Copy link
Member

marcan commented Nov 4, 2022

Ha, so AOP uses the same IOP ring buffer/EPIC endpoints with a different tag? Finally some consistency...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants