Realtime MP3 encoder 48kHz Stereo

[psychoacoustic]

I have ported and optimised an MP3 compliant encoder to ESP32, just wondering if anyone is interested in the project. If so I will clean up the code and post it here.

Current status:
Encodes @48000Hz 16bit Stereo using between 37-43% on one core depending on where you choose to place buffers and tables
Memory:
30312bytes MALLOC_CAP_8BIT for global state
21904bytes MALLOC_CAP_8BIT encoding struct
2 * 9216bytes tables can be in IRAM
2 * 2304bytes buffers also can be in IRAM
Total: 75256

Quality: (From the Youtube Free Audio Library https://www.youtube.com/audiolibrary/music)
Captured streaming over wifi in Chrome 48kHz Joint Stereo 256kbps using Audacity saved as FLAC:
All Hail the Queen.flac
http://pc.cd/twDctalK
Alone Time.flac
http://pc.cd/n6KrtalK
In the Distance.flac
http://pc.cd/Kzq7
Mover.flac
http://pc.cd/T30rtalK
On the Windy Road.flac
http://pc.cd/5t8
Parisian Funk.flac
http://pc.cd/U9F
Names included so you can compare these with the original in the Youtube library.

Moving things into IRAM pushes performance to the 43% end, otherwise its around 37%.
By using MP3 as a format, I can stream audio to ANY web browser. All that I've tried work fine, including phones and tablets.

[ESP_Sprite]

Nice! I think it would be very useful to have as an open-source component, if you're willing to make it one. Can I ask what library you ported?

[psychoacoustic]

Yes of course, the encoder is called shine mp3 encoder. I started with the fixed point implementation but am abandoning that in favour of single precision floating point after some research and synthetic testing on ESP32 show that asm optimisation should be much faster using the FPU. The FPU instructions madd.s and lsiu are of particular interest as the 32bit integer options are quite limited.

Does the ESP32 have one FPU per core as would seem to be the case? Also the release notes I've found suggest a 4 instruction pipeline and up to 2 FP instructions per clock? I'm assuming that without longer instruction words this is intended to handle madd.s and msub.s by performing them as two FP instructions.

Note that shine is fast because it does not perform psychoacoustic analysis and I therefore only recommend use for streaming, not for recording.

[ESP_Sprite]

The ESP32 has indeed one FPU per core. I think the FPU is pipelined through the main Xtensa pipeline, which iirc is 7-stage, but I'm not too sure about the details there.

[Carlos.Latour]

Is it Possible to get the code ? (do you have a github addy, or a zip file we can download )

The result is rather Good, are you feeding the 'source' sound from mics ? or from a local file ?

Thanks,
Carlos.
PS: I'm interested in doing this for mic input (mono-SB16 @ 16K sampling or better ) with output @ 8Kbps.

[psychoacoustic]

Carlos, sorry for the incredibly late reply, as shine only implements MPEG1 Layer 3 the lowest supported bitrate is 32kbps and samplerate is 32kHz, and I haven't tested that. Generally the lower the bitrate, the poorer the performance of shine due to no psychoacoustic processing.

The source is digital optical input from a Windows PC playing the samples directly from the youtube library linked in the OP.

There would be much better options for your application, I imagine there are some suitable ones in the ESP-ADF.

[allex1978]

Hi psychoacoustic, I'm very interested in your ported library. can you share it somehow?

Also i saw you video with "ESP32 Audio processor"...amazing beautiful GUI.

[PaulNi]

Very impressive!

Back in my high school years, when Shenzhen was still pumping out MP3 players, there were makers of very cheap codec chips that were connecting to common microcontrollers and were handling MP3 decoding for them over serial or parallel.

A year ago, I had a need to find one, but to my surprise the price for those chips went through the roof! $10+ for a simple codec chip. Now it is cheaper to buy a microcontroller with a code on board, than a standalone chip.

[psychoacoustic]

@allex1978, thanks mate, I can't take all the credit for the GUI as the knobs are available on one of those javascript sites, I can't remember where I'll look it up.. here I think this is it https://github.com/jhnsnc/precision-inputs . Little bit tough to get working but they really are worth the effort. Though for some reason firefox can't draw the arcs on the outside.

The shine mp3 library is available on git I think, or maybe one of those odd collections of old mp3 codecs. It does take a not insignificant amount of effort to shoehorn into the ESP32 though, I will endeavour to actually get around to creating git repos for some of the things I did on this project:
- ShineESP32
- A tiny (no) memory footprint 'in-place' FFT stolen from lameMP3, which was originally from Stanford I think, but it's been GPL'd or something now, but wow it was hiding away in there. Saves 8 x bufferlength bytes vs http://www.robinscheibler.org/2017/12/12/esp32-fft.html if you also do left and right sequentially instead of in parallel. That is to mean Left <FFT>> Freqs->Modify <iFFT>> Left(EQd) then do right channel.
- Constant Q spectrum analysis, one band for every note on the scale from C0 to B8 http://pages.mtu.edu/~suits/notefreqs.html the point of which is to detect notes on the lower end with high time resolution which can't be done with FFTs. This is mainly about building a filter bank that's stable enough and low enough order to give the desired time resolution.

For the sample code what do you think would be the best input output methods? MP3 streaming from I2S to HTTP? I can't think of any other use for it. What do most people in the community have audio-wise when they have audio on ESP32? It was just a challenge from a mate, when I showed him the GUI on my phone he said 'but where's the sound?'.

Can you help tidy up the shine lib?

I still think full psychoacoustics is possible on ESP32 with two FPUs but 1) shine is integer based and 2) I'd rather spend time getting AAC decoding or something for bluetooth audio.

@PaulNi LOL in Australia they've always been $10, they've probably gone up because it's the only way to get an Arduino Uno to play an MP3!

[getvobot]

@psychoacoustic thanks for the information.

I tried porting of Shine encoder to esp32. No optimization has been made yet. The ESP32 is running at 240MHz. Use -Os while compiling. I pushed the example code here: https://github.com/myvobot/esp32_mp3_encoder.

A simple benchmark shows:
Read WAV from SPIFFS, don't write to MP3, it takes 11s to convert a 10s mono WAV.
Read WAV from SPIFFS, and write to MP3, it takes 22s to convert a 10s mono WAV (and slower if run more than once).
No significant diffence if we turn on the PSRAM，TICK=1000.

The result is not as good as discussed yet. Could you please share some optimization ideas?