Understanding the semaphore leak on ESP32/ESP32C3’s NuttX WiFi Driver

The initial issue was “fixed” with a workaround using a NuttX feature (or hook) that enables some function to be called during the task exit (when CONFIG_SCHED_ONEXIT is enabled). More info: https://github.com/apache/incubator-nuttx/pull/4092/

Unfortunately this workaround stopped to work few days ago when on_exit() function was moved from kernel side to user side: https://github.com/apache/incubator-nuttx/pull/6197

Then my colleague Gustavo decided to fix this issue with a proper solution: https://github.com/apache/incubator-nuttx/pull/6322

He also gave a detailed explanation about the root causes of the issue:

The Wi-Fi library creates a new semaphore for every thread that performs connection operations, so we cannot have a global pointer.
Since these semaphores are thread-local, this motivated the initial implementation based on pthread_key_t, so that the semaphores were being stored in Thread Local Storage, and then could be destroyed on thread termination. The solution here was based on the one implemented for ESP-IDF, which works as expected.

But on NuttX it resulted in the semaphores not being destroyed. I'll try to explain why.
The Wi-Fi library operates in a dedicated Kernel Thread, named wifi. But the pthread_key_t and the destructor for the semaphores were allocated to the Thread Local Storage of the init thread.
Every network-related request from the application will be handled by the wifi kernel thread and its child pthreads. The issue is that those child pthreads do not belong to the same Task Group from the init thread, which is the one whose TLS area contains the semaphore destructor.

So the catch here is that NuttX provides this process-like abstraction which segregates pthreads created from different tasks. So a pthread created from Task B won't be able to share keys of type pthread_key_t with another pthread from Task A.

That is it, now the leakage is fixed!

Recording audio call on Android

recorder = new MediaRecorder();
recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
recorder.setAudioSamplingRate(8000);
recorder.setAudioEncodingBitRate(12200);
recorder.setOutputFile(audiofile.getAbsolutePath());
try {
  recorder.prepare();
} catch (IllegalStateException e) {
  e.printStackTrace();
} catch (IOException e) {
  e.printStackTrace();
}
recorder.start();

Source: https://issuetracker.google.com/issues/36907207?pli=1 msg# 504
Update: according to https://developer.android.com/reference/android/media/MediaRecorder.AudioSource.html it should use VOICE_CALL instead of MIC.

Testing Pascal on NuttX

These are the steps to test Pascal VM (https://github.com/patacongo/Pascal) on NuttX:

$ cd nuttxspace/nuttx
$ cd ../apps/
$ ln -s /home/alan/nuttxspace/Pascal pascal
$ ls -l
total 112
-rw-rw-r-- 1 alan alan 8588 mar 22 20:35 Application.mk
drwxrwxr-x 1 alan alan 68 mai 9 14:32 boot
drwxrwxr-x 1 alan alan 126 mai 9 14:31 builtin
drwxrwxr-x 1 alan alan 158 mai 9 14:32 canutils
drwxrwxr-x 1 alan alan 90 mai 9 14:32 crypto
-rw-rw-r-- 1 alan alan 2000 mar 22 20:35 Directory.mk
-rw-rw-r-- 1 alan alan 1415 mar 22 20:35 DISCLAIMER-WIP
drwxrwxr-x 1 alan alan 2374 mai 9 14:32 examples
drwxrwxr-x 1 alan alan 178 mai 9 14:32 fsutils
drwxrwxr-x 1 alan alan 84 mai 9 14:32 gpsutils
drwxrwxr-x 1 alan alan 178 mai 9 14:32 graphics
drwxrwxr-x 1 alan alan 54 mar 22 20:35 import
drwxrwxr-x 1 alan alan 246 mar 22 20:35 include
drwxrwxr-x 1 alan alan 78 mai 9 14:32 industry
drwxrwxr-x 1 alan alan 156 mai 9 14:32 interpreters
-rw-rw-r-- 1 alan alan 60749 mar 22 20:35 LICENSE
-rw-rw-r-- 1 alan alan 3273 mar 22 20:35 Make.defs
-rw-rw-r-- 1 alan alan 6640 mar 22 20:35 Makefile
drwxrwxr-x 1 alan alan 74 mai 9 14:32 math
drwxrwxr-x 1 alan alan 86 mai 9 14:32 mlearning
drwxrwxr-x 1 alan alan 136 mar 22 20:35 modbus
drwxrwxr-x 1 alan alan 464 mai 9 14:32 netutils
-rw-rw-r-- 1 alan alan 161 mar 22 20:35 NOTICE
drwxrwxr-x 1 alan alan 1132 mai 9 14:31 nshlib
lrwxrwxrwx 1 alan alan 28 mai 9 14:32 pascal -> /home/alan/nuttxspace/Pascal
drwxrwxr-x 1 alan alan 112 mai 9 14:32 platform
-rw-rw-r-- 1 alan alan 10377 mar 22 20:35 README.md
drwxrwxr-x 1 alan alan 672 mai 9 14:32 system
drwxrwxr-x 1 alan alan 282 mai 9 14:32 testing
drwxrwxr-x 1 alan alan 202 mar 22 20:35 tools
drwxrwxr-x 1 alan alan 124 mai 9 14:32 wireless
alan@dev:~/nuttxspace/apps$ cd ../nuttx
$ ./tools/configure.sh sim:nsh
Copy files
Select CONFIG_HOST_LINUX=y
Refreshing…
#
#configuration written to .config
#

$ make menuconfig

$ make

In file included from ./libexec_runloop.c:45:
/home/alan/nuttxspace/apps/pascal/insn16/include/libexec.h:61:4: error: #error CONFIG_PASCAL_POINTERSIZE must be 8
61 | # error CONFIG_PASCAL_POINTERSIZE must be 8
| ^~~~~
ERROR: cc failed: 1

Fix:

Application Configuration —>
Pascal Support —>
Target Machine Description —>
(8) Size of a pointer

$ make
...
LD: nuttx
CC: -x c allsyms.tmp
/usr/bin/ld: nuttx.rel: in function libexec_FloatOps': /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:315: undefined reference tosqrt'
/usr/bin/ld: /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:324: undefined reference to sin' /usr/bin/ld: /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:333: undefined reference tocos'
/usr/bin/ld: /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:342: undefined reference to atan' /usr/bin/ld: /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:351: undefined reference tolog'
/usr/bin/ld: /home/alan/nuttxspace/Pascal/insn16/libexec/libexec_float.c:360: undefined reference to `exp'
collect2: error: ld returned 1 exit status

Fix:

Library Routines —>
[*] Standard Math library

$ ./nuttx
login: admin
password:
User Logged-in!

NuttShell (NSH) NuttX-10.3.0-RC0
MOTD: username=admin password=Administrator
nsh> ?
help usage: help [-v] []

. cd echo hexdump mkfatfs ps sleep uname
[ cp exec kill mkrd pwd source umount
? cmp exit losetup mount readlink test unset
basename dirname false ln mv rm time usleep
break dd free ls poweroff rmdir true xd
cat df help mkdir printf set truncate

Builtin Apps:
dumpstack gpio hello nsh prun sh
nsh>