ZDS-2023_ POSIX Roadmap for Zephyr LTSv3.pdf

1、POSIX Roadmap for Zephyr LTSv32023-06-29:Embedded Open Source SummitChris FriedtEmbedded SWE,MetaZephyr POSIX API MaintainerHow does Meta use Zephyr?Meta Scalable Video Processor(MSVP)Why video transcode ASICs?4B videos/day on FacebookPower,Storage,Performance9x faster throughput for H26450 x faster

2、 throughput for VP96x better performance for HQ VOD50%less power consumptionAV1 Coming Soon.How does Meta use Zephyr?Meta Training and Inference Accelerator(MTIA)Why AI ASICs?Feeds,Ads,Content,Ranking.DLRM Models:4.5 GB up to 750 GBPower,Storage,Performance2x Efficiency of todays GPUsPyTorch 2.0MTIA

3、 ISCA 2023Agenda01 Overview of POSIX in Zephyr02 Goals for LTSv303 How its going04 What next?01 Overview of POSIX in ZephyrThe Way Back Machine.01 OVERVIEW OF POSIX IN ZEPHYRPOSIX Turns 35 Years Old!01 OVERVIEW OF POSIX IN ZEPHYRBell Labs UNIX Turns 50 Years Old!*01 OVERVIEW OF POSIX IN ZEPHYR*50 ye

4、ars since UNIX was announced outside of Bell LabsHistory of POSIX in Zephyr01 OVERVIEW OF POSIX IN ZEPHYRWhy POSIX?Portability Mature API Powers 1B _()_/Powers 2B Powers 16B Mobile01 OVERVIEW OF POSIX IN ZEPHYR02 Goals for LTSv3High-Level Goals1.Improve Maintainability2.Improve the application/libc/

5、toolchain Interface3.Improve application/library Portability02 GOALS FOR LTSv3Maintainability Abstract POSIX resources as integers(common representation among libcs)e.g.pthread_mutex_t,pthread_t Re-use Zephyr synchronization primitives within POSIX E.g.use k_mutex internally rather than dogfooding p

6、thread_mutex_t fdtable-zvfs Common library that can be referenced by subsys/net,POSIX,etc ARCH_POSIX POSIX_API ARCH_POSIX Maintainer-RFC#58305 02 GOALS FOR LTSv3Interface Important:POSIX is an interface not a subsystem Any library code required to support POSIX should be Part of Zephyr itself As min

7、imal as necessary to support the interface Must remember to avoid layering violations Support standard include paths for 3rd-party applications and libraries E.g.rather than Kconfig for POSIX feature test macros._POSIX_TIMERS Standard interface for libc/toolchain headers Support POSIX with external

8、libc/toolchain(e.g.IAR)Zephyr must supply POSIX declarations when the libc does not02 GOALS FOR LTSv3Portability:PSE51Ref:IEEE 1003.1-2017PSE51:Minimal Real-time System ProfileSingle,multi-threaded process,no file system,no user or group support,selected optionsDetailed in IEEE Std 1003.13,also see

9、man 7 posixoptionsCompilation Environment:#define _POSIX_AEP_REALTIME_MINIMAL 200312L#define _POSIX_AEP_REALTIME_LANG_C99Options Requirements:_POSIX_C_LANG_JUMP,_POSIX_SIGNALS,_POSIX_SINGLE_PROCESS,_POSIX_THREADS_BASE,_POSIX_CLOCK_SELECTION,_POSIX_FSYNC,_POSIX_MEMLOCK,_POSIX_MEMLOCK_RANGE,_POSIX_REA

10、LTIME_SIGNALS,_POSIX_SHARED_MEMORY_OBJECTS,_POSIX_SYNCHRONIZED_IO,_POSIX_CPUTIME,_POSIX_THREAD_PRIO_INHERIT,_POSIX_THREAD_PRIO_PROTECT,_POSIX_TIMEOUTS,_POSIX_TIMERSExceptions:POSIX_DEVICE_IO FILE ops,scanf(),vscanf()POSIX_SINGLE_PROCESS confstr(),*env()_POSIX_SPORADIC_SERVER(kernel?)02 GOALS FOR LTS

11、v3Portability:PSE51_POSIX_SINGLE_PROCESS Percent Complete:33%Remaining:2sysconf()-incredibly useful.There is also RFC#56670uname()-also kind of useful Exceptions:confstr(),environ,getenv(),setenv(),unsetenv()02 GOALS FOR LTSv3Portability:PSE51_POSIX_SIGNALS Percent Complete:12.5%Remaining:7sigaction

12、()?,sigaddset(),sigdelset(),sigemptyset(),sigfillset(),sigismember(),sigpending(),Exceptions:alarm(),kill(),pause(),raise(),signal(),sigprocmask(),sigsuspend(),sigwait()Notes:With some effort,it is even possible to create a per-thread signal handler.The only thing that cannot be supported with PSE51

13、 are signals that implicitly affect the whole process,since there is only 1 process There may be some lower-layer modifications necessary to create a cancellation point02 GOALS FOR LTSv3Portability:PSE51_POSIX_THREADS_BASE Percent Complete:76%Remaining:12pthread_atfork()pthread_barrierattr_destroy()

14、,pthread_barrierattr_init()pthread_barrierattr_getpshared(),pthread_barrierattr_setpshared()pthread_cleanup_pop(),pthread_cleanup_push()pthread_equal()pthread_kill()pthread_sigmask()pthread_setcancelstate(),pthread_testcancel()Exceptions:None!02 GOALS FOR LTSv3Option Requirements:_POSIX_CLOCK_SELECT

15、ION Percent Complete:0%Remaining:3pthread_condattr_getclock(),pthread_condattr_setclock(),clock_nanosleep(),Exceptions:None Notes:Implies _POSIX_TIMERS02 GOALS FOR LTSv3Option Requirements:_POSIX_SHARED_MEMORY_OBJECTS Percent Complete:0%Remaining:4 mmap()munmap()shm_open()shm_unlink()Exceptions:None

16、!02 GOALS FOR LTSv3Option Requirements:_POSIX_CPUTIME Percent Complete:0%Remaining:1 clock_getcpuclockid()Exceptions:None!Notes:Implies _POSIX_TIMERS Implications:per-cpu counters!Devicetree boolean property:cpu-counter02 GOALS FOR LTSv3Option Requirements:_POSIX_TIMERS Percent Complete:77%Remaining

17、:2clock_getres()timer_getoverrun()Exceptions:aio_suspend()(not in the 1003.1-2017 spec)Notes:This should be done at the Zephyr layer.POSIX becomes a thin wrapper around the Zephyr(#19030,#40099),CLOCK_MONOTONIC-K_CLOCK_MONOTONIC,k_nanosleep(),etc Highly relevant for e.g.Time Synchronized Channel Hop

18、ping(TSCH)in 802.15.4 Official Zephyr chosen nodes in Devicetreezephyr,real_time_clock=&rtc0zephyr,monotonic_clock=&cpu_counter_002 GOALS FOR LTSv303 How its goingSince Becoming Maintainer.After 6 months,9/16 Tasks complete (See 2023-01 Slides from Architecture Meeting)03 HOW ITS GOINGSince Becoming

19、 Maintainer.After 10 months,improved eventfd_read()and eventfd_write()performance by 10 x*with only 1 teensy bug03 HOW ITS GOINGSince Becoming Maintainer.After 11 months,improved pthread_create()and pthread_join()performance by?x*also uncovered k_thread race condition test would crash prior to the c

20、hange,so unable to measure true perf03 HOW ITS GOINGSince Becoming Maintainer.After 12 months,17/54 Tasks complete?*(See RFC#51211)03 HOW ITS GOINGSince Becoming Maintainer.After 12 months,17 38/54=70%Tasks complete*03 HOW ITS GOING*We just broke down the remaining Tasks into smaller,more manageable

21、 ones04 What next?Dynamic Thread Stacks Dynamic Zephyr thread stacks are nearly complete!1!This is MASSIVE-will not only facilitate proper pthreads but(as a result)ISO C11 threads()ISO C+11 std:thread,std:mutex,std:condition_variable,.Other languages?Simple API for both pool and heap allocation(so O

22、K for safety critical)k_thread_stack_allocate(.)k_thread_stack_free(.)Incredible last-minute collaboration with Intel Coming soon.04 WHAT NEXT?CONFIG_ARCH_POSIX&CONFIG_POSIX_API If all goes well,the ability to build/run/test Zephyrs POSIX API on native_posix and native_posix_64 Actually,native_sim,a

23、nd native_sim_64(new boards in the process of being added)Recent work presented at the Arch meeting by the POSIX Arch Maintainer(now at Nordic)04 WHAT NEXT?The great news about breaking down the remaining tasks into smaller,more manageable tasks,is that a large percentage of them are trivial to impl

24、ement Great opportunity for new contributors to Zephyr!POSIX Collaborators welcome!Existing contributors to Zephyr could get their name into MAINTAINERS.yml Pick the low-hanging fruit04 WHAT NEXT?List of missing POSIX functions with trivial implementationssysconf(),uname(),sigaddset(),sigdelset(),si

25、gemptyset(),sigfillset(),sigismember(),sigpending(),pthread_atfork(),pthread_barrierattr_destroy(),pthread_barrierattr_init(),pthread_barrierattr_getpshared(),pthread_barrierattr_setpshared(),pthread_cleanup_pop(),pthread_cleanup_push(),pthread_equal(),pthread_kill(),pthread_sigmask(),pthread_condattr_getclock(),pthread_condattr_setclock(),clock_getcpuclockid()04 WHAT NEXT?Questions?/Feedback