EOSS-EU23-Introducing a New Zephyr Sensing Subsystem.pdf

1、#EMBEDDEDOSSUMMITHebo Hu,Ke HanIntel CorporationIntroducing a New Zephyr Sensing Subsystem Zephyr Sensing Subsystem Agenda Items Why do we need it?Background What is our goal?Motivation What is it?Introduction What can it do?Application How far did we go?Status How does it work?Demo What is next?Nex

2、t PlanZephyr Sensors TodayTypical Zephyr Sensor Usages device tree config sensor fetching sensor triggering sensor attribute sensor calibration sensor processing 140 device drivers 900 commitsImprovement area:sensor applications are directly interacting with sensor device drivers for all sensor oper

3、ations,there is a lack of high-level abstraction,general function management,multi-client arbitration,etc.in Zephyr Sensor solutionBackgroundSensor Device DriverSensor Device DriverSensor Device DriverSensor ApplicationSensor ApplicationSensor ApplicationMotivationTake Zephyr Sensor Solution to Next

4、 LevelPlan:introduce a new sensing subsystem/framework to support Sensor high level abstraction General function management Multi-client arbitration Sensor topology and scheduling Sensor fusion and processing Sensor timestamping Sensor batching And more Furthermore:With additional components,new Sen

5、sing Subsystem shall be able to serve as sensor framework in sensor hub firmware,to support OS sensing stack(Windows*,Chrome*,Android*,Linux*)and CHRE*.Sensor Device DriverSensor Device DriverSensor Device DriverSensor ApplicationSensor ApplicationSensor ApplicationNew Zephyr Sensing Subsystem*Other

6、 names and brands may be claimed as the property of othersIntroductionUser Space(if configured)Kernel SpacePlatform HWSensor Driver APISensorAPI(single client)I2CSPIUARTGPIO.GyroAccMagALSZephyr Low Level APIsIPCSensing Subsystem API（multiple clients support）Zephyr High Level APIsZephyr existing sens

7、or device driver APIZephyrProposed new Zephyr subsystemExisting sensor stack in ZephyrSensing Subsystem Scheduler taskRuntimeSensor Objects Mgmt.Sensing Sensor APIPhysical SensorPhysical SensorVirtual SensorPhysical SensorPhysical SensorPhysical SensorSensor Subsystem Proposal from Intel(PR#45370)Se

8、nsing Subsystem Upstream PR1 from Intel(PR#55389)Sensing Subsystem Upstream PR2 from Intel(to be created)RTIO SubsystemIO(DMA/I2C/SPI/)New Sensing Subsystem runs on top of existing Zephyr sensor device drivers,serves as sensor management framework,and provides higher level sensor APIs for multiple Z

9、ephyr applications to useClient 2Client 1Client 3Zephyr ApplicationSensor AbstractionVirtual Sensor:sensor algorithmsPhysical Sensor:wrapper of exiting sensor device driverSensor commonality handled by sensor frameworkTree ModelConfig flow from top to bottomData streaming flow from bottom to topSing

10、le thread scheduling driven by data and configMultiple Clients w/ArbitrationEach Sensor can be opened&configured by multiple clients including App clients and Sensor clientsArbitration:report interval,change sensitivity,etc.Support CHREAPI can meet CHRE PALs requirementsSupport None-CHRE Zephyr Appl

11、icationZephyr KernelCHREZephyr ApplicationHost Link Phy 1Virt 1Phy 2 Drv1SensingSubsystemarbitrationDrv2Other Client Off-loadable NanoappSensor device driver in Zephyr KernelArbitrationPhysical Sensor in Sensor FrameworkSensor Implemented in nanoappSensor Exposed by Sensor FrameworkVirtual Sensor in

12、 Sensor FrameworkConfig PathData PathIntroductionIntroductionZephyr Sensing Subsystem FeaturesDesign PrinciplesScopeo focus on sensor fusion,arbitration,sampling,timing and schedulingSensor Abstractiono sensor object for representing sensor instance,physical&virtualo reporting tree model for sensor

13、fusion,including data&config pathsData Driven Modelo polling mode:periodical sampling rateo interrupt mode:data ready,threshold interrupt etc.Schedulingo single thread main loop for all sensor objects sampling&processingBuffer Mode for BatchingSensor Power ManagementRely on Zephyr APIso access physi

14、cal sensor devices via Sensor APIo others,such as Task,Memory,RTIOConfigurable Via Device Treeo enabled or not,supported sensors if enabledo defines sensor reporting relationships(including reporter sensors for virtual sensors)Referenceo reference from Intel ISH proprietary sensor framework design w

15、hich has been in mass production for 10+years w/active evolutionUpstreamo fully open source as Zephyr native subsystemo start upstreaming in 2023o work together with Google*Configurabilityo reusable standalone subsystemo build on Zephyr existing low-level Sensor APIo provide Zephyr high level Sensin

16、g Framework APIo separate optional CHRE Sensor PAL Implementation module to support CHREo decouple from any external communication protocols,its Zephyr Applications role to handle different protocols(MQTT,HID,or Proprietary,all configurable)Applicationo Chrome,Linux,Windows,Android OS sensing suppor

17、to CHRE(Context Hub Runtime Environment)PAL support*Other names and brands may be claimed as the property of othersOS SoftwareISH FirmwareIntel Integrated Sensor Hub(ISH)Implementation Example for Chrome OSUser Space(if configured)Kernel SpaceISH HWSensor API(or V2)Zephyr Low Level APIsHIDSensing Su

18、bsystem APIZephyr High Level APIsHID APISensor Driver APIGyroAccMagALSHECIHECI APIZephyr ApplicationIntel HAL(SEDI)I2CDMAUARTGPIO.IPCHPETPeripheral APIs(IPC/HPET/I2C/DMA/UART/)Shim Drivers for Intel HALPeripheral Driver APIs(IPC/HPET/I2C/)Chrome OSPCI Bus DriverIPC DriverISHTP driverIIOHID HECI Clie

19、ntHECI over IPC over PCIHost LinkIPCHECI PackageHID SensorSend to HostHID PackageISH SOC SpecificPower Mgmt.Paging Back-end Exception.AlgoIntel/Google(upstream to Chrome)Existing Open-Source CodeGoogleIntel(upstream to Zephyr)Vendor/OEM(Close source)Intel(reference code)EC toolsEC TaskEC HECI Client

20、Sensing Subsystem Scheduler taskRuntimeSensor Objects Mgmt.Sensing Sensor APIPhysical SensorPhysical SensorVirtual SensorPhysical SensorPhysical SensorPhysical SensorApplicationSensor PAL ImplementationNanoappCHREFrameworkSensor PAL APIOff-loadable NanoappAcc2 EventAcc1 EventHinge AngleAlgoSupport I

21、ntel Chrome Client Reference Platform sensor BOMFeatures implemented All sensing subsystem APIs except batching Fusion framework with physical and virtual sensor abstraction Multiple clients with CHRE and HID sensor apps running at the same time Multiple clients arbitration on sensor report interval

22、 and sensitivity Utilization of Zephyr sensor device API for physical sensors Polling and interrupt data driven Single thread scheduling Configurable with device tree E2E sensor streaming data path with HID/IIO for demo(physical sensor+virtual sensor)Upstream workSensing Subsystem Proposal has been

23、reviewed with Zephyr communitySensing Subsystem Upstream PR1 has been submitted and is under reviewStatusDemoIntel Client Reference Board with ISH(Minute-IA)Physical sensors:Base Acc:BM160(I2C)Lid Acc:BM160(SPI)Virtual sensors:Hinge angle(Base Acc+Lid Acc as inputs)Motion detector(Lid Acc as input)F

24、RDM-K64F Board(ARM Cortex-M4F)Physical sensors:Base Acc:BM160Lid Acc:fxos8700 Virtual sensors:Hinge angle(Base Acc+Lid Acc as inputs)Motion detector(Lid Acc as input)Thanks Tom Burdicks great enabling work!UpstreamingNext PlanZephyr 3.4 Introduce sensing subsystem core(PR)Zephyr 3.5Extend sensing su

25、bsystem features and board supportZephyr 3.6(LTS 3)Stabilize sensing subsystem End to End Chrome Sensors SupportMeet Google/Intel Chrome production requirementsEnable Intel ISH with Latest UPX Open Dev Board Advanced Sensing Features CHRE end to end supportCHRE end to end supportCrossCross-OS suppor

26、t with HID sensorsOS support with HID sensorsHuman presence sensingHuman presence sensingSensor batchingSensor batchingMultiMulti-thread processingthread processingMoreMoreNotices&Disclaimers Intel technologies may require enabled hardware,software or service activation.No product or component can b

27、e absolutely secure.Your costs and results may vary.Intel does not control or audit third-party data.You should consult other sources to evaluate accuracy.All product plans and roadmaps are subject to change without notice.The products described may contain design defects or errors known as errata w

28、hich may cause the product to deviate from published specifications.Current characterized errata are available on request.Intel Corporation.Intel,the Intel logo,and other Intel marks are trademarks of Intel Corporation or its subsidiaries.Other names and brands may be claimed as the property of others.back