TestingAWirelessIoTProduct_AndreasMueller_Reto_Schneider.pdf

《TestingAWirelessIoTProduct_AndreasMueller_Reto_Schneider.pdf》由會員分享，可在線閱讀，更多相關《TestingAWirelessIoTProduct_AndreasMueller_Reto_Schneider.pdf（68頁珍藏版）》請在三個皮匠報告上搜索。

1、IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksTesting a Wireless IoT ProductThings We Wish Someone Told Us BeforeAndreas Mller&Reto SchneiderJune 29,2023IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio C

2、ommunication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingS

3、DR&GNU RadiopytestConclusionQuestionsAppendix&LinksDisclaimerBackgroundWe are no RF experts,just enthusiastic developers who wentwhere nobody they have not went before.Affi liationworking for Gardena(Husqvarna Group)collaboration with Lemonbeat GmbHno affi liation with mentioned toolsBiasopen source

4、tools usable under Linuxtools usable via API or CLIIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksContextIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksMissionFuture Proof Systemhardware agnosticmake and collaborate,instea

5、d of buybackward compatibilityprotocol re-implementation based on Zephyr RTOSLemonbeat PHY/MAC to provide IPv6 connectivityLemonbeat PHY/MAC860MHz.870MHz2GFSK,100 kbps30 channels(4+1 active)13dBm output power104dBm sensitivityIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAp

6、pendix&LinksTestingBackgrounddifferent types of testing:unit testing,device based,.device based testing with actual transceiver&MCUGoalsverifi cationavoiding regressionsevaluating performanceChallengestests must be repeatableenemy#1 for repeatability:RF interferenceIntroRadio Communication&Shielding

7、SDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksRF Spectrum:Home Offi ce:Good TimesIntroRadio Communication&Sh

8、ieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksRF Spectrum:Home Offi ce:InterferenceIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksRF Spectrum:Home Offi ce:InterferenceIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&

9、LinksRF Spectrum:Offi ce SituationOffi ce Situation4 teams developingproducts&features20+gateways,50+devicesResultinterference from our own devices(not so much 3rd party)RF spectrum usually pretty busyBottom LineOur best option is shielding the test system.IntroRadio Communication&ShieldingSDR&GNU R

10、adiopytestConclusionQuestionsAppendix&LinksRF Spectrum:MitigationsUnsuccessful/Non-feasible Mitigationscoordinate testing time-slotsdiffi cult,inflexible,no guaranteesmove testing to secluded locationexpensive&cumbersomeBottom LineOur best option is shielding the test system.IntroRadio Communication

11、&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksBackground:dB/dBm Values&Link BudgetdB and dBmBel(B):logarithmic value indicating a proportiondB:one tenth(deci)of a Bele.g.120dB=12B=1012dBm is a power value:dB with reference 1mWe.g.20dBm=102 1mW=10WLink Budget(Oversimplifi ed)accountin

12、g of RF power with gains&lossesdifference between TX power and weakest RX signal(i.e.lowest power we can still receive)e.g.TX power+13dBm and sensitivity 104dBm:path loss must be 117dBIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksBackground:RF ShieldingRF Shie

13、ldingfor effective shielding,device must be fully enclosed inconductive materialunfi ltered connections(USB,etc.)conduct interferencerule of thumb:enclosure with opening(longest dimension)1/20 wavelength of interference signal provides 20dBshieldingafor 865MHz:wavelength 35cm smallest opening 1.7cm

14、should result in 20dB shieldingwe need much more than 20dBahttps:/ Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksShielding:Failed Approach I:Aluminum Rack BoxIssuesnot designed for shielding(despite RF shielding kit)cover not fully connected(gap between cover&box)unfi lt

15、ered connectors 60dBmeasured attenuation:85dB at 865MHzgreat,but not enoughIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSolution:Two-Stage ShieldingTwo-Stage ShieldingTwo-stage shielding withshielded rack(55dB)andshield boxes(85dB)inside.Two-Stage Shielding:T

16、otalAttenuation(Theoretical)140dB offi ce to deviceunder test(DUT)140dB DUT to offi ce 170dB between boxesIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksShielding:Final Setup&LearningsBottom LineFor shielding,there is no easyway to cheap out.Just pay$for a comm

17、ercial solution.aaThere are some cheap but diffi cultoptions.Ask us if you really want toknow.IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSide Quest:Using a Microwave Oven for Shielding?Microwave for Shieldingmust shield at2.4GHz by designshould also workwel

18、l at 868MHz(longer wavelength)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksUsing a Microwave Oven for Shielding:Test Setup(I)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksUsing a Microwave Oven for Shielding:Test Setup(

19、II)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksUsing a Microwave Oven for Shielding:ResultsResultsonly about 20dB.30dB attenuation at868MHzbetter at 2.4GHz:50dBIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksPart II:RF C

20、onnectionRF Connection:Motivationsignal from antenna reflected inside shield boxinconsistent&uncontrolled RF pathsSolutionsRF absorption foamonly partial solution:RF paths still uncontrollednot tried by usconnect devices via coaxial cablesclearly defi ned RF pathsallows measuring power,testing sensi

21、tivity,.IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksRF Connection:Power SplitterQ:How to connect all devices?Without antennas,we need a replacement for our medium(air).A:Power Splittersymmetrical splitter required(resistive type)resistive power splitter has

22、highattenuation:1N2ideal power splitter:1Nwe need attenuation anywaysconfusion:11-way 12 portsprice:50.1000$(for 10 ports)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksPower Splitter:DIYRoll Your Ownjust a few SMD resistors&SMA connectorsonly needs to work wel

23、l at865MHzenclosure not reallyneededcan also include othercomponents,e.g.attenuationmanufacturing cost(incl.assembly)95$for 5 pieces8-Port Splitter with AttenuationIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksCommercial vs DIYCommercial Power Splitterworks we

24、ll across all specifi ed frequenciesexpensive,10 ports hard to fi nd,long lead timesaccurate,useful in reference test jigDIYworks well at required frequenciescheapno case/no shieldinguseful for test jigs&in home offi ceIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&

25、LinksAttenuationAttenuationprevents overload&damageideally 40dB.80dB 80dB makes no difference(direct coupling TX RX)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksProgrammable AttenuationProgrammable Attenuatorconfi gurable attenuation(via Ethernet or USB)range

26、 e.g.0.60dBinsertion loss,e.g.4dBallows sensitivity testingavoid direct couplingseparate shield boxesIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksShield Box with Power Splitter&AttenuatorsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestion

27、sAppendix&LinksTest System:Block DiagramIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAp

28、pendix&LinksSDRWhat is Software-defi ne radio(SDR)?idea:build radio in software rather than hardwaree.g.signal fi ltering,modulation/demodulation,etc.SDR in an ideal world:reality:limited sample rate,bit depth,bus speed,CPU,.analog front-end needed(e.g.fi ltering,mixing to IF)baseband processing nee

29、ded(e.g.FPGA)general purpose SDR hardware allows flexible useIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSDR HardwareHardware SelectionRequired hardware depends on:RX only vs RX/TXtarget frequencyrequired bandwidth/sample ratefull-duplex vs half-duplexrequir

30、ed clock stabilityrequired number of channelsspace(ideally keep SDR inside shield box)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSDR HardwareLow-Cost:RTL-SDR DongleFor low-bandwidth(2MSPS)RX only,RTL-SDR dongle for$20 may be suffi cient:Medium-Cost:bladeRF

31、2.0bladeRF 2.0 micro is a nicemedium cost($540)option:IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSDR:Use CasesWhat do we use it for?checking for interferencegenerating interferencenoisespecifi c signal typessending and receiving packetsin combination with s

32、capyallows using bogus header values(fuzzying)basic qualitative measurementse.g.check if signal is presentnot built for accuracy(esp.amplitude better usespectrum analyzer)IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksGNU RadioWhat is GNU Radio?GNU Radio is a f

33、ree&open-source software developmenttoolkit that provides signal processing blocks to implementsoftware radios.aGNU Radio blocks:software equivalent to Mini-Circuitscomponentse.g.20dB amplifi er xi=xi 100ahttps:/www.gnuradio.org/about/What is it useful for?allows quickly building SDR flow graphsmany

34、 protocol implementations freely availablesame implementation for virtual&physical testsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksSDR&GNU Radio:Practical ExampleRF InterferenceEmulating Interferencesignal generatoractual headsetGNU Radio&SDRGNU Radio Flow-

35、Graph for FM InterferenceIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&Linkspyt

36、est:IntroAbout pytestpytest is a mature full-featured Python testing tool thathelps you write better programs.aalso perfectly suited for hardware based testingahttps:/pytest.orgWhat we likeeasy to use,but supports complex use casestest auto discoveryparametrization(using same test with different par

37、ameters)many useful pluginsmodular fi xturesIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&Linkspytest:FixturesFixturesprovides a defi ned,reliable and consistent contextaa web server listening on a specifi c porthardware in a defi ned statescopes:session,module,fun

38、ctiondependenciespytest handles arrangement of fi xturesahttps:/docs.pytest.org/en/7.3.x/explanation/fi xtures.htmlHintsuse sensible scopes&dependencieshide complexity in fi xtures,keep tests simpleIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksFixtures:Example

39、Example:Fixtures for Power Supplypytest.fixture(scope=session)def power_supply():power_supply=MyPowerSupply()yield power_supplypytest.fixture(scope=function)def power_supply_3v3(power_supply):power_supply.set_voltage(3.3)power_supply.enable_power()yield power_supplypower_supply.disable_power()IntroR

40、adio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&Linkspytest&Zephyr:IntegrationZephyr shellmanual testingusing shell commands from pytesthint:disable logs,shell colors&shell echomany useful existing modules,kernelliberally add new shell commands for own codeIntroRadio Commu

41、nication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksZephyr:More Useful FeaturesZephyr&pytestparsing.config in pytestsanity checks(feature actually enabled?)adapt test to build(e.g.selected RF channel)zperfOutside pytestearly testing by other teams:Ethernet over USBMCUmgrFOTA testin

42、gShell via UDPIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksPractical Example:Sensitivity TestingQuestionHow much attenuation can there be between sender andreceiver until we start losing packets?Approach 1NUM_PKTS=10for attenuation in range(40,71):attenuator.

43、set_attenuation(attenuation)dut.rx_mode()sender.send_pkts(NUM_PKTS)pkt_loss=1-(dut.pkts_rx()/NUM_PKTS)#store resultsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksExample:Sensitivity Testing:Conclusive Results?ExpectationWe determined the sensitivityto be about

44、 52 dB+base pathattenuation(60 dB)112 dB.What Actually HappenedDUT crashed when step was at52 dB.IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksExample:Sensitivity Testing:Using pytestApproach 2:Using pytestpytest.mark.parametrize(att,range(40,71)def test_sensi

45、tivity(dut,sender,att)attenuator.set_attenuation(att)#.Benefi tsUsing fi xture to get defi ned state for each step(e.g.DUT isreset).However.Still not all variables controlled,e.g.device warming up overtime.IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksExample:

46、Sensitivity Testing:Random OrderAproach 3:Random Orderpytest-random-order plugin,can randomize steps(no codechange required!):pytest-random-order test_sensitivity.pyResult This avoids spurious correlation with uncontrolled(buttime-dependent)variables.Bottom Linecontrol as many variables as possiblea

47、void correlation with uncontrolled variablesIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestio

48、nsAppendix&LinksConclusionsWhy testing with physical devices?useful during developmentverify new featuresperformance testingreproduce issuesuseful during maintenanceprevent regressionsmodel&react to developments in 868MHz spectrumWhat helps?shielded&cable-connected setup for repeatabilitypytest is g

49、reat fi t for hardware-based testingSDR&GNU Radio helps a lot with RF workIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&LinksIntroRadio Communication&ShieldingSDR&GN

50、U RadiopytestConclusionQuestionsAppendix&LinksQuestions?Thank youThank you for your attention.ContactYou can reach us IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksOutline1Intro2Radio Communication&Shielding3SDR&GNU Radio4pytest5Conclusion6Questions7Appendix&L

51、inksIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksAppendix:USB(I)USBUSB tends to become unreliable when 10+devices areconnected:discovery takes longerdevices not correctly discoveredunstable connectionProblem Sourcesbad or broken cablesnot properly plugged in

52、connectorsin some cases just not plugged in straightin some cases not fully plugged in due to mechanical issueselectrical issues,e.g.ground loopsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksAppendix:USB(II)SolutionsUSB 2 tends to be more reliable than USB 3uh

53、ubcontrollooks great,but didnt fully work with our USB hubsYepkit YKUSH&YKUSH 3nice hardware,works(mostly)wellsoftware no longer actively developed;segfaults in somecasesConclusionWe found no solution were fully happy with.If possible,just use Ethernet instead:-/MiniCircuits devices often support bo

54、th USB and EthernetIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksAppendix:Used EquipmentCost Saving:Using 2nd Hand Equipmentolder(e.g.80s)RF equipment often has good qualitycan be bought at reasonable price off eBay or fromspecialized vendorideally should have

55、 recent calibrationchoose device with GPIB interface to allow integration intoautomated testing with GPIB to USB converterbonus:equipment often comes with really comprehensiveservice&repair manuals,often including schematicsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppe

56、ndix&LinksAppendix:AliExpressAliExpress RF Devicesoften built around a specifi c chip using reference circuit(more or less)saves time compared to using the chip&going DIY routeperformance often surprisingly OKish given the priceno guarantees,may behave unexpectedlyIntroRadio Communication&ShieldingS

57、DR&GNU RadiopytestConclusionQuestionsAppendix&LinksAliExpress:Example:RF Power MeterAliExpress RF Power Meterbuilt around AD8318logarithmic detectorspecs of RF powermeter on AliExpress aremostly just the specs ofthe AD8318 chipprovides output viasimple UART protocolIntroRadio Communication&Shielding

58、SDR&GNU RadiopytestConclusionQuestionsAppendix&LinksComparison:AliExpress vs Mini-CircuitsIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksComparison:AliExpress vs Mini-CircuitsPros:Mini-Circuitsabout 10 x moreaccuratewider rangeperforms as specifi edwell documen

59、tedinterfaceuseful in reference testjigPros:AliExpressabout 100 x cheaperbehaves mostly asexpecteduseful for development&testing in home offi ceIntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:DisclaimerDisclaimerAll links are for tools and products we gene

60、rally like.For thecheaper products,this doesnt necessarily mean the product isof high quality,but rather we think the quality is reasonablegiven the price.We decided to avoid pointing out any of the products we dislike,as were not sure if the products are actually bad or we simplydidnt do enough tes

61、ting or used a product the wrong way.IntroRadio Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:RF Hardware(I)RF Power Splitterhttps:/ Shield Boxeshttps:/ Shielding Cabinethttps:/ocotec.fi/en/product-platforms/rf-emc-shielding/IntroRadio Communication&ShieldingSDR&GN

62、U RadiopytestConclusionQuestionsAppendix&LinksLinks:RF Hardware(II)RF Building Blocks:Cables,Plugs,Attenuators,etc.https:/ Power Meterhttps:/ Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:SDRGNU Radiohttps:/www.gnuradio.org/https:/www.cgran.org/SDR Hardwarehttps:/

63、Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:Other HardwarePower Measurementhttps:/ ler-Kit-2GPIB Interfacehttps:/prologix.biz/(note:we only tested the USBinterface;it works well but unfortunately got signifi cantlymore expensive a while ago)USBhttps:/ Communicati

64、on&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:DIYRF:Power Splitterhttps:/ Attenuatorhttps:/www.electronics-tutorials.ws/attenuators/pi-pad-attenuator.htmlManufacturinghttps:/ Communication&ShieldingSDR&GNU RadiopytestConclusionQuestionsAppendix&LinksLinks:SoftwareTesting:pytest&Useful Pluginshttps:/pytest.orghttps:/