Daniel DeGrasse- Graphics Support in Zephyr RTOS.pdf

1、Graphics Support In Zephyr RTOSDaniel DeGrasse,NXP1PUBLICABSTRACTIntro to 2D GraphicsCurrent Graphics Support in ZephyrOverview of Graphics HardwareZephyrs Use CasesExisting APIsZephyr API ProposalFuture WorkOpen Discussion2PUBLICVECTOR VS RASTER GRAPHICSVector graphicsMathematical description of im

2、ageBest for logos,engravings,fonts,UI elementsRaster Graphics2D pixel arrayBest for photographs or other imagesIntro to Graphics3PUBLICBlendingPorter-duff blending modesPrimarily applies when one shape has alpha componentCan also be used for more complicated clipping operationsClippingAny output sha

3、pe can be clippedBoundaries are typically just a rectangle,versus blending two shapes togetherBLENDING AND CLIPPINGIntro to Graphics4PUBLIC Path commands allow drawing complex shapesUsed in formats like SVG,as well as 2D vector graphics enginesRendering can be done within path,or only on pixels with

4、in an odd number of pathsPATHSIntro to Graphics5PUBLICCURRENT STATE OF ZEPHYR GRAPHICS SUPPORTCFB subsystemFonts can be rasterized,and rendered onto a monochrome displayBasic shape drawing(rectangles and points)also supportedMonochrome onlyLVGLUpstream has support for several graphical acceleration

5、engines,but Zephyr does not enable thisRich set of APIs for graphics accelerators to implementAll drawing is done in software when using ZephyrState of Support6PUBLICNXP PXPMultiple blending modesBlends alpha surface onto background(process surface)90/180/270-degree rotationColor space conversionPro

6、cess surface scalingCan output directly to the LCD controller without an intermediate bufferNXP SmartDMAPixel format conversionDirect output to display via MIPI-DSI or FlexIO attached display90/180/270-degree rotationPXP AND SMARTDMAHardware Engines7PUBLICFast clear-set buffer to solid colorBlends f

7、oreground surface onto backgroundColor space conversionsColor lookup table to limit input framebuffer sizeDMA2DHardware Engines8PUBLICBased on Vivante GPU IPSupported by VGLite APIBlendingVector based pathsGradients and solid fillCan clip output image within a pathTransformation matrix for arbitrary

8、 rotation,scaling,and translationPath clippingGC355/GCNANOLITE-VHardware Engines9PUBLICZEPHYR USE CASESBasic vector graphicsRectangles,ellipses,arcsArbitrary vector pathsIncludes arbitrary scaling and transformationBasic raster graphicsScaling,90/180/270-degree rotationBlendingClippingAdvanced raste

9、r graphicsClipping raster within arbitrary pathArbitrary scaling and transformDirect output to displayGraphics engine transforms buffer and writes output directly to displayUse Cases1 0PUBLICVGLITE APIVector GraphicsPath drawing based on draw commands(similar to SVG format)Gradient and solid color f

10、illRotate or scale path using transformation matrixRaster GraphicsClip image within path based on draw commandsRotate or scale image using transformation matrixSubset of Porter-duff blending modesDesigned by Verisilicon for use with 2D GPUs like those on NXP RT1170 and RT595API lacks“simple”graphics

11、 functionsNot realizable on hardware like NXP PXP or ST DMA 2DNo support for writing directly to a display outputExisting APIs1 1PUBLICLVGL BACKENDVector GraphicsRectangle,arc,letter,line,and polygon draw functionsNo support for paths like those present in VGLiteRaster GraphicsDraw image with output

12、 coordinates,angle,and zoom(arbitrary transformation)Subset of porter-duff blend modes supportedDirect output to display could be implemented within display flush callbackNo support for arbitrary paths like those in VGLiteCan be supported by all listed hardware enginesExisting APIs1 2PUBLICARM2DOpti

13、mized software backend for Cortex-M MCUsSupports adding custom GPU backendsDraw operationsDraw rectangleRaster operationsCopy image into output buffer,with alpha blendingTransform image(rotation and scaling)Color space conversionDrawing APIs are limited,no direct output to displayAPI can be implemen

14、ted as a software backendExisting APIs1 3PUBLICAPI PROPOSALAPIDescriptionHardware Enginesgpu2d_draw_rectDraw basic rectanglePXP,DMA2D,GC355,GCNanoLiteVgpu2d_draw_lineDraw lineGC355,GCNanoLiteVgpu2d_draw_arcDraw arc,or full circleGC355,GCNanoLiteVgpu2d_draw_pathDraw path using lines,cubic and quadrat

15、ic curvesGC355,GCNanoLiteVgpu2d_blit_pathCopy source buffer onto path described by lines,cubic and quadratic curvesGC355,GCNanoLiteVgpu2d_blit_imgCopy source buffer onto output buffer with optional 90/180/270-degree rotationPXP,DMA2D,GC355,GCNanoLiteVgpu2d_flushRun all queued GPU operationsAllgpu2d_

16、write_displayWrite buffer directly to the display,with optional 90/180/270-degree rotationSMARTDMA,PXPAPI Proposal1 4PUBLICRender operations queueAllows for multiple operations to be queuedFlush must be called before next frame can be renderedDirect to display renderingSame operations queue,but no n

17、eed to write final buffer to displayAPI CONCEPTS API Proposal1 5PUBLICVirtual GPU driver can wrap multiple GPU devicesOperations queued by graphics framework will be flushed before display render occursAPI CONCEPTS API Proposal1 6PUBLICALTERNATIVESLVGLAlready implemented for many hardware blocks,jus

18、t needs to be ported to ZephyrProven abstraction layerLocks Zephyr into LVGL,other frameworks like QT cannot be implementedDoes not expose all drawing capabilities of more complex 2D GPUs like GC355/GCNanoLiteVVGLiteWell defined API,already has conformance tests and implementationsManaged by Verisil

19、icon,who controls definitionNo APIs for engines like DMA2D or PXPAPI Proposal1 7PUBLICFUTURE WORKLVGL Module UpdatesRFC for Graphics APIAPI ImplementationsFuture Work1 8PUBLIC1 9PUBLIC1 9PUBLICNXP,THE NXP LOGO AND NXP SECURE CONNECTIONS FOR A SMARTER WORLD ARE TRADEMARKS OF NXP B.V.ALL OTHER PRODUCT

20、 OR SERVICE NAMES ARE THE PROPERTY OF THEIR RESPECTIVE OWNERS.2023 NXP B.V.Backup Slides2 0PUBLICPATH APISOpcodePurposeMOVEMove cursor to coordinateLINEDraw line from cursor to coordinate,update cursorQUADDraw quadratic curve from cursor to coordinate with one control point,update cursor.CUBICDraw c

21、ubic curve from cursor to coordinate with two control points,update cursor.ARC_CLOCKWISEDraw clockwise arc from cursor to coordinate,with a given sweep angle and horizontal/vertical radiusARC_COUNTER_CLOCKWISEDraw counter clockwise arc from cursor to coordinate,with a given sweep angle and horizonta

22、l/vertical radiusBackup Slides2 1PUBLICAPI STRUCTURESgpu2d_bufbuffer dataheight,width,stridepixel formatgpu2d_src_datacolor or gradient definitiongpu2d_matrix3x2 transformation matrixgpu2d_arccenter point,inner/outer radius,start/end anglegpu2d_rectrectangle dimensionsBackup Slides2 2PUBLICDIRECT 2D

23、Microsoft 2D API,utilizes GPU acceleration where possibleVector GraphicsRectangle drawing,as well as rounded rectanglesDrawing ellipsesText drawingArbitrary paths using draw commands like VGLiteRotation and scaling via transformation matrixRaster GraphicsMajority of porter-duff blend modesCan scale

24、a bitmap to fit within a rectangle then copy to output bufferBackup Slides2 3PUBLICG2DDesigned for engines like the PXPRaster GraphicsCopy image into output buffer,with set location90/180/270-degree rotationHorizontal and vertical flipSubset of porter-duff blending modesUpscale/downscaleBackup Slide

25、s2 4PUBLICLINUX DRMAbstracts command queue for all GPUsUserspace is responsible for submitting requests to the graphics execution managerMesa uses graphics execution manager requests to implement APIs like OpenGL and VulkanDRM uses opcodes to send commands to GPU,vendors can add custom opcodesWould make applications like GPGPU easier with Zephyr,but we would likely use dedicated hardware for those functions on MCUsPotentially more abstraction than we want on smaller MCUsBackup Slides