2019年AI和ffmpeg_gstreamer的應用實踐.pdf

1、AI和FFmpeg/gstreamerAgenda FFmpeg Gstreamer FeatherNet for face anti-spoofingFFMPEGFFmpeg is the most popular open-source multimedia manipulation tools with a library of plugins that can be applied to various parts of the audio and video processing pipelines and have achieved wide adoption across the

2、 world Video encoding,decoding and transcoding are some of the most popular applications of FFmpeg,and Multiplatform is supported such as Linux/Android/Windows.ffmpeg-qsv and ffmpeg-vaapi are providing HW acceleration for intel platforms.repo link:https:/git.ffmpeg.org/ffmpeg.githttps:/git.libav.org

3、/FFmpegDNN in FFmpegGuo YejunDNN in FFmpegGstreamer結構9FeatherNet for Face Antispoofing10Face Anti-spoofing competitionCVPR2019us11Feather for(Face Anti-spoofing)next level details數據源由Intel realsense采集12Feather:Feathernet,MobileLiteNetA/BOur Model:as lite as FeatherMore preciseBlocks used in FeatherN

4、etsBN ReLU63x3 DWConv1x1 Conv1x1 ConvBN ReLU66 x ccBNOutputInputcAddAddBN ReLU61x1 Conv1x1 ConvBN ReLU6BNOutputInput1x1 Conv6 x c2x2 AVG Pool(stride=2)3x3 DWConv(stride=2)BNBlockB:Down-Sampling BlockBlockA:Inverted Residual BlockccccApproachcBN ReLU61x1 Conv1x1 ConvBN ReLU6BNOutputInput6 x c3x3 DWCo

5、nv(stride=2)cBlockC:Down-Sampling BlockWithout AVGPoolingFeatherNetA-BlockA,BlockCFeatherNetB-BloackA,BlockBNetwork ArchitectureApproachStreaming ModuleThe last blocks output is down-sampledby a depth-wise convolution layer and flattened directly into a feature vector.ApproachStreaming Module Approa

6、chRF of center unitRF of corner unitLast 7x 7 Feature Map(one channel)Input Image Units at different position in feature map correspond different receptive field ExperimentsA Newly Collected Dataset:Multi-Modal Face Dataset(MMFD)Intel RealSense SR300 depth camera is utilized 15000+real sample,28000+

7、fake samples,15 subjects RGB,Depth and IR modalities 2 new attack ways-flat face photo with eyes and mouth cut-curved face photo with eyes and mouth cut Variations-distance to camera,face pose,emotion,wearing glass/hat or notExperimentsData Augmentation for MMFDProblem:Although we used the same capt

8、uring device as CAISA-SURFused,the character of the depth image are different.Solution:Depth Imagesfrom CASIA-SURFDepth Images from MMFDProcessed depth images from MMFDExperimentsPerformance improved by introducing MMFDCASIA-SURFMMFD0.00168MMFD0.00677CASIA-SURF0.009710.0010.0030.0050.0070.0090.0110.

9、0130.0150.0170.019ACER0.008030.00294ExperimentsModelSizevs.ACERFishNet150MobileNet-V2ShuffleNet-V2FeatherNetB00.0010.0020.0030.0040.005051015202530Params(Millions)ACERFeatherNetB0.350.00168ShuffleNet-V21.260.00451MobileNet-V22.230.00228FishNet15024.960.00144Params(Millions)ExperimentsAblationExperim

10、entsDepth_Model_1Depth_Model_2Depth_Model_N-1Depth_Model_N10realfakeIR_Modelupdated ensemble score with IR scoresfakerealfakerealhigh scoresUncertain samples are further classified by IR classifierlow scoresEnsemble classifierDepthImagesIR ImagesEnsembleensemblecascadeCompetitionMulti-Modal Fusion S

11、trategyRESULT:0.0013(ACER),0.999(TPRFPR=10e-2),0.998(TPRFPR=10e-3)and 0.9814(TPRFPR=10e-4)DemoImage Capture RGB image(1280*720)Depth image(640*480)Aligned RGB(640*480)FAKEIDAlign to RGB image Realsense:image capture and alignmentOpenVINO:CNN inferenceOpenCV:intermediate data processing the biggest face boxdetected facesthe biggest face box alignedLIVEface detection face anti-spoofing Landmark detection&face alignment Face recognitionworkflowQ&A