專場10.1-Cloud Bigtable 在廣告技術中的使用-郭斌.pdf

1、Cloud Bigtable在廣告技術中的使用郭斌 Google Cloud 架構師What is Cloud BigtableCloud Bigtable is a petabyte-scale,fully managed NoSQL database service for use cases where low latency random data access,scalability and reliability are critical.Being fully managed,it can scale seamlessly.It can also integrate with t

2、he Apache ecosystem and supports the HBase API.Its the home for demanding persistent Key-Value and Columnar workloads on Google CloudSpeedHigh write throughput per node for random access workloadsHigh throughputConsistent 99th percentile low singledigit read and write latencyLow latencyWhats unique

3、about Cloud Bigtable?Replication modeZero touch,integrated and tuned for real-world HA use casesWorkload isolationApp profile to support different workloads against different regions.For example,batch vs serving pathBuilt for petabyte-scale consumer internetEarned its reputation with globally-availa

4、ble Google products like Ads,YouTube,Search,Maps,and Google AnalyticsExceptional throughputand storage capabilitiesDesigned and built for big dataApache HBaseImplemented in C+Multi-cluster deployments with eventual consistencyApp-profiles-multi-cluster routingNo customizations to Bigtable nodes(CPUs

5、,Memory,Disk IO)No customizations to Memtable,Blockcache“Hands-free”managed platformNo access to tablet servers for coprocessors,no server customizationsCloud BigtableImplemented in Java(heap and gc tuning)Peer replication similar to multi-cluster deployments with eventual consistency(manual backfil

6、l)Custom client connection handling for multi-cluster Custom machines and tuning for HBase cluster Requires management of tables,regions,balancing,and optimizationsSupport coprocessors,custom HBase buildsVSFrom developers point of view:Difference between HBase and Cloud BigtableHBaseCBTAccess:JavaHB

7、ase authenticationIAM(Java Bigtable HBase or Bigtable client libraries)Access:other programming languagesApache ThriftIAM(Bigtable client libraries)APIs compatibilityHBase APIsBesides HBase compatible APIs,CBT has native APIsShellHBase ShellCBTTest environmentHBase single nodegcloud beta emulators b

8、igtable startWhats real-time bidding?Source:https:/adtechbook.clearcode.cc/media-buying-methods/User matching(用戶匹配）There are different user-identification methods.From web browsers,cookie syncing is used,and match table can be used between SSPs,DSPs,DMPs,etc.Bigtable is well suited for such workload

9、sUser profile store(用戶畫像)Fast lookups extract user segments for selecting segments;The store is updated frequently based on the users interaction with ads,site they visit,or actions they take.Joining impressions,clicks and conversions(機器學習）Perform a key lookup to join data immediately to use real-ti

10、me data to train models online.AdtechReal-time biddingWeb BrowsersAdTech companies use third-party cookie and cookie syncing to identify users across different websites.Cookie-matching table on Cloud Bigtable is used for this purposeMobile devicesProvide devices advertising ID(e.g.IDFA and AID),but

11、privacy setting allows users to block companies from collecting their advertising iderntifer.User identificationSegment or featureCloud Bigtable is a distributed low-latency user profile.The user store contains the segment a specific user belongs to,which is used to as inputs for bidding decision lo

12、gic.Advertisers use audience segments to target users across the web and mobile platforms.Real-time user profileReal-time predictionFeature Serving APIAd serverContainer EngineData LakeCloud StorageFeature TransformsCloud DataflowFeature CreationEventsApache KafkaData sourcesTraining StoreBigQuerySe

13、rving StoreCloud BigtableFeature StorageFeature AccessFeature lookupScales linearlyLow latencyHigh frequency reads and writesDurable storageRequests,impressions and clicks are combined with key fields in the request,to be used as input features to the machine learning models.Historical feature values with a longer retention time are stored in Cloud BigQuery for training purposes.EventsCloud Pub/Sub