陳研-RustConfChina2023.pdf

《陳研-RustConfChina2023.pdf》由會員分享，可在線閱讀，更多相關《陳研-RustConfChina2023.pdf（35頁珍藏版）》請在三個皮匠報告上搜索。

1、第三屆中國Rust開發者大會Candid:the interface description language of the Internet Computer smart contractsYan ChenDFINITY FoundationRust China Conf 2023 How DFINITY uses procedure macro to extend RustOverviewOverview Macro system is a hidden gem in Rust.We share our experience in safely extending Rust languag

2、es without modifying the compiler.Running example:Candid,a strongly typed serialization library Extended language features:Backward compatible API upgrades with subtyping Type reflection Structural typingMotivationMotivation How to maintain interoperability between microservices?MotivationMotivation

3、 How to maintain interoperability between microservices?UpgradeMotivationMotivation How to maintain interoperability between microservices?APIs are allowed to evolve without breaking existing client Wire format contains type information Decoder knows two types:wire type,and expected typeServing API

4、v2Client only knows API v1Send v1 dataDecode v1 as v2Send v2 resultDecode v2 as v1MotivationMotivation How to maintain interoperability between microservices?APIs are allowed to evolve without breaking existing client Wire format contains type information Decoder knows two types:wire type,and expect

5、ed typeServing API v2Client only knows API v1Send v1 dataDecode v1 as v2Send v2 resultDecode v2 as v1ContravarianceCovarianceSubtyping and safe upgradesSubtyping and safe upgrades Outbound data(provided by service)can upgrade to more specific data Inbound data(required by service)can upgrade to more

6、 general data Orientation is inverted for higher order functionsv3v2v1v1v2v3f :Send(record age=42:nat8)Send(record name=”Admin”;age=42:nat8)ExampleExampletype Profile=record name:text;service:getProfile:(nat)-(Profile);API v1type Profile=record name:text;age:nat8;service:getProfile:(nat)-(Profile);A

7、PI v2getProfileSend(42:nat)Decode as()Decode as(record name:text)Candid:a strongly typed interface description Candid:a strongly typed interface description languagelanguage Primitive types nat,int,nat8-64,int8-64,float32,64,bool,text Composite types vec,opt,record,variant Reference types func,servi

8、ce Supports recursive types Structural typing Subtyping for upgrade safetytype tree=variant empty;node:record left:tree;val:int;right:tree ;service:transform:(func(int)-(int),tree)-(tree);getProfile:(nat32)-(record name:text);Candid exampleCandid bindingsCandid bindingsFrontendWasm BackendCandidRust

9、MotokoCLIJavaScriptUICoq verifiedAgendaAgenda Serialize a message How to implement type reflection in Rust Export interface description How to simulate monomorphization and share states across procedure macro Import interface description How to control generated codeHow to serialize a Candid message

10、?How to serialize a Candid message?Candid message contains both type and value serde can only serialize some of the valuesRust valueserde function42:u8serialize_u8Some(42):Optionserialize_someNone:Optionserialize_noneOk(42):Resultserialize_struct_variantvec!42:Vecserialize_seqvec!:Vecserialize_seqHo

11、w to serialize a Candid message?How to serialize a Candid message?Candid message contains both type and value serde can only serialize some of the values Similarly,deserialization needs to know the expected Candid type Possible solutions Add a new phase in rustc or rust-analyzer Procedure macroDeriv

12、e types with procedure macroDerive types with procedure macro Deriving types from token stream directly No def-use info,cannot handle type aliasing and scoping Types can be generated from other macros Hard to handle recursive typesProcedure macroToken streamToken stream#candid_methodfn getProfile(id

13、:u32)-Profile.Derive types with procedure macroDerive types with procedure macro Deriving types from token stream directly No def-use info,cannot handle type aliasing and scoping Types can be generated from other macros Hard to handle recursive typesProcedure macroToken streamToken stream#candid_met

14、hodfn getProfile(id:u32)-Profile.Lesson learned:dont try to analyze Rust code,but to put the code back in the context and let the compiler do the analysis!Type reflection in Rust?Type reflection in Rust?Common pattern:when you dont know how to implement a feature,define a trait!trait CandidType fn t

15、y()-ASTimpl CandidType for u8 fn ty()-AST AST:Nat8 impl CandidType for Option fn ty()-AST AST:Opt(Box:new(T:ty()impl CandidType for Box fn ty()-AST T:ty()fn typeOf(_:&T)-AST T:ty()Derive Derive CandidTypeCandidType for struct/for struct/enumenum Use procedure macro to derive CandidType traitimpl Can

16、didType for Profile fn ty()AST:Record(vec!field(“name”,String:ty(),)#derive(CandidType)struct Profile name:String,impl CandidType for Tree fn ty()AST:Variant(vec!field(“Empty”,AST:Null),field(“Node”,AST:Record(vec!field(“left”,Tree:ty(),field(“val”,Int:ty(),field(“right”,Tree:ty(),),)What about recu

17、rsive types?What about recursive types?#derive(CandidType)enum Tree Empty,Nodeleft:Box,val:Int,right:Box,What about recursive types?What about recursive types?Need a unique identifier for each Rust type std:any:TypeId:of()-TypeId Memoize T:ty()with memo table HashMapfn ty()-AST let id=TypeId:of:();i

18、f let Some(t)=memo.find(&id)match*t AST:Unknown=AST:Knot(id),_=t.clone(),else memo.insert(id,AST:Unknown);let t=Self:_ty();memo.insert(id,t.clone();t Tree:ty()=AST:Variant(vec!field(“Empty”,AST:Null),field(“Node”,AST:Record(vec!field(“left”,AST:Knot(42),field(“val”,Int:ty(),field(“right”,AST:Knot(42

19、),),)What about recursive types?What about recursive types?Need a unique identifier for each Rust type std:any:TypeId:of()-TypeId where T:static+?Sized static because TypeId:of()=TypeId:of:()pub struct TypeId id:usize,impl TypeId pub fn of()-Self TypeId id:TypeId:of:as usize,Recap:Derive types with

20、procedure macroRecap:Derive types with procedure macro Deriving types from token stream directly No def-use info,cannot handle type aliasing and scoping Types can be generated from other macros Hard to handle recursive types#candid_methodfn getProfile(id:u32)-Profile.Generating code at the same loca

21、tion ensures proper aliasing,scoping,and macro expansion Generated code is often dynamic,e.g.,trait,memoization.AgendaAgenda Serialize a message How to implement type reflection in Rust Export interface description How to simulate monomorphization and share states across procedure macro Import inter

22、face description How to control generated codeExport service signature to CandidExport service signature to Candid Call T:ty()for each type All types are inlined,which is not very human readable Doesnt work for recursive types Need a way to export the type bindings#Derive(CandidType)struct Profile n

23、ame:String,#candid_methodfn getProfile(id:u32)-ProfileRustservice:getProfile:(nat32)-(record name:text)CandidHow to export type bindings?How to export type bindings?Old trick:Define an export_type function in CandidType trait to store type bindings needed for each type Type names in different module

24、s/namespaces collapse in Candidstruct A(u8);mod inner struct A(u32);struct AEquiv(u8);Rusttype A=nat8;type A_1=nat32;/*type AEquiv=nat8;*/CandidHow to export type bindings?How to export type bindings?Old trick:Define an export_type function in CandidType trait to store type bindings needed for each

25、type Type names in different modules/namespaces collapse in Candid Polymorphic types only know the instantiated types at the call siteenum Result Ok(T),Err(E),fn f()-Resultfn g()-ResultRusttype Result=variant Ok:nat32;Err:text;type Result_2=variant Ok:text;Err:nat16;service:f:()-(Result);g:()-(Resul

26、t_2);CandidHow to export type bindings?How to export type bindings?Old trick:Define an export_type function in CandidType trait to store type bindings needed for each type Type names in different modules/namespaces collapse in Candid Polymorphic types only know the instantiated types at the call sit

27、e Recursive type AST:Knot(id)needs to convert to a variable nameAST:Variant(vec!field(“Empty”,AST:Null),field(“Node”,AST:Record(vec!field(“left”,AST:Knot(42),field(“val”,Int:ty(),field(“right”,AST:Knot(42),),)Candid ASTtype Tree=variant Empty;Node:record left:Tree;val:int;right:Tree ;CandidHow to ex

28、port type bindings?How to export type bindings?One Rust type can map to several Candid types(polymorphic types)Inside T:ty(),maintain a global HashMap fn std:any:type_name()-&static str Append index with duplicate names Several Rust types can map to one Candid type(recursive types)Maintain a global

29、HashMap to match structurally equivalent types Equivalence checking of recursive types can be done in O(nlogn)time 1 Rewrite T:ty()based on the HashMap 1 N Gauthier,F Pottier.Numbering Matters:First-Order Canonical Forms for Second-Order Recursive Types,ICFP 2004.Export service signature to CandidEx

30、port service signature to Candid How to collect method names across attributes?Use lazy_static!in the macro(doesnt work for incremental compilation)Custom macro service!f:()-()(need to repeat the defs)Attribute macro on impl(recently supported)#candid_methodfn f()-()#candid_methodfn g()-()#candid_me

31、thodfn h()-()Rustservice:f:()-();g:()-();h:()-();CandidAgendaAgenda Serialize a message How to implement type reflection in Rust Export interface description How to simulate monomorphization and share states across procedure macro Import interface description How to control generated codeImport exte

32、rnal serviceImport external service Convert Candid types back to Rust Rewrite AST by converting inlined records/variants into type definitions Identify recursive types and put it into Box#derive(CandidType)struct FArg x:Int;y:Int;async fn f(arg:FArg)-()call(service_id,f,arg).awaitRustservice:f:(reco

33、rd x:int;y:int)-();=type FArg=record x:int;y:int;service:f:(FArg)-();CandidImport external serviceImport external service Convert Candid types back to Rust One-to-many mapping from Candid to Rust typesfn g(arg:Vec)-()fn g(arg:HashMap)-()fn g(arg:&Box(&str,Arc)-()Rustservice:f:(record x:int;y:int)-()

34、;g:(vec record text;int)-();CandidImport external serviceImport external service Convert Candid types back to Rust One-to-many mapping from Candid to Rust types Use a Config struct to control code generation(in progress)#derive(CandidType,Clone)struct Pos x:Int;y:Int;async fn f(arg:Pos)-()async fn g

35、(arg:HashMap)-()Rustservice:f:(record x:int;y:int)-();g:(vec record text;int)-();config.type_attributes(.,#derive(CandidType,Clone);config.type_name(f.0,Pos);config.use_type(g.0,HashMap);CandidSummarySummary Procedure macro allows us to extend Rust language safely without modifying the compiler DFINITY has a Rust SDK to develop smart contracts on the Internet Computer We did all the complicated work,so that developers dont have to!Rust SDK tutorial:https:/internetcomputer.org/docs/current/tutorials/We have a workshop tonight to demo the Rust SDK!Thank you！