| CARVIEW |
??4??@??????ċ????w?K??̌????Ă݂悤?i?????^?]???{?b?g?J?[DeepRacer?ҁj?FAWS DeepRacer?????i1/3 ?y?[?W?j
?????w?K?????߂Ă̐l?ɍœK?ȁuAWS DeepRacer?̃R???\?[???ƃV?~?????[?V???????v???g???āA?f?B?[?v???[?j???O??̌????Ă݂悤?B?R???\?[????ŋ????w?K?̊e?n?C?p?[?p?????[?^?[??ݒ肵?ă??f???Ɋw?K?????A????ɕ]?????A?o?[?`???????[?X?Ƀf?v???C????܂ł̎菇?????????B
???̋L???͉??????ł??B????o?^?i?????j????ƑS?Ă????????????܂??B
?@AWS DeepRacer?́A???????s???s?????{?b?g?J?[?ł???i??1???ڂ????????????j?B???̑??s?G???W???̍쐬?ɂ́A?f?B?[?v???[?j???O?́u?????w?K?v?ƌĂ???@???p??????i??2???ڂ????????????j?B?????w?K?ɂ??u?w?K?v??u?]???v?́A?V?~?????[?V???????i???V?~?????[?^?[?j??ōs????悤?ɂȂ??Ă???A?Ԃ̑?????A?܂苭???w?K?̃n?C?p?[?p?????[?^?[?i???w?K?????邽?߂̊e?퐔?l?j?Ȃǂ́AAWS?R???\?[???i???e?푀??????邽?߂?Web?y?[?W?j??Őݒ聕?????ł???悤?ɂȂ??Ă???i??3???ڂ????????????j?B
?@?{?e?ł́A????܂ł̘A?ڂʼn?????????e?̎??H?҂Ƃ??āADeepRacer?̃R???\?[?????g???ă??f?????w?K???]?????ăo?[?`???????[?X?ő??点??܂ł̎菇?????????B?O?????????u?R???\?[???̐ݒ荀?ځv?̒m?????K?v?ƂȂ邽?߁A?K?v?ɉ??????O??̋L?????Q?Ƃ??Ăق????B
???߂Ă?DeepRacer???f???̍쐬?Ɗw?K
?@DeepRacer?R???\?[???????߂Ďg???l?ɂ́A?????̓??{??`???[?g???A???????ɖ𗧂B???????Ȃ???A?n?C?p?[?p?????[?^?[?Ȃǂ̐ݒ?͓ǎҔC???ɂȂ??Ă???̂ŁA?????w?K?̏??S?҂ɂƂ??Ắi?s????ł??܂??w?K???????Ȃ??Ȃǁj?܂????P?[?X???????̂ł͂Ȃ????Ǝv???i?Ƃ??????A?M?Ҏ??g???ŏ??͂??܂????????ɋ?J?????j?B
?@?????Ŗ{?e?ł́A?M?҂??????Ă݂āA??r?I???܂??????A???????V???v?????????ݒ???e???A?ǎ҂ɂ??܂˂Ă??炤???Ƃ?z?肵?āADeepRacer?R???\?[???̊?{?I?ȑ???菇?????????i??2019?N8??19?????_?̏??B????A?ݒ荀?ڂȂǂ??ς??\????????̂ŁA???????????????????j?B
?X?e?b?v1?F AWS DeepRacer?R???\?[?????J??
?@??L?????N????J???A?}1?Ɏ????悤?ɁA?E??́m?R???\?[???ɃT?C???C???n?{?^?????N???b?N???āAAWS?R???\?[???ɃT?C???C??????BAWS?A?J?E???g???????Ă??Ȃ??ꍇ?́A?????h?L???????g?uAWS ?A?J?E???g?쐬?̗???v???Q?l?ɂ???Ȃǂ??āA?܂??̓A?J?E???g???쐬????K?v??????B
?@?R???\?[???̃z?[???y?[?W???J??????A?}2?Ɏ????悤?ɁA?E??ɂ??郊?[?W???????u?č????? (?o?[?W?j?A?k??)?v?iNorth Virginia?j?ɕύX??????ŁA?m?T?[?r?X??????????n???ɁuDeepR?v?iacer?j?ȂǂƓ??͂??A?\?????????ꗗ????mAWS DeepRacer?n??I??????B?????M???_?i2019?N8??19???j?ł́A?o?[?W?j?A?k???ȊO?́A?Ⴆ?u?A?W?A?p?V?t?B?b?N (????)?v?Ȃǂ́AAWS DeepRacer?̃R???\?[?????T?|?[?g???Ă??Ȃ??????̂Œ??ӂ??Ăق????B
?X?e?b?v2?F ???f???̍쐬???J?n????
?@DeepRacer?R???\?[???̃z?[???y?[?W???J??????A?}3?Ɏ????悤?ɁA???ɂ??郁?j???[????mReinforcement learning?n?i?????w?K?j?????N???N???b?N????i???????͒????ɂ???mGet started?n?{?^?????N???b?N????j?B
?@????ɂ??A?mReinforcement learning?n?y?[?W???J???i?}4?j?B???̃y?[?W?ł́A?쐬?ς݂̃??f???̈ꗗ???\???????i???????_?ł̓??f???͖??쐬?Ȃ̂ŕ\??????Ȃ??j?B?܂??A????????V?K?̃??f?????쐬???邱?Ƃ??ł???B
?@????̓??f????V?K?ɍ쐬???????̂ŁA?mCreate model?n?{?^?????N???b?N????B
?X?e?b?v3?F DeepRacer???\?[?X?̍쐬
?@?mCreate model?n?y?[?W???J??????A?g?b?v?ɁmOverview?n????????B?????ł́ADeepRacer?̋????w?K??A?w?K?v???Z?X?A?w?K??̃f?v???C?ɂ??Ă̊T?v?????????Ă???i?{?A?ڂ?ǂ?ł????ǎ҂Ȃ?A???̓??e?͗??????Ă???͂??Ȃ̂ŁA?ǂޕK?v?͂Ȃ????낤?j?B
?@???̉??ɂ???mAccount resources?n???́mCreate resources?n?{?^?????N???b?N???悤?i?}5?j?B????ɂ??A?uIAM???[???v??DeepRacer???\?[?X?X?^?b?N?i??̓I?ɂ́uS3?o?P?b?g?v?uAmazon SageMaker?v?uRoboMaker?V?~?????[?V?????A?v???P?[?V?????v?ȂǁADeepRacer?ɕK?v?Ȋ֘A???\?[?X?j???????쐬?????B
?@?쐬?ɂ?5?????x??????B???ꂼ??쐬??????????ƁA?}6?́mYou have valid IAM roles?n?^?mYou have a valid AWS DeepRacer resources stack?n?̍??ɂ???悤?ȗΐF?̃`?F?b?N?}?[?N???t???B
?}6?@?֘A???\?[?X?̍쐬?????y?Q?l?z?????쐬?????T???v?????f??
?@???Ȃ݂ɁA???̒i?K?Ő}4?́mReinforcement learning?n?y?[?W???J???ƁA?}7?̂悤?ɁA3?قǃT???v???̃??f???iSample-Prevent-zig-zag?^Sample-Stay-on-track?^Sample-Follow-center-line?j??????????Ă???̂??m?F?ł???B?K?v?ɉ????āA???ꂩ??N???[???i???R?s?[?j???ĐV???????f?????쐬???邱?Ƃ??ł???B
?@?Ȃ??A?ȍ~?̎菇?ɐi?ނɂ́A?ēx?A?mCreate model?n?{?^?????N???b?N???āmCreate model?n?y?[?W???J?????????ƁB
?@???悢?悱??????A???f???쐬?̂??߂̊e??ݒ???s???Ă????B
?X?e?b?v4?F ???f?????Ɛ????̐ݒ?
?@?mCreate model?n?y?[?W?́mModel details?n???܂ŃX?N???[?????A
- ?mModel name?n?i???f?????j???F ?K?{?B64?????܂ŁB?A???t?@?x?b?g?̑啶???^???????A?????ƃn?C?t???i-?j???g????
- ?mModel description?n?i?????j???F ?I?v?V?????i???K?{?ł͂Ȃ??j?B255?????܂?
??2?ӏ??ɁA?C?ӂ̒l????͂???i?}8?j?B
?@?{?e?ł́A
- ?mModel name?n???F DeepInsiderEngine1
- ?mModel description?n???F Deep Insider??DeepRacer?G???W????1???ł??B
?Ɠ??͂????Ɖ??肵?āA?ȍ~?̘b??i?߂Ă????B
?X?e?b?v5?F ?V?~?????[?V???????̑I??
?@???ɁA?mEnvironment simulation?n???܂ŃX?N???[?????A
- re:Invent 2018
- Shanghai Sudu Training
- Empire City Training
- Kumo Torakku Training
- London Loop Training
- Bowtie Track
- Oval Track
- AWS Track
?Ȃǂ??烌?[?X????I??????i?}9?j?B
?@?I????Ƃ??ẮA?J?[?u?̑????ȂǑ??s?̓?????l?????āA???̎????̎??͂ɍ??킹?????̂?I??????Ƃ悢?B????́A?ł???\?I?ŁA???S?Ҍ????ȁA
- re:Invent 2018
??I?????悤?B
?@???Ȃ݂ɁA??V?????ʂ̃??[?X???ɍ??킹?Ď??????邱?ƂŁA???[?X?̐??т??グ???@??????B???̂悤?Ȏ?@???̗p????P?[?X?ł́A?V?~?????[?V???????ł??郌?[?X???̑I???͏d?v?ł???B???̂悤?Ȏ?@?̏ꍇ?A???[?X????ς????Ƃ???ɁA?܂Ƃ??ɑ??s?ł??Ȃ??Ȃ?\?????l??????B
?X?e?b?v6?F ?s????Ԃ̐ݒ?
?@?????āA?mAction space?n???܂ŃX?N???[?????čs????Ԃ?ݒ肷??i?}10?j?B
?@?s????ԂƂ??̐ݒ?w?j?ɂ??ẮA?O??A?ڂ????????????B?悭??????Ȃ??ꍇ?́A?O??̉??????ǂ???????A?????Ȃ?ɓK?Ȓl??ݒ肷??Ƃ悢?B????́A?V???v???????芴?̂??鑖???z?肵?āA
- ?mMaximum steering angle?n?i?ő?X?e?A?????O?p?x?j???F 30 ?x?idegrees?j
- ?mSteering angle granularity?n?i?X?e?A?????O?p?x?̕??????j???F 5 ?i?K
- ?mMaximum speed?n?i?ő?X?s?[?h?j???F 1 m/s
- ?mSpeed granularity?n?i?X?s?[?h?̕??????j???F 1 ?i?K
?Ɛݒ肵?悤?B
?@?O??????????????A?s????Ԃ̐ݒ?́A???f???̐V?K?쐬???i?܂?ŏ???1??j?????ŁA?Ċw?K?̍ۂɕύX?ł??Ȃ??i?????̊e??ݒ?́A???f???̊w?K??????ɕύX?^???????āA???f?????Ċw?K????????j?Ƃ????_?ɒ??ӂ??Ăق????B?Ċw?K???ɍs????Ԃ̐ݒ??ς??????ꍇ?ɂ́A?w?K?ς݃??f???̃N???[???i???R?s?[?j?ł͂Ȃ??A?ʂ̃??f????V?K?ɍ쐬????K?v??????B
?X?e?b?v7?F ??V???̎???
?@????ɁA?mReward function?n???܂ŃX?N???[?????ĕ?V????????????i?}11?j?B
?@??V???₻?̎??????@?ɂ??Ă??A?O??A?ڂ????????????B?悭??????Ȃ??ꍇ?́A?ēx?A?O??̉??????ǂ???????A?????Ȃ?̃??W?b?N???l???ēƎ???Python?R?[?h?Ŏ???????Ƃ悢?B????́A???߂Ă̐l?ł??????Ɏ????ł??A?i?T???v???̒??ł́j??r?I?A???x???????A
- ?T???v???R?[?h?F Prevent zig-zag
?𗘗p???悤?B??V????????????菇?̐????͑O??s?????̂ŁA?ڍׂ͊??????邪?A?ȒP?ɂ????ƁmReward function examples?n?{?^?????N???b?N???āA?mPrevent zig-zag?n??I?????A?Ō?ɁmUse code?n?{?^?????N???b?N???邾???ł???i???O??̐}8?`?}9???Q?Ɓj?B
?@???̃T???v???R?[?h?̓??e??e?s?̈Ӗ??ɂ??ẮA???X?g1?ɂ???R?[?h???̃R?????g???Q?l?ɂ??Ăق????B
def reward_function(params):
'''
?W?O?U?O???s???ɘa???邽?߂ɃX?e?A?????O?Ƀy?i???e?B???Ȃ???V???̗?
'''
# ???̓p?????[?^?[?i?Ԃ̊e????j??ǂݍ???
distance_from_center = params['distance_from_center'] # ?Z???^?[???C??????̋???
track_width = params['track_width'] # ?R?[?X?̕?
steering = abs(params['steering_angle']) # ?X?e?A?????O?p?x?̐?Βl
# ?Z???^?[???C??????̋?????\???A3?i?K?̃R?[?X???̃}?[?J?[???v?Z????
marker_1 = 0.1 * track_width # ?R?[?X????10???̋???
marker_2 = 0.25 * track_width # ?R?[?X????25???̋???
marker_3 = 0.5 * track_width # ?R?[?X????50???̋???
# ?Ԃ??Z???^?[???C???ɋ߂???߂??قǁA??????V??^????
if distance_from_center <= marker_1:
reward = 1.0 # 10???????𑖍s???̏ꍇ??1.0?|?C???g?̕?V
elif distance_from_center <= marker_2:
reward = 0.5 # 25???????Ȃ?0.5?|?C???g
elif distance_from_center <= marker_3:
reward = 0.1 # 50???????Ȃ?0.1?|?C???g
else:
reward = 1e-3 # ?????????O?Ȃ?1e-3?i??0.001?j?|?C???g
# ?N???b?V?????????A?R?[?X?A?E?g?ɋ߂???Ԃƌ??Ȃ??Ƃ???????
# ?y?i???e?B???ۂ??A?X?e?A?????O?p?x??臒l
ABS_STEERING_THRESHOLD = 15 # ??Βl??15?x
# ?s????Ԃ̐ݒ?ɉ????ĕύX????B????́A0?^?}15?^?}30?x??3?i?K?Ȃ̂ŁA
#?i?}15?x???傫???j?}30?x?̃X?e?A?????O?ɂ̂݃y?i???e?B???Ȃ?
# ?Ԃ̃X?e?A?????O?p?x??臒l???傫???ꍇ?ɂ́A??V?Ƀy?i???e?B???Ȃ?
if steering > ABS_STEERING_THRESHOLD:
reward *= 0.8 # ?y?i???e?B?Ƃ??āA??V??20???J?b?g?i??80???t?^?j????
# ?v?Z???ꂽ??V?́A???̌Ăяo??????float?l?Ƃ??ĕԋp????
return float(reward)
?X?e?b?v8?F ?n?C?p?[?p?????[?^?[?̐ݒ聕????
?@???ɁA?mHyperparameters?n???܂ŃX?N???[?????ăn?C?p?[?p?????[?^?[??ݒ聕????????i?}12?j?B?mHyperparameters?n???͕????Ă???ꍇ??????B???̏ꍇ?́A????́m???n?{?^?????N???b?N???ĊJ???Ăق????B
?@?n?C?p?[?p?????[?^?[?₻?̐ݒ???ʂɂ??ẮA?O??A?ڂ????????????B?悭??????Ȃ??ꍇ?́A?ēx?A?O??̉??????ǂ???????A?????Ȃ?ɓK?Ȓl??ݒ肷??Ƃ悢?B????́A?????ݒ?̂܂܂ŁA
- ?mGradient descent batch size?n?i?o?b?`?T?C?Y?j???F 64
- ?mNumber of epochs?n?i?G?|?b?N???j???F 10
- ?mLearning rate?n?i?w?K???j???F 0.0003
- ?mEntropy?n?i?G???g???s?[?j???F 0.01
- ?mDiscount factor?n?i???????j???F 0.999
- ?mLoss type?n?i?????^?C?v?j???F Huber
- ?mNumber of experience episodes between each policy-updating iteration?n?i?g???[?j???O???Ƃ̌o???G?s?\?[?h?̐??j???F 20
??ݒ肵?悤?B
?X?e?b?v9?F ??~?????̐ݒ?
?@?Ō?ɁA?mStop conditions?n???܂ŃX?N???[?????āA?w?K???~?????????ݒ肷??i?}13?j?B
?@?ʏ?A?f?B?[?v???[?j???O?Ȃǂ̊w?K?ł́A?u?ǂ??܂Ŋw?K?????邩?v?ȂǁA?w?K???~?????邽?߂́u????̊?v?????B?Ⴆ?u?????̃f?[?^??????w?K?????邩?v??A?u???f???̐??x?i?????m?x?Ȃǁj?????肵?Ă????瑁?߂Ɋw?K???~????v?i??????~?FEarly stopping?j?Ȃǂ̊???B
?@???M???_?i2019?N8??19???j??DeepRacer?R???\?[???ł́A???̊?̈?Ƃ??ām?ő厞?ԁn???ݒ?ł???B?V?~?????[?V???????ł͉???ł????[?X???𑖂??̂ōی????Ȃ??B?????ŁA???Ԃ???Ɋw?K???I???????悤?Ƃ????킯?ł???B
?@?m?ő厞?ԁn???́A5?`480???iminutes?j?i?܂?8???Ԃ܂Łj???ݒ?\?ŁA?f?t?H???g?́u60 mins?i???j?v?ƂȂ??Ă???B????́A?قǂ悭?w?K???i?ނ悤?ɁA
- ?mMaximum time?n?i?ő厞?ԁj???F 120 ??
??ݒ肵?悤?B
?X?e?b?v10?F ?w?K?i?g???[?j???O?j?̊J?n
?@?ȏ?ŁA???f???̊e??ݒ?͊??????B???Ƃ́A?y?[?W?̍Ō?܂ŃX?N???[?????āA?mStart training?n?{?^?????N???b?N????i?}14?j?A???悢??w?K?i?g???[?j???O?j???J?n?????B
?@?????Ĉȉ??ł́A?w?K?`?]???܂ł̎菇?????Ă??????B
Copyright© Digital Advantage Corp. All Rights Reserved.
?A?C?e?B???f?B?A????̂??m?点
??IT eBook
???ڂ̃e?[?}
?ҏW??????̂??m?点
RSS?ɂ???
?A?C?e?B???f?B?AID?ɂ???
???[???}?K?W???o?^
??IT?̃??[???}?K?W???́A ???????A???ׂĖ????ł??B???Ѓ??[???}?K?W???????w?ǂ????????B
ITmedia?̓A?C?e?B???f?B?A??????Ђ̓o?^???W?ł??B
???f?B?A?ꗗ | ????SNS | ?L???ē? | ???₢???킹 | ?v???C?o?V?[?|???V?[ | RSS | ?^?c??? | ?̗p??? | ??????