| CARVIEW |
Python?Ńj???[?????l?b?g???[?N???????Ă݂悤?F?j???[?????l?b?g???[?N????
?j???[?????l?b?g???[?N?̎d?g?݂⋓?????A???w???_????ł͂Ȃ??APython?R?[?h???痝?????悤?B?܂??̓j???[?????l?b?g???[?N?iDeep Neural Network?j?̏??`?d???t???X?N???b?`?Ŏ???????B
???̋L???͉??????ł??B????o?^?i?????j????ƑS?Ă????????????܂??B
?{?A?ځi??b?ҁj?̖ړI
?@?X?N???b?`?i?????҂????????\?[?X?R?[?h???????胉?C?u?????[???g?????肹???ɁA?????Ȃ??[???̏?Ԃ???R?[?h???L?q???邱?Ɓj?Ńf?B?[?v???[?j???O??j???[?????l?b?g???[?N?iDNN?FDeep Neural Network?A?ȉ??ł́u?j???[?????l?b?g?v?ƕ\?L?j?????????Ċw?Ԍn?̏??Ђ⓮??u???A?L???͂???????Ǝv???܂??B?????Ŋw?ۂɁA?u?덷?t?`?d?v?i?o?b?N?v???p?Q?[?V?????j?̂Ƃ???ō??܂??āA?????̓X???[???Ă???l?͏??Ȃ??Ȃ??̂ł͂Ȃ??ł??傤???B?X?̐?????v?Z???̂𗝉????Ă??Ă??A???ƂȂ??S?̑??????߂??ɁA
- ?u?덷?t?`?d?????S?ɗ??????Ă???v
- ?u????ŃR?[?h????????v
- ?u?l?ɐ????ł???v
?Ǝ??M???????Č????Ȃ??l???????̂ł͂Ȃ????Ǝv???܂??B
?@?{?A?ځi??b?ҁj?͂??????????l?Ɍ??????L???ɂȂ?܂??B???̋L???̓j???[?????l?b?g?̎d?g?݂??A???w???_????ł͂Ȃ?Python?R?[?h????w?????Ƃ?_???Ă??܂??B?u??????Z?ȍ~?̐??w?͋?肾???ǁA?R?[?h?Ȃ?ǂ߂邺?I?v?Ƃ??????ɂ̓s?b?^???̋L???ł??B???Ўʌo?ł͂Ȃ??????????ɂ???ŃR?[?h????????悤?ɂȂ?܂??傤?B
?{?A?ځi??b?ҁj?̓???
?@?{?A?ځi??b?ҁj?̓????́A???`?㐔?ilinear algebra?A?s?Z?j???g??Ȃ????Ƃł??B?܂?NumPy???g???܂???B??{?I?Ɋ|???Z?⑫???Z?Ȃǂ̒??w?܂ł̐??w?݂̂ŁA?j???[?????l?b?g?̃??W?b?N???R?[?f?B???O???Ă????܂??i?????Z???x???̐??w?ɑΉ?????R?[?h???????o?Ă??܂????v?Z?͂??Ȃ??̂ŁA???w???w???x???̒m???ő??v?ł??j?B
?@???`?㐔???g???Ɗm???ɃR?[?h???Z???Ȃ?????I?ł????A?V???v???Ȃ̂ʼn??ƂȂ??????ł????C?ɂȂ?܂??B???????d?g?݂ƂȂ郍?W?b?N?𗝉????邽?߂ɂ́A?j???[?????l?b?g?̏??`?d??t?`?d?ł́u???l?̗???v?𒀈?ǂ??Ă????K?v??????Ǝv???܂??B???̍ۂɂ́A???`?㐔?̒??ɉB???ꂽ?????́u?|???Z?Ƒ????Z?̐????v?̒??⎆?̏?œW?J?????ǂ??K?v??????܂??B?Ⴆ??u??Wx?{b?Ƃ??????`?㐔?̎??ɂ́Au???ix1?~w1?{x2?~w2?{?c?{xn?~wn?{b?j?~?m?[?h?????̐??????B??Ă???̂ŁA???̃??W?b?N?𗝉?????ɂ́A?????W?J????K?v??????킯?ł??B???̂??Ƃ??A?덷?t?`?d?𗝉?????ۂ̖W???̈???ɂȂ??Ă???ƕM?҂͊????܂????B
?@?????ł???Ȃ?A?ŏ???????`?㐔?ɉB????Ă??鐔????W?J??????ԂŎ??????Ă??܂??悢?A?Ƃ????̂??{?A?ځi??b?ҁj?̒??z?ł??B?S?Ă̐??????R?[?h?ɏ????o????Ă???̂ŁA?R?[?h??ǂނ????Ń??W?b?N?????X?ɒǂ??Ă?????Ƃ????킯?ł??i?}1?j?B???̒??⎆?̏?Ő?????W?J????K?v?͂???܂???B
?@?Ƃ͂????A?u???`?㐔?̎????X?̎??Ƀo?????Ƒ?ʂ̃R?[?h?ɂȂ??Ă??܂???ˁH?v?Ƃ?????肪????܂??B?m???ɂ??̒ʂ?ł????A???`?㐔?ŏ??????镔???́u?J??Ԃ??̐????v?ɂȂ??Ă???̂ŁA?uPython?̌J??Ԃ??????ł???for???[?v?v?ɒu???????邱?Ƃ??\?ł??B?Ⴆ?ΐ??`?㐔?̎??ɂ?????Wx?Ƃ????????́A?}1?̐Ԙg?̂悤?ɓW?J?ł??A?????for???[?v?ŒZ???܂Ƃ߂??܂??B????ł??璷?ł????A?M?҂̊??z?ł́A????????R?[?h?ʂ??ȂƎv???܂??i???茳?ŃR?????g?s?Ƌ?s???O???čs?????v?Z????ƁA???`?㐔?̃R?[?h??őS???Ŗ?100?s?ł??????Afor???[?v?̃R?[?h??ł͑S???Ŗ?180?s?ł????B?l?ɂ???ẮA???w?ŔY??100?s?̃R?[?h?????A???w?ŔY?܂Ȃ?180?s?̃R?[?h?̕??????????܂???ˁj?B???̓_??????u???`?㐔?̌v?Z???e???l????????A?V???v???ȎZ?p?v?Z?????̂܂ܕ\???????R?[?h??ǂޕ????????v?Ƃ????l?????̋L???ł??B
?@???Ȃ??A?j???[?????l?b?g?Ŏg?????w?ɂ́A???`?㐔?̑??A?Δ????ipartial differential?j??????܂??B??̓I?ɂ͊????????⑹?????̔??????s???K?v??????܂??B???????{?A?ڂł́A???????̔??????s?????߂̓????????̂܂܃R?[?h?Ƃ??ċL?ڂ??邱?ƂŁA?????̌v?Z?͎??グ?܂???B?Ⴆ???????????̃V?O???C?h???̓????̃R?[?h?͂?????̗p?ꎫ?T?ɋL?ڂ???Ă?????̂??g???܂??i?????̕????̓J???j???OOK?Ƃ????Ă????????j?B?ǂ????Ă??????????ƃR?[?h?ɂȂ邩???C?ɂȂ?l?́A?A?ځwAI?E?@?B?w?K?̐??w???? ?x?̕Δ????̉??Ȃǂ??Q?Ƃ??Ă݂Ă????????B
?@???j???[?????l?b?g???w?Ԃ??Ǝ??̂????߂ĂƂ??????́A???O???w?j???[?????l?b?g???[?N?̎d?g?݂̗????~???߂Ă̎????i?O?ҁ^???ҁ^??ҁj?x?ɖڂ?ʂ??Ă????ƁA??b?p??ƊT?O?A?Ӗ??????????????????̂ŁA???{?A?ڂ??????ł???Ǝv???܂??B
?@?{?A?ڂ́A?܂??u??b?ҁv?Ƃ??āA
- ????i?O?ҁj?F ???`?d??????
- ????i???ҁj?F ?t?`?d??????
- ???X??i??ҁj?F ???f???̍œK???????????A???ۂɉ?A????????
?Ƃ???3?{?̋L???????J?\??ł??B???̌?ɁA???`?㐔???g????????A?e?@?\?̂??ׂ????????̑??҂????????Ă??܂??B
?@?O?u???ƈӋC???݂??????Ȃ??Ă??܂??܂??????A???悢??{?҂ɓ???܂??B??b?ҁi????`???X??j?̓??e?ɑΉ?????m?[?g?u?b?N?͉??L?̃????N??Ŏ??s?^????ł??܂??i??Google Colab?̃C???f???g?̓X?y?[?X2?̐ݒ?ł????A?{?A?ڂ̓X?y?[?X4?ŋL?q?????̂ŁA?{?A?ڂ̃R?[?h???R?s?[???y?[?X?g????ꍇ?̓X?y?[?X????4?ɐ?ւ??Ă????????j?B?{?e?ł́A?R?[?h??Z?????邽?߁A?قƂ?ǂ̊???docstring?i?h?L???????g?R?????g?j???ȗ????܂??????A?m?[?g?u?b?N???ɂ͊܂߂Ă??܂??B
?j???[?????l?b?g???[?N?̐}
?@?܂???{?I?ȃj???[?????l?b?g?i???̗?ł́A???͑w?F2?A?B??w?F3?A?o?͑w?F1?j?̐}???m?F???Ă????܂??傤?i?}2?j?B?ȒP?Ƀ|?C???g????????Ă????ƁA?j???[?????l?b?g?ŗ\??????Ƃ??̏????????`?d?iforward propagation?j?ŁA?P???i???w?K?j????Ƃ??̏????i?ڍ?q?j?̒??ŗv?ƂȂ?̂??t?`?d?i?o?b?N?v???p?Q?[?V?????Fbackpropagation?j?ł??ˁB?Ȃ??`?d?i?ł?ρj?Ƃ́A???͂Ȃǂ̐??l???A?l?b?g???[?N???̌??????i?R?l?N?V?????A?????N?A?{?A?ڂł??G?b?W?ƌĂԁj??ʂ??Ď??̑w???????͑O?̑w?̃j???[?????i???j?b?g?A?{?A?ڂł??m?[?h?ƌĂԁj?ɓ`????Ă??????Ƃł??B
?@?ʏ?͐}2?̍??̂悤?ɉ??ɕ`???܂????A?{?A?ڂł͏ォ?珇?Ԃɐ??????????₷???悤?A?i??{?I?Ɂj?}2?̉E?̂悤?ɏc?ɕ`?????Ƃɂ??܂????B???͑w?????ɂ??Ă???̂ŁA???`?d?????????ɐi?ނ??ƂɂȂ?܂??B?{?A?ڂň?ԏd?v?ȃe?[?}???u?t?`?d?v?Ȃ̂ŁA?ォ?牺?Ɍ??????ēǂ߂?}?ɂ??܂????B
?P???i?w?K?j?????S?̂̎???
?@????ł͂????????A?????????Ƀ[??????R?[?h???????Ă????Ƃ????z??ŁA?R?[?h???????n?߂܂??傤?B?t???X?N???b?`?????̃R?c?́A?ŏ????犮???ȃR?[?h???????Ȃ????Ƃł??B???C???ƂȂ鏬???ȓ?????????????āA????????X?ɖc??܂??Ă????܂??B?[?w?u?w?K?v?̃??C???́A?j???[?????l?b?g?́u?P???v?????ł???ˁB???????珑???Ă????܂??i???X?g1?j?B
?@???Ȃ???b?ҁi????`???X??j?̃j???[?????l?b?g?̎????ł́APython?̃N???X?͎g?킸?ɁA?S??Python?̊???for???[?v?ŏ????Ă????܂??B???`?㐔?iNumPy?j???g??Ȃ??̂ŁA?f?[?^??1???????i???\?`???f?[?^??1?s???j???????܂??B?f?[?^???܂Ƃ߂ď???????~?j?o?b?`?w?K?Ȃǂ̊e?w?K???@?ւ̑Ή??́u???f???̍œK???v?ɊW???镔???Ȃ̂ŁA???X??i??ҁj?ł??炽?߂Đ??????܂??B
# ??肠???????ŁA??̊????`???āA?R?[?h?????s?ł???悤?ɂ??Ă???
def forward_prop(cache_mode=False):
" ???`?d???s?????B"
return None, None, None
y_true = [1.0] # ????l
def back_prop(y_true, cached_outs, cached_sums):
" ?t?`?d???s?????B"
return None, None
LEARNING_RATE = 0.1 # ?w?K???ilr?j
def update_params(grads_w, grads_b, lr=0.1):
" ?p?????[?^?[?i?d?݂ƃo?C?A?X?j???X?V??????B"
return None, None
# ---?????܂ł͉??̎????B???????炪?K?v?Ȏ???---
# ?P??????
y_pred, cached_outs, cached_sums = forward_prop(cache_mode=True) # ?i1?j
grads_w, grads_b = back_prop(y_true, cached_outs, cached_sums) # ?i2?j
weights, biases = update_params(grads_w, grads_b, LEARNING_RATE) # ?i3?j
print(f'?\???l?F{y_pred}') # ?\???l?F None
print(f'????l?F{y_true}') # ????l?F [1.0]
?@?j???[?????l?b?g?̌P???ɕK?v?Ȃ??Ƃ́A???X?g1?̒ʂ?A
??3?????ł??i?}3?j?B???ꂼ??̊??̖߂?l???A???̊??̈????ɓn????Ďp????Ă??܂??ˁB?e?߂?l??????̏ڍׂ́A???ꂼ??̊??̎??????ɂ???docstring?i?h?L???????g?R?????g?j?Ȃǂł??炽?߂Đ??????܂??B
?}3?@?P???i?w?K?j?????????????j???[?????l?b?g???[?N?}?@????ł́A?e????1?????????Ă????܂??B???A????????ɓ??삷?邩???????邽?߂ɁA???̃j???[?????l?b?g?̃A?[?L?e?N?`???[???`???ă??f???Ƃ??Đ??????Ă????A?T???v???̌P???f?[?^??????Ă????????Ǝv???܂??B
???f???̒?`?ƁA???̌P???f?[?^
?@?}2??}3?Ŏ??????̂Ɠ????A???͑w?̃m?[?h??2?A?B??w?̃m?[?h??3?A?o?͑w?̃m?[?h??1?̃??f???imodel?ϐ??j???`???܂??傤?B??b?ҁi????`???X??j?̓N???X???g??Ȃ??̂ŁA?^?v???ŕ\?????悤?Ǝv???܂??B?w?\???ilayers?ϐ??j??d?݁iweights?ϐ??j??o?C?A?X?ibiases?ϐ??j??Python?̃??X?g?ŕ\?????܂??B1?????̌P???f?[?^?ix?ϐ??j??1?????̃??X?g?ŕ\?????܂??i???X?g2?j?B
# ?j???[?????l?b?g???[?N??3?w?\??
layers = [
2, # ???͑w?̓??́i?????ʁj?̐?
3, # ?B??w1?̃m?[?h?i?j???[?????j?̐?
1] # ?o?͑w?̃m?[?h?̐?
# ?d?݂ƃo?C?A?X?̏????l
weights = [
[[0.0, 0.0], [0.0, 0.0], [0.0, 0.0]], # ???͑w???B??w1
[[0.0, 0.0, 0.0]] # ?B??w1???o?͑w
]
biases = [
[0.0, 0.0, 0.0], # ?B??w1
[0.0] # ?o?͑w
]
# ???f?????`
model = (layers, weights, biases)
# ???̌P???f?[?^?i1?????j??????
x = [0.05, 0.1] # x_1??x_2??2?̓?????
?@?d?݂̏????l?͈ӊO?ɏd?v?Ȃ̂ł????A?????ł͒P???ɑS??0?Ƃ??܂????B?{???ł???A?u?O?̑w?̃m?[?h???~???̑w?̃m?[?h???v?ŃG?b?W?i?????N?j??????????????Ȃǂ??āA?K?v?Ȑ??Ƒ??????z??\???̏d?݂????????????ׂ??ł????A?R?[?h??Z?????邽?߂Ƀn?[?h?R?[?f?B???O???܂????i?]?T?̂?????́A?????Ŏ????????̏??????????Ŏ??????Ă݂?ƁA???R?[?f?B???O???y???߂邩??????܂???j?B
?X?e?b?v1. ???`?d?̎???
1?̃m?[?h?ɂ????鏇?`?d?̏???
?@?j???[?????l?b?g?̍ŏ??P?ʂ̓m?[?h?ł??B?܂???1?̃m?[?h?ɂ????鏇?`?d?̏??????R?[?f?B???O???܂??傤?i???X?g3?j?B???͑w?͉??????܂???̂ŁA?B??w?Əo?͑w?ɂ?????m?[?h?̋??ʏ??????L?q???܂??Bx?ϐ??́A???X?g2?ŋL?q?????P???f?[?^?ł??B
# ??肠???????ŁA??̊????`???āA?R?[?h?????s?ł???悤?ɂ??Ă???
def summation(x,weights, bias):
" ?d?ݕt?????`?a?̊??B"
return 0.0
def sigmoid(x):
" ?V?O???C?h???B"
return 0.0
def identity(x):
" ?P?????B"
return 0.0
w = [0.0, 0.0] # ?d?݁i???̒l?j
b = 0.0 # ?o?C?A?X?i???̒l?j
next_x = x # ?P???f?[?^???m?[?h?ւ̓??͂Ɏg??
# ---?????܂ł͉??̎????B???????炪?K?v?Ȏ???---
# 1?̃m?[?h?̏????i1?j?F ?d?ݕt?????`?a
node_sum = summation(next_x, w, b)
# 1?̃m?[?h?̏????i2?j?F ????????
is_hidden_layer = True
if is_hidden_layer:
# ?B??w?i?V?O???C?h???j
node_out = sigmoid(node_sum)
else:
# ?o?͑w?i?P?????j
node_out = identity(node_sum)
?@1?̃m?[?h?̏??`?d?????ɕK?v?Ȃ??Ƃ́A???X?g3?̒ʂ?A
- ?i1?j?d?ݕt?????`?a?̊??F summation()???Ƃ??Ď???
- ?i2?j?????????F ?????ł?sigmoid()????identity()???Ƃ??Ď???
?Ƃ???2?̐??w???????ł??i?}4?j?B
?@???ꂼ??̊??̒??g?̎??????????܂??B
?d?ݕt?????`?a
?@?d?ݕt?????`?a?iweighted linear summation?A?ȉ??ł́u???`?a?v?ƕ\?L?j?Ƃ́A????m?[?h?ւ̕????̓??́ix1?Ax2?Ȃǁj?ɁA???ꂼ??̏d?݁iw1?Aw2?Ȃǁj???|???đ??????킹?āA?Ō?Ƀo?C?A?X?ib?j?𑫂????l?ł??i?O?f?̐}4?̍??BPyTorch??Linear?N???X??ATensorFlow?^Keras??Dense?N???X?A???ɂ?Affine?w?Ȃǂɑ??????܂??j?B?????for???[?v?ŋL?q?????̂????X?g4?ł??B
def summation(x, weights, bias):
# ??1?f?[?^???A?܂?x??weights?́u?ꎟ?????X?g?v?Ƃ????O??B
linear_sum = 0.0
for x_i, w_i in zip(x, weights):
linear_sum += x_i * w_i # i?́u?ԍ??v?i???w?͊?{?I??1?X?^?[?g?j
linear_sum += bias
return linear_sum
# ???`?㐔???g???ꍇ?̃R?[?h??F
# linear_sum = np.dot(x, weights) + bias
?@???łɁA????̋t?`?d?i?̒??Ŏg???Δ????j?ŕK?v?ƂȂ???`?a?̕Γ????ipartial derivative function?A?{?A?ڂ̃R?[?h?ł͑S?āuder?v?ƋL?q????j?????X?g5?Ɏ??????Ă????܂??B
def sum_der(x, weights, bias, with_respect_to='w'):
# ??1?f?[?^???A?܂?x??weights?́u?ꎟ?????X?g?v?Ƃ????O??B
if with_respect_to == 'w':
return x # ???`?au???e?d??w_i?ŕΔ????????x_i?ɂȂ?ii?̓m?[?h?ԍ??j
elif with_respect_to == 'b':
return 1.0 # ???`?au???o?C?A?Xb?ŕΔ????????1?ɂȂ?
elif with_respect_to == 'x':
return weights # ???`?au???e????x_i?ŕΔ????????w_i?ɂȂ?
????with_respect_to?ɂ??ẮA?Ⴆ?u?ϐ?x?Ɋւ??Ă???f(x,w,b)?̕Γ????v?i????f(x,w,b)??ϐ?x???Δ??????邱?Ɓj???A?p??Łupartial derivative of f(x,w,b) with respect to x?v?ƕ\?????邽?߁A???̂悤?ɖ????????B
?@???`?a?̎????e?d?݁^?o?C?A?X?^???͂ŕΔ???????ƁA???̂悤?Ȍv?Z???ʂɂȂ?܂??B?J??Ԃ??ɂȂ?܂????A???w?v?Z?̐????͊??????܂??B
?@?????ӓ_?Ƃ??āA?????Ƃ??ēn????????x??d??weights?ɂ́A?ꎟ?????X?g?̌`?ŁA?O?̑w???ɑ??݂???m?[?h?????̒l???????Ă??܂??i?Ⴆ?ΐ}4?̒????Ɏ??????u1?̃m?[?h?v?ł???A?O?̑w?̃m?[?h????2?Ȃ̂ŁA2?̒l???????????X?g?ɂȂ??Ă??܂??j?B???X?g5?̕Δ????ł́A?Ⴆ??x1??x2?Ƃ???2?̕ϐ??Ɋւ??Ă??ꂼ??Ő??`?a?̕Δ??????s?????ƂɂȂ?̂ŁA?o?͂?2?̒l???????????X?g?ɂȂ?܂??B?o?C?A?Xb??1?????Ȃ??̂ŁAfloat?l?ɂȂ??Ă??܂??B
?????????F?V?O???C?h??
?@?B??w?ł́A?ł???b?I?ȃV?O???C?h???iSigmoid function?j???Œ?I?Ɏg?????Ƃɂ??܂??B?V?O???C?h???̐????ƁA?????????????Python???A???̓????͗p?ꎫ?T???Q?l?ɂ??Ă????????B???X?g6?ɃV?O???C?h???A???X?g7?ɂ??̓????̎????R?[?h???f?ڂ??܂??B???͂????l?Əo?͂????l??float?l?ł??B
import math
def sigmoid(x):
return 1.0 / (1.0 + math.exp(-x))
# ???`?㐔?̏ꍇ??math??np?ɕς???i???O??import numpy as np?j
def sigmoid_der(x):
output = sigmoid(x)
return output * (1.0 - output)
?????????F?P????
?@?o?͑w?ł́A??A?????C???[?W???āA???̂܂܂̒l???o?͂??銈???????ł???P?????iIdentity function?j???g?p???܂??B??????̗p?ꎫ?T?????Q?l?ɁB???X?g8?ɍP?????A???X?g9?ɂ??̓????̎????R?[?h???f?ڂ??܂??B
def identity(x):
return x
def identity_der(x):
return 1.0
?@?ȏ?ŁA???X?g3?ŋ?Œ?`???Ă???3?̊??̎????͏I???܂????B?u1?̃m?[?h?ɂ????鏈???v?̎?????????Ŋ????ł??B
???`?d?̏????S?̂̎???
?@?j???[?????l?b?g?ɂ́A?w??????A???̒??ɕ????̃m?[?h?????݂???Ƃ????\???ł??B?]???āA
- ?e?w??1??????????for???[?v??
- ?w?̒??̃m?[?h??1??????????for???[?v??2?i?K?\?????K?v??
- ???̒??Ɂu1?̃m?[?h?ɂ????鏈???v???L?q
- ?w?̒??̃m?[?h??1??????????for???[?v??2?i?K?\?????K?v??
????悢?킯?ł??B
?@???̍l???ɉ????āA???`?d?̏????S?̂??s??forward_prop()???????????Ă݂??̂????X?g10?ł??B?⑫?????̂??߂̃R?????g?????߂Ɋ܂߂Ă???̂ŏ?????????????Ǝv???܂????A?|?C???g?ƂȂ鏇?`?d?̖{???????͒Z???ł??B2??for???[?v?ƁA?u?m?[?h???Ƃ̏d?݂ƃo?C?A?X???擾?v???Ă??镔???A?y???X?g3?̃R?[?h?z?ƋL?ڂ??ꂽ?????i???ɑ????Ŏ?????9?s?j?ɒ??ڂ??Ă????????B?????ȊO?̃R?[?h?́A?v?Z???ʂ??L???b?V???ɕۑ??i???L?^?j???邽?߂̂??܂??܂????????Ȃ̂ŁA?ǂݔ???Ă??\???܂???B
?@?????ӓ_?Ƃ??āA???X?g2?ō쐬????model?i??(layers, weights, biases)?̃^?v???j??x?????????Ƃ??Ď???悤?A????????layers, weights, biases??x??擪?ɒlj????āA???X?g1?ō쐬????forward_prop()???̃V?O?l?`???[?i?????̈????ƁA???̑g?ݍ??킹?j?????ς??܂????B?Ō?̕??Ɂu?\???̎??s??v?Ƃ??ď?????forward_prop()???̌Ăяo???ł́A?^?v???ł???model?̓C?e???u???i???J??Ԃ??\?ȃI?u?W?F?N?g?j?Ȃ̂ŁA*?C?e???u???A???p?b?N???Z?q???g????*model?Ə??????ƂŁA?擪?ɂ???3?̉??????ɑ??ă^?v?????Ƃ܂Ƃ߂ăZ?b?g???Ă??܂??i?Q?l?F?uPython 3.5?̐V?@?\?v?j?B
def forward_prop(layers, weights, biases, x, cache_mode=False):
"""
???`?d???s?????B
- ?????F
(layers, weights, biases)?F ???f?????w?肷??B
x?F ???̓f?[?^???w?肷??B
cache_mode?F ?\??????False?A?P??????True?ɂ???B????ɂ??߂?l???ς??B
- ?߂?l?F
cache_mode??False???͗\???l?݂̂?Ԃ??BTrue???́A?\???l?????łȂ??A
?L???b?V???ɋL?^?ς݂̐??`?a?i???j?l?ƁA?????????̏o?͒l???Ԃ??B
"""
cached_sums = [] # ?L?^?????S?m?[?h?̐??`?a?i???j?̒l
cached_outs = [] # ?L?^?????S?m?[?h?̊????????̏o?͒l
# ?܂??́A???͑w?????`?d????
cached_outs.append(x) # ?????????????ɏo?͒l???L?^
next_x = x # ???݂̑w?̏o?́ix?j?????̑w?ւ̓??́inext_x?j
# ???ɁA?B??w??o?͑w?????`?d????
SKIP_INPUT_LAYER = 1
for layer_i, layer in enumerate(layers): # ?e?w??????
if layer_i == 0:
continue # ???͑w?͏?ŏ????ς?
# ?e?w?̃m?[?h???Ƃɏ??????s??
sums = []
outs = []
for node_i in range(layer): # ?w?̒??̊e?m?[?h??????
# ?m?[?h???Ƃ̏d?݂ƃo?C?A?X???擾
w = weights[layer_i - SKIP_INPUT_LAYER][node_i]
b = biases[layer_i - SKIP_INPUT_LAYER][node_i]
# ?y???X?g3?̃R?[?h?z???????火
# 1?̃m?[?h?̏????i1?j?F ?d?ݕt?????`?a
node_sum = summation(next_x, w, b)
# 1?̃m?[?h?̏????i2?j?F ????????
if layer_i < len(layers)-1: # -1?͏o?͑w?ȊO?̈Ӗ?
# ?B??w?i?V?O???C?h???j
node_out = sigmoid(node_sum)
else:
# ?o?͑w?i?P?????j
node_out = identity(node_sum)
# ?y???X?g3?̃R?[?h?z?????܂Ł?
# ?e?m?[?h?̐??`?a?Ɓi?????????́j?o?͂????X?g?ɂ܂Ƃ߂Ă???
sums.append(node_sum)
outs.append(node_out)
# ?e?w???̑S?m?[?h?̐??`?a?Əo?͂??L?^
cached_sums.append(sums)
cached_outs.append(outs)
next_x = outs # ???݂̑w?̏o?́iouts?j?????̑w?ւ̓??́inext_x?j
if cache_mode:
return (cached_outs[-1], cached_outs, cached_sums)
return cached_outs[-1]
# ?P???????i1?j???`?d?̎??s??
y_pred, cached_outs, cached_sums = forward_prop(*model, x, cache_mode=True)
# ????قǍ쐬???????f???ƌP???f?[?^???????Ŏ??悤???ς???
print(f'cached_outs={cached_outs}')
print(f'cached_sums={cached_sums}')
# ?o?͗?F
# cached_outs=[[0.05, 0.1], [0.5, 0.5, 0.5], [0.0]] # ???͑w?^?B??w1?^?o?͑w
# cached_sums=[[0.0, 0.0, 0.0], [0.0]] # ?B??w1?^?o?͑w?i?????͑w?͂Ȃ??j
?@?R?[?h???ɂ??R?????g?????Ă??܂????A?C??t???Ăق????|?C???g???ȉ??ł??G??Ă????܂??B
?@?܂??A???͑w?͏o?݂͂̂ŁA???`?a?⊈????????????܂???B????āA?o?͂̃L???b?V???icached_outs?j?ɂ????l??lj????Ă??܂??B???̌??ʁA???`?a?̃L???b?V???icached_sums?j?͓??͑w??1?s???????Ȃ????X?g???e?ɂȂ??Ă??܂??̂ŁA???p????ۂɒ??ӂ??Ă????????B
?@???ɁA?e?m?[?h?̏o?́ix??outs?j?́A???̑w?ɂ???m?[?h?ւ̓??́inext_x?j?ɕω????邱?Ƃ??ӎ?????̂???ł??i?R?[?h?̃??W?b?N??ǂ???ŕM?Ҏ??g?????????₷???Ɗ??????????ł??j?B???X?g10?ł?next_x = ???????Ƃ????R?[?h?ŁA???̕ω????I?ɂ??܂????B???????w????w?փf?[?^???`?d???Ă????Ă???Ƃ???ɑ??????܂??ˁB
?@???Ȃ݂??m?[?g?u?b?N?̕??ł́A?R?[?h????print()?????d???ނ??ƂŁi???S?ăR?????g?A?E?g???Ă??܂??j?A?r???̌v?Z???e?????ԂɃe?L?X?g?o?͂????悤?ɂ??Ă݂܂????B???X?g10?ł́A?ȉ??̂悤?ɏo?͂???܂??B
????1?w?i???͑w?j-?S?āi2?j?̓????ʁF
?@?????̓f?[?^?F ???????Ȃ???out([0.05, 0.1])
????2?w-??1?m?[?h?F
?@???d?ݕt?????`?a?F x_i(0.05)?~w_i(0.0)?{x_i(0.1)?~w_i(0.0)?{b(0.0)??sum(0.0)
?@???????????i?B??w?̓V?O???C?h???j?F sigmoid(0.0)??out(0.5)
????2?w-??2?m?[?h?F
?@???d?ݕt?????`?a?F x_i(0.05)?~w_i(0.0)?{x_i(0.1)?~w_i(0.0)?{b(0.0)??sum(0.0)
?@???????????i?B??w?̓V?O???C?h???j?F sigmoid(0.0)??out(0.5)
????2?w-??3?m?[?h?F
?@???d?ݕt?????`?a?F x_i(0.05)?~w_i(0.0)?{x_i(0.1)?~w_i(0.0)?{b(0.0)??sum(0.0)
?@???????????i?B??w?̓V?O???C?h???j?F sigmoid(0.0)??out(0.5)
????3?w-??1?m?[?h?F
?@???d?ݕt?????`?a?F x_i(0.5)?~w_i(0.0)?{x_i(0.5)?~w_i(0.0)?{
?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@?@x_i(0.5)?~w_i(0.0)?{b(0.0)??sum(0.0)
?@???????????i?o?͑w?͍P?????j?F identity(0.0)??out(0.0)
cached_outs=[[0.05, 0.1], [0.5, 0.5, 0.5], [0.0]]
cached_sums=[[0.0, 0.0, 0.0], [0.0]]
?@???l??0.0????Ȃ̂ŎQ?l?ɂȂ?܂???˥??????B??q?̃??X?g11???Q?l?ɏd?݂?o?C?A?X?Ȃǂ?ς??Ă݂āA?{???Ɍv?Z?ʂ?ɂȂ邩?̃`?F?b?N?Ȃǂ????Ă݂Ă??悢?ł??傤?i???m?[?g?u?b?N?̕??ɂ͕ʂ̌v?Z?p?^?[???̃R?[?h??????Ă????܂????j?B
???`?d?ɂ??\???̎??s??
?@???Ȃ݂ɁA?\?????͓r???̌v?Z???ʂ͕s?v?Ȃ̂ŁA????cache_mode???w?肷??K?v??????܂???i???X?g11?j?B
# ?قȂ?DNN?A?[?L?e?N?`???[???`???Ă݂?
layers2 = [
2, # ???͑w?̓??́i?????ʁj?̐?
3, # ?B??w1?̃m?[?h?i?j???[?????j?̐?
2, # ?B??w2?̃m?[?h?i?j???[?????j?̐?
1] # ?o?͑w?̃m?[?h?̐?
# ?d?݂ƃo?C?A?X?̏????l
weights2 = [
[[-0.2, 0.4], [-0.4, -0.5], [-0.4, -0.5]], # ???͑w???B??w1
[[-0.2, 0.4, 0.9], [-0.4, -0.5, -0.2]], # ?B??w1???B??w2
[[-0.5, 1.0]] # ?B??w2???o?͑w
]
biases2 = [
[0.1, -0.1, 0.1], # ?B??w1
[0.2, -0.2], # ?B??w2
[0.3] # ?o?͑w
]
# ???f?????`
model2 = (layers2, weights2, biases2)
# ???̌P???f?[?^?i1?????j??????
x2 = [2.3, 1.5] # x_1??x_2??2?̓?????
# ?\???????i1?j???`?d?̎??s??
y_pred = forward_prop(*model2, x2)
print(y_pred) # ?\???l
# ?o?͗?F
# [0.3828840428423274]
?@???ɃV???v???Ō??n?I?Ȏ????ł????A???̂悤?ɔC?ӂ̑w???ƃm?[?h???̑S??????DNN?iDeep Neural Network?j?̃A?[?L?e?N?`???[???`???āADNN???f???ɂ??\?????s???܂??B
?@??_?Ƃ??ẮA?O?q?̒ʂ?A?d?݂??????G?b?W?i?????N?j?????????????Ă??Ȃ????߁A?蓮?ŋL?q????K?v??????A???ꂪ?ƂĂ??ʓ|?Ȃ??Ƃł??B?G?b?W??w???ƁA?m?[?h???Ƃɐ????グ??K?v??????܂????A?M?҂̏ꍇ?́u?j???[?????l?b?g???[?N Playground - Deep Insider?v?Ő??𐔂??Ă????̂悤?ɒ?`???܂????i???Q?l?܂Łj?B
????̃X?e?b?v?̏????F???ւ̉??????̒lj?
?@?ȏ?ō???̓??e?͏I???Ȃ̂ł????A????̓??e???y?????邽?߁A??????????R?[?h???????Ă????܂??B
?@??قǂ̃??X?g10??forward_prop()????layers, weights, biases?Ƃ???????????lj????āA???????Ƃ???*model?I?u?W?F?N?g???Z?b?g?ł???悤?ɂ??܂????B
?@???l?ɁA???X?g1?ō쐬????back_prop()????update_params()???ɂ?????????????lj????āA???̃V?O?l?`???[?????ς??Ă????܂??i???X?g12?j?B
def back_prop(layers, weights, biases, y_true, cached_outs, cached_sums):
" ?t?`?d???s?????B"
return None, None
def update_params(layers, weights, biases, grads_w, grads_b, lr=0.1):
" ?p?????[?^?[?i?d?݂ƃo?C?A?X?j???X?V??????B"
return None, None
?@????͋t?`?d?ɓ??邽?߂̏????ҁA?E?H?[???A?b?v??ƌĂׂ???̂ł????B?u?????܂łȂ番?????Ă???v?Ƃ????l?????Ȃ??Ȃ??Ǝv???܂??B
?@?܂?????́u?Δ????v?Ƃ????p?ꂪ?o?Ă??āAsum_der()?^sigmoid_der()?^identity_der()?Ƃ???3?̕Γ??????`???܂????B?????́A????̋t?`?d?̎????̒??ŌĂяo???܂??B
?@??????Δ????ɂ??ĊȒP?ɐ??????Ă????ƁA?????Ƃ́u??f(x)???`???Ȑ??ɂ?????A?ϐ?x?n?_?ł̌X???v?????߂邱?Ƃł??B?Δ????Ƃ́A????f(x,w,b)?Ƒ??ϐ??ł???Ƃ??́A?ϐ?x?^?ϐ?w?^?ϐ?b?Ɋւ???????ł??B?Δ????̌v?Z???ʂł???Δ????W?????A?j???[?????l?b?g???[?N?̏d?݂?o?C?A?X?́u???z?igradient?j?v?ƂȂ?܂??B???̌??z???g???ďd?݂?o?C?A?X???X?V???邱?ƂŁA???f?????œK?????Ă????܂??B
?@?????Ɠ????͍??Z???w?A?Δ????ƕΓ????͑?w???w?̗̈?ł????A??????????w?v?Z???̂͂???قǓ??????܂???B?u?????ƒm?肽???v?u?????Ōv?Z???āAsum_der()?^sigmoid_der()?^identity_der()?????????????Ɏ??????????v?Ƃ????l?́A?܂??͔????Ȃ?u?A?ځwAI?E?@?B?w?K?̐??w???? ?x?̔????̉??v????A?Δ????Ȃ?u???A?ڂ̕Δ????̉??v????w?юn?߂邱?Ƃ??????߂??܂??B
?@????????́A???悢??{??̋t?`?d????????܂??B???y???݂ɁB
Copyright© Digital Advantage Corp. All Rights Reserved.
?A?C?e?B???f?B?A????̂??m?点
??IT eBook
???ڂ̃e?[?}
?ҏW??????̂??m?点
RSS?ɂ???
?A?C?e?B???f?B?AID?ɂ???
???[???}?K?W???o?^
??IT?̃??[???}?K?W???́A ???????A???ׂĖ????ł??B???Ѓ??[???}?K?W???????w?ǂ????????B
ITmedia?̓A?C?e?B???f?B?A??????Ђ̓o?^???W?ł??B
???f?B?A?ꗗ | ????SNS | ?L???ē? | ???₢???킹 | ?v???C?o?V?[?|???V?[ | RSS | ?^?c??? | ?̗p??? | ??????