HTTP/2 200
content-type: application/octet-stream
x-guploader-uploadid: ABgVH8_Z4AdUFFjR37FFRl85E1vOCX1tSi4dJ_YLs7UY2tJWfg7MsIVDh-UC8GDV144Y9VFsPdwHCFA
expires: Sat, 19 Jul 2025 09:32:09 GMT
date: Sat, 19 Jul 2025 08:32:09 GMT
cache-control: public, max-age=3600
last-modified: Thu, 15 Aug 2024 03:16:09 GMT
etag: "7c6f243ffff91a9955f219b33ab83eb5"
x-goog-generation: 1723691769396702
x-goog-metageneration: 1
x-goog-stored-content-encoding: identity
x-goog-stored-content-length: 29157
x-goog-hash: crc32c=PMaJ7w==
x-goog-hash: md5=fG8kP//5GplV8hmzOrg+tQ==
x-goog-storage-class: MULTI_REGIONAL
accept-ranges: bytes
content-length: 29157
server: UploadServer
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Tce3stUlHN0L"
},
"source": [
"##### Copyright 2020 The TensorFlow Authors."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"cellView": "form",
"execution": {
"iopub.execute_input": "2024-08-15T03:11:13.091188Z",
"iopub.status.busy": "2024-08-15T03:11:13.090935Z",
"iopub.status.idle": "2024-08-15T03:11:13.095061Z",
"shell.execute_reply": "2024-08-15T03:11:13.094499Z"
},
"id": "tuOe1ymfHZPu"
},
"outputs": [],
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qFdPvlXBOdUN"
},
"source": [
"# Introduction to Variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MfBg1C5NB3X0"
},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AKhB9CMxndDs"
},
"source": [
"A TensorFlow **variable** is the recommended way to represent shared, persistent state your program manipulates. This guide covers how to create, update, and manage instances of `tf.Variable` in TensorFlow.\n",
"\n",
"Variables are created and tracked via the `tf.Variable` class. A `tf.Variable` represents a tensor whose value can be changed by running ops on it. Specific ops allow you to read and modify the values of this tensor. Higher level libraries like `tf.keras` use `tf.Variable` to store model parameters. "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xZoJJ4vdvTrD"
},
"source": [
"## Setup\n",
"\n",
"This notebook discusses variable placement. If you want to see on what device your variables are placed, uncomment this line."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:13.098484Z",
"iopub.status.busy": "2024-08-15T03:11:13.098274Z",
"iopub.status.idle": "2024-08-15T03:11:15.463532Z",
"shell.execute_reply": "2024-08-15T03:11:15.462801Z"
},
"id": "7tUZJk7lDiGo"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-08-15 03:11:13.352524: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
"2024-08-15 03:11:13.373928: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
"2024-08-15 03:11:13.380240: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n"
]
}
],
"source": [
"import tensorflow as tf\n",
"\n",
"# Uncomment to see where your variables get placed (see below)\n",
"# tf.debugging.set_log_device_placement(True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vORGXDarogWm"
},
"source": [
"## Create a variable\n",
"\n",
"To create a variable, provide an initial value. The `tf.Variable` will have the same `dtype` as the initialization value."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:15.468027Z",
"iopub.status.busy": "2024-08-15T03:11:15.467280Z",
"iopub.status.idle": "2024-08-15T03:11:17.672223Z",
"shell.execute_reply": "2024-08-15T03:11:17.671461Z"
},
"id": "dsYXSqleojj7"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
"I0000 00:00:1723691475.985751 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691475.989650 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691475.992820 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691475.996591 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.008057 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.011646 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.014444 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.017845 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.021212 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.024742 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.027614 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691476.031160 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.244355 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.246449 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.248442 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.250525 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.252548 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.254492 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.257197 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.259182 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.261129 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.263068 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.264968 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.266962 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.304803 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.307421 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.309352 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.311366 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.313328 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.315262 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.317178 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.319165 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.321104 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.323540 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.325885 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n",
"I0000 00:00:1723691477.328388 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n"
]
}
],
"source": [
"my_tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]])\n",
"my_variable = tf.Variable(my_tensor)\n",
"\n",
"# Variables can be all kinds of types, just like tensors\n",
"bool_variable = tf.Variable([False, False, False, True])\n",
"complex_variable = tf.Variable([5 + 4j, 6 + 1j])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VQHwJ_Itoujf"
},
"source": [
"A variable looks and acts like a tensor, and, in fact, is a data structure backed by a `tf.Tensor`. Like tensors, they have a `dtype` and a shape, and can be exported to NumPy."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.675690Z",
"iopub.status.busy": "2024-08-15T03:11:17.675422Z",
"iopub.status.idle": "2024-08-15T03:11:17.681473Z",
"shell.execute_reply": "2024-08-15T03:11:17.680831Z"
},
"id": "GhNfPwCYpvlq"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shape: (2, 2)\n",
"DType:
\n",
"As NumPy: [[1. 2.]\n",
" [3. 4.]]\n"
]
}
],
"source": [
"print(\"Shape: \", my_variable.shape)\n",
"print(\"DType: \", my_variable.dtype)\n",
"print(\"As NumPy: \", my_variable.numpy())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eZmSBYViqDoU"
},
"source": [
"Most tensor operations work on variables as expected, although variables cannot be reshaped."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.684703Z",
"iopub.status.busy": "2024-08-15T03:11:17.684243Z",
"iopub.status.idle": "2024-08-15T03:11:17.699697Z",
"shell.execute_reply": "2024-08-15T03:11:17.699085Z"
},
"id": "TrIaExVNp_LK"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"A variable: \n",
"\n",
"Viewed as a tensor: tf.Tensor(\n",
"[[1. 2.]\n",
" [3. 4.]], shape=(2, 2), dtype=float32)\n",
"\n",
"Index of highest value: tf.Tensor([1 1], shape=(2,), dtype=int64)\n",
"\n",
"Copying and reshaping: tf.Tensor([[1. 2. 3. 4.]], shape=(1, 4), dtype=float32)\n"
]
}
],
"source": [
"print(\"A variable:\", my_variable)\n",
"print(\"\\nViewed as a tensor:\", tf.convert_to_tensor(my_variable))\n",
"print(\"\\nIndex of highest value:\", tf.math.argmax(my_variable))\n",
"\n",
"# This creates a new tensor; it does not reshape the variable.\n",
"print(\"\\nCopying and reshaping: \", tf.reshape(my_variable, [1,4]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qbLCcG6Pc29Y"
},
"source": [
"As noted above, variables are backed by tensors. You can reassign the tensor using `tf.Variable.assign`. Calling `assign` does not (usually) allocate a new tensor; instead, the existing tensor's memory is reused."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.702797Z",
"iopub.status.busy": "2024-08-15T03:11:17.702549Z",
"iopub.status.idle": "2024-08-15T03:11:17.709855Z",
"shell.execute_reply": "2024-08-15T03:11:17.709238Z"
},
"id": "yeEpO309QbB2"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ValueError: Cannot assign value to variable ' Variable:0': Shape mismatch.The variable shape (2,), and the assigned value shape (3,) are incompatible.\n"
]
}
],
"source": [
"a = tf.Variable([2.0, 3.0])\n",
"# This will keep the same dtype, float32\n",
"a.assign([1, 2]) \n",
"# Not allowed as it resizes the variable: \n",
"try:\n",
" a.assign([1.0, 2.0, 3.0])\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "okeywjLdQ1tY"
},
"source": [
"If you use a variable like a tensor in operations, you will usually operate on the backing tensor. \n",
"\n",
"Creating new variables from existing variables duplicates the backing tensors. Two variables will not share the same memory."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.712821Z",
"iopub.status.busy": "2024-08-15T03:11:17.712581Z",
"iopub.status.idle": "2024-08-15T03:11:17.724427Z",
"shell.execute_reply": "2024-08-15T03:11:17.723846Z"
},
"id": "2CnfGc6ucbXc"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[5. 6.]\n",
"[2. 3.]\n",
"[7. 9.]\n",
"[0. 0.]\n"
]
}
],
"source": [
"a = tf.Variable([2.0, 3.0])\n",
"# Create b based on the value of a\n",
"b = tf.Variable(a)\n",
"a.assign([5, 6])\n",
"\n",
"# a and b are different\n",
"print(a.numpy())\n",
"print(b.numpy())\n",
"\n",
"# There are other versions of assign\n",
"print(a.assign_add([2,3]).numpy()) # [7. 9.]\n",
"print(a.assign_sub([7,9]).numpy()) # [0. 0.]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZtzepotYUe7B"
},
"source": [
"## Lifecycles, naming, and watching\n",
"\n",
"In Python-based TensorFlow, `tf.Variable` instance have the same lifecycle as other Python objects. When there are no references to a variable it is automatically deallocated.\n",
"\n",
"Variables can also be named which can help you track and debug them. You can give two variables the same name."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.727769Z",
"iopub.status.busy": "2024-08-15T03:11:17.727332Z",
"iopub.status.idle": "2024-08-15T03:11:17.735078Z",
"shell.execute_reply": "2024-08-15T03:11:17.734490Z"
},
"id": "VBFbzKj8RaPf"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[False False]\n",
" [False False]], shape=(2, 2), dtype=bool)\n"
]
}
],
"source": [
"# Create a and b; they will have the same name but will be backed by\n",
"# different tensors.\n",
"a = tf.Variable(my_tensor, name=\"Mark\")\n",
"# A new variable with the same name, but different value\n",
"# Note that the scalar add is broadcast\n",
"b = tf.Variable(my_tensor + 1, name=\"Mark\")\n",
"\n",
"# These are elementwise-unequal, despite having the same name\n",
"print(a == b)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "789QikItVA_E"
},
"source": [
"Variable names are preserved when saving and loading models. By default, variables in models will acquire unique variable names automatically, so you don't need to assign them yourself unless you want to.\n",
"\n",
"Although variables are important for differentiation, some variables will not need to be differentiated. You can turn off gradients for a variable by setting `trainable` to false at creation. An example of a variable that would not need gradients is a training step counter."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.738366Z",
"iopub.status.busy": "2024-08-15T03:11:17.737840Z",
"iopub.status.idle": "2024-08-15T03:11:17.742903Z",
"shell.execute_reply": "2024-08-15T03:11:17.742271Z"
},
"id": "B5Sj1DqhbZvx"
},
"outputs": [],
"source": [
"step_counter = tf.Variable(1, trainable=False)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DD_xfDLDTDNU"
},
"source": [
"## Placing variables and tensors\n",
"\n",
"For better performance, TensorFlow will attempt to place tensors and variables on the fastest device compatible with its `dtype`. This means most variables are placed on a GPU if one is available.\n",
"\n",
"However, you can override this. In this snippet, place a float tensor and a variable on the CPU, even if a GPU is available. By turning on device placement logging (see [Setup](#scrollTo=xZoJJ4vdvTrD)), you can see where the variable is placed. \n",
"\n",
"Note: Although manual placement works, using [distribution strategies](distributed_training.ipynb) can be a more convenient and scalable way to optimize your computation.\n",
"\n",
"If you run this notebook on different backends with and without a GPU you will see different logging. *Note that logging device placement must be turned on at the start of the session.*"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.746342Z",
"iopub.status.busy": "2024-08-15T03:11:17.745819Z",
"iopub.status.idle": "2024-08-15T03:11:17.769166Z",
"shell.execute_reply": "2024-08-15T03:11:17.768577Z"
},
"id": "2SjpD7wVUSBJ"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[22. 28.]\n",
" [49. 64.]], shape=(2, 2), dtype=float32)\n"
]
}
],
"source": [
"with tf.device('CPU:0'):\n",
"\n",
" # Create some tensors\n",
" a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n",
" b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])\n",
" c = tf.matmul(a, b)\n",
"\n",
"print(c)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PXbh-p2BXKcr"
},
"source": [
"It's possible to set the location of a variable or tensor on one device and do the computation on another device. This will introduce delay, as data needs to be copied between the devices.\n",
"\n",
"You might do this, however, if you had multiple GPU workers but only want one copy of the variables."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"execution": {
"iopub.execute_input": "2024-08-15T03:11:17.772535Z",
"iopub.status.busy": "2024-08-15T03:11:17.771906Z",
"iopub.status.idle": "2024-08-15T03:11:17.780573Z",
"shell.execute_reply": "2024-08-15T03:11:17.779997Z"
},
"id": "dgWHN3QSfNiQ"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[ 1. 4. 9.]\n",
" [ 4. 10. 18.]], shape=(2, 3), dtype=float32)\n"
]
}
],
"source": [
"with tf.device('CPU:0'):\n",
" a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n",
" b = tf.Variable([[1.0, 2.0, 3.0]])\n",
"\n",
"with tf.device('GPU:0'):\n",
" # Element-wise multiply\n",
" k = a * b\n",
"\n",
"print(k)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fksvRaqoYfay"
},
"source": [
"Note: Because `tf.config.set_soft_device_placement` is turned on by default, even if you run this code on a device without a GPU, it will still run. The multiplication step will happen on the CPU.\n",
"\n",
"For more on distributed training, refer to the [guide](distributed_training.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SzCkWlF2S4yo"
},
"source": [
"## Next steps\n",
"\n",
"To understand how variables are typically used, see our guide on [automatic differentiation](autodiff.ipynb)."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "variable.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
![](\"https://www.tensorflow.org/images/tf_logo_32px.png\"){kind=logo}
View on TensorFlow.org\n",
" ![](\"https://www.tensorflow.org/images/colab_logo_32px.png\"){kind=logo}
Run in Google Colab\n",
" ![](\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\")
View source on GitHub\n",
" ![](\"https://www.tensorflow.org/images/download_logo_32px.png\"){kind=logo}
Download notebook\n",
"