CS224W - Colab 1¶
In this Colab, we will write a full pipeline for learning node embeddings. We will go through the following 3 steps.
To start, we will load a classic graph in network science, the Karate Club Network. We will explore multiple graph statistics for that graph.
We will then work together to transform the graph structure into a PyTorch tensor, so that we can perform machine learning over the graph.
Finally, we will finish the first learning algorithm on graphs: a node embedding model. For simplicity, our model here is simpler than DeepWalk / node2vec algorithms taught in the lecture. But it’s still rewarding and challenging, as we will write it from scratch via PyTorch.
Now let’s get started!
Note: Make sure to sequentially run all the cells, so that the intermediate variables / packages will carry over to the next cell
1) Graph Basics¶
To start, we will load a classic graph in network science, the Karate Club Network. We will explore multiple graph statistics for that graph.
Setup¶
We will heavily use NetworkX in this Colab.
[1]:
import networkx as nx
Zachary’s karate club network¶
The Karate Club Network is a graph describes a social network of 34 members of a karate club and documents links between members who interacted outside the club.
[2]:
G = nx.karate_club_graph()
# G is an undirected graph
type(G)
[2]:
networkx.classes.graph.Graph
[3]:
# Visualize the graph
nx.draw(G, with_labels = True)
Question 1: What is the average degree of the karate club network? (5 Points)¶
[4]:
def average_degree(num_edges, num_nodes):
# TODO: Implement this function that takes number of edges
# and number of nodes, and returns the average node degree of
# the graph. Round the result to nearest integer (for example
# 3.3 will be rounded to 3 and 3.7 will be rounded to 4)
avg_degree = 0
############# Your code here ############
avg_degree = round(2 * num_edges / num_nodes)
#########################################
return avg_degree
num_edges = G.number_of_edges()
num_nodes = G.number_of_nodes()
avg_degree = average_degree(num_edges, num_nodes)
print("Average degree of karate club network is {}".format(avg_degree))
Average degree of karate club network is 5
Question 2: What is the average clustering coefficient of the karate club network? (5 Points)¶
[5]:
def average_clustering_coefficient(G):
# TODO: Implement this function that takes a nx.Graph
# and returns the average clustering coefficient. Round
# the result to 2 decimal places (for example 3.333 will
# be rounded to 3.33 and 3.7571 will be rounded to 3.76)
avg_cluster_coef = 0
############# Your code here ############
## Note:
## 1: Please use the appropriate NetworkX clustering function
avg_cluster_coef = round(nx.average_clustering(G), 2)
#########################################
return avg_cluster_coef
avg_cluster_coef = average_clustering_coefficient(G)
print("Average clustering coefficient of karate club network is {}".format(avg_cluster_coef))
Average clustering coefficient of karate club network is 0.57
Question 3: What is the PageRank value for node 0 (node with id 0) after one PageRank iteration? (5 Points)¶
Please complete the code block by implementing the PageRank equation: \(r_j = \sum_{i \rightarrow j} \beta \frac{r_i}{d_i} + (1 - \beta) \frac{1}{N}\)
[6]:
def one_iter_pagerank(G, beta, r0, node_id):
# TODO: Implement this function that takes a nx.Graph, beta, r0 and node id.
# The return value r1 is one interation PageRank value for the input node.
# Please round r1 to 2 decimal places.
r1 = 0
############# Your code here ############
## Note:
## 1: You should not use nx.pagerank
for neighbor in G.neighbors(node_id):
r1 += beta * r0 / G.degree[neighbor]
r1 += (1 - beta) / G.number_of_nodes()
# round r1 to 2 decimal places.
r1 = round(r1, 2)
#########################################
return r1
beta = 0.8
r0 = 1 / G.number_of_nodes()
node = 0
r1 = one_iter_pagerank(G, beta, r0, node)
print("The PageRank value for node 0 after one iteration is {}".format(r1))
The PageRank value for node 0 after one iteration is 0.13
Question 4: What is the (raw) closeness centrality for the karate club network node 5? (5 Points)¶
The equation for closeness centrality is \(c(v) = \frac{1}{\sum_{u \neq v}\text{shortest path length between } u \text{ and } v}\)
[9]:
def closeness_centrality(G, node=5):
# TODO: Implement the function that calculates closeness centrality
# for a node in karate club network. G is the input karate club
# network and node is the node id in the graph. Please round the
# closeness centrality result to 2 decimal places.
closeness = 0
############# Your code here ############
## Note:
## 1: You can use networkx closeness centrality function.
## 2: Notice that networkx closeness centrality returns the normalized
## closeness directly, which is different from the raw (unnormalized)
## one that we learned in the lecture.
# normalized closeness centrality(u) = (number of nodes - 1) / sum(distance from u to all other nodes)
# raw closeness centrality(u) = 1 / sum(distance from u to all other nodes)
# normalized closeness centrality
closeness = nx.closeness_centrality(G, u=node)
# unnormalized
closeness = closeness / (len(nx.node_connected_component(G, node)) - 1)
# result to 2 decimal places
closeness = round(closeness, 2)
#########################################
return closeness
node = 5
closeness = closeness_centrality(G, node=node)
print("The node 5 has closeness centrality {}".format(closeness))
The node 5 has closeness centrality 0.01
2) Graph to Tensor¶
We will then work together to transform the graph \(G\) into a PyTorch tensor, so that we can perform machine learning over the graph.
Setup¶
Check if PyTorch is properly installed
[10]:
import torch
print(torch.__version__)
1.11.0
PyTorch tensor basics¶
We can generate PyTorch tensor with all zeros, ones or random values.
[11]:
# Generate 3 x 4 tensor with all ones
ones = torch.ones(3, 4)
print(ones)
# Generate 3 x 4 tensor with all zeros
zeros = torch.zeros(3, 4)
print(zeros)
# Generate 3 x 4 tensor with random values on the interval [0, 1)
random_tensor = torch.rand(3, 4)
print(random_tensor)
# Get the shape of the tensor
print(ones.shape)
tensor([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]])
tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])
tensor([[0.5299, 0.1829, 0.5090, 0.1943],
[0.3763, 0.1543, 0.8716, 0.9986],
[0.7011, 0.6685, 0.0468, 0.0709]])
torch.Size([3, 4])
PyTorch tensor contains elements for a single data type, the dtype.
[12]:
# Create a 3 x 4 tensor with all 32-bit floating point zeros
zeros = torch.zeros(3, 4, dtype=torch.float32)
print(zeros.dtype)
# Change the tensor dtype to 64-bit integer
zeros = zeros.type(torch.long)
print(zeros.dtype)
torch.float32
torch.int64
Question 5: Get the edge list of the karate club network and transform it into torch.LongTensor. What is the torch.sum value of pos_edge_index tensor? (10 Points)¶
[14]:
def graph_to_edge_list(G):
# TODO: Implement the function that returns the edge list of
# an nx.Graph. The returned edge_list should be a list of tuples
# where each tuple is a tuple representing an edge connected
# by two nodes.
edge_list = []
############# Your code here ############
# for edge in G.edges():
# edge_list.append(edge)
edge_list = list(G.edges())
#########################################
return edge_list
def edge_list_to_tensor(edge_list):
# TODO: Implement the function that transforms the edge_list to
# tensor. The input edge_list is a list of tuples and the resulting
# tensor should have the shape [2 x len(edge_list)].
edge_index = torch.tensor([])
############# Your code here ############
edge_index = torch.tensor(edge_list).T
#########################################
return edge_index
pos_edge_list = graph_to_edge_list(G)
pos_edge_index = edge_list_to_tensor(pos_edge_list)
print("The pos_edge_index tensor has shape {}".format(pos_edge_index.shape))
print("The pos_edge_index tensor has sum value {}".format(torch.sum(pos_edge_index)))
The pos_edge_index tensor has shape torch.Size([2, 78])
The pos_edge_index tensor has sum value 2535
Question 6: Please implement following function that samples negative edges. Then answer which edges (edge_1 to edge_5) can be potential negative edges in the karate club network? (10 Points)¶
[15]:
import random
def sample_negative_edges(G, num_neg_samples):
# TODO: Implement the function that returns a list of negative edges.
# The number of sampled negative edges is num_neg_samples. You do not
# need to consider the corner case when the number of possible negative edges
# is less than num_neg_samples. It should be ok as long as your implementation
# works on the karate club network. In this implementation, self loops should
# not be considered as either a positive or negative edge. Also, notice that
# the karate club network is an undirected graph, if (0, 1) is a positive
# edge, do you think (1, 0) can be a negative one?
neg_edge_list = []
############# Your code here ############
# all possible edges - positive edges
pos_edge_list = list(G.edges())
# consider all possible and eliminate positive edges
for node1 in G.nodes():
for node2 in G.nodes():
if node1 >= node2:
continue
if (node1, node2) in pos_edge_list:
continue
neg_edge_list.append((node1, node2))
# return number of samples in the negative list
neg_edge_list = random.sample(neg_edge_list, num_neg_samples)
#########################################
return neg_edge_list
# Sample 78 negative edges
neg_edge_list = sample_negative_edges(G, len(pos_edge_list))
# Transform the negative edge list to tensor
neg_edge_index = edge_list_to_tensor(neg_edge_list)
print("The neg_edge_index tensor has shape {}".format(neg_edge_index.shape))
# Which of following edges can be negative ones?
edge_1 = (7, 1)
edge_2 = (1, 33)
edge_3 = (33, 22)
edge_4 = (0, 4)
edge_5 = (4, 2)
############# Your code here ############
## Note:
## 1: For each of the 5 edges, print whether it can be negative edge
for edge in [edge_1, edge_2, edge_3, edge_4, edge_5]:
print(sorted(edge) not in G.edges)
#########################################
The neg_edge_index tensor has shape torch.Size([2, 78])
False
True
False
False
True
3) Node Emebedding Learning¶
Finally, we will finish the first learning algorithm on graphs: a node embedding model.
Setup¶
[16]:
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
print(torch.__version__)
1.11.0
To write our own node embedding learning methods, we’ll heavily use the `nn.Embedding <https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html>`__ module in PyTorch. Let’s see how to use nn.Embedding:
[17]:
# Initialize an embedding layer
# Suppose we want to have embedding for 4 items (e.g., nodes)
# Each item is represented with 8 dimensional vector
emb_sample = nn.Embedding(num_embeddings=4, embedding_dim=8)
print('Sample embedding layer: {}'.format(emb_sample))
Sample embedding layer: Embedding(4, 8)
We can select items from the embedding matrix, by using Tensor indices
[18]:
# Select an embedding in emb_sample
id = torch.LongTensor([1])
print(emb_sample(id))
# Select multiple embeddings
ids = torch.LongTensor([1, 3])
print(emb_sample(ids))
# Get the shape of the embedding weight matrix
shape = emb_sample.weight.data.shape
print(shape)
# Overwrite the weight to tensor with all ones
emb_sample.weight.data = torch.ones(shape)
# Let's check if the emb is indeed initilized
ids = torch.LongTensor([0, 3])
print(emb_sample(ids))
tensor([[ 0.2974, 0.5347, 0.8816, -1.5757, 0.4014, -0.8200, -2.2392, 0.3276]],
grad_fn=<EmbeddingBackward0>)
tensor([[ 0.2974, 0.5347, 0.8816, -1.5757, 0.4014, -0.8200, -2.2392, 0.3276],
[-0.5875, -0.4527, -1.1418, -0.7950, -0.6010, 0.3649, 1.2411, -1.3440]],
grad_fn=<EmbeddingBackward0>)
torch.Size([4, 8])
tensor([[1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1.]], grad_fn=<EmbeddingBackward0>)
Now, it’s your time to create node embedding matrix for the graph we have! - We want to have 16 dimensional vector for each node in the karate club network. - We want to initalize the matrix under uniform distribution, in the range of \([0, 1)\). We suggest you using `torch.rand <https://pytorch.org/docs/stable/generated/torch.rand.html>`__.
[19]:
# Please do not change / reset the random seed
torch.manual_seed(1)
def create_node_emb(num_node=34, embedding_dim=16):
# TODO: Implement this function that will create the node embedding matrix.
# A torch.nn.Embedding layer will be returned. You do not need to change
# the values of num_node and embedding_dim. The weight matrix of returned
# layer should be initialized under uniform distribution.
emb = None
############# Your code here ############
emb = nn.Embedding(num_node, embedding_dim)
# initalize the matrix under uniform distribution,
# in the range of [0,1) by using torch.rand
emb.weight.data = torch.rand(num_node, embedding_dim)
#########################################
return emb
emb = create_node_emb()
ids = torch.LongTensor([0, 3])
# Print the embedding layer
print("Embedding: {}".format(emb))
# An example that gets the embeddings for node 0 and 3
print(emb(ids))
Embedding: Embedding(34, 16)
tensor([[0.2114, 0.7335, 0.1433, 0.9647, 0.2933, 0.7951, 0.5170, 0.2801, 0.8339,
0.1185, 0.2355, 0.5599, 0.8966, 0.2858, 0.1955, 0.1808],
[0.7486, 0.6546, 0.3843, 0.9820, 0.6012, 0.3710, 0.4929, 0.9915, 0.8358,
0.4629, 0.9902, 0.7196, 0.2338, 0.0450, 0.7906, 0.9689]],
grad_fn=<EmbeddingBackward0>)
Visualize the initial node embeddings¶
One good way to understand an embedding matrix, is to visualize it in a 2D space. Here, we have implemented an embedding visualization function for you. We first do PCA to reduce the dimensionality of embeddings to a 2D space. Then we visualize each point, colored by the community it belongs to.
[20]:
def visualize_emb(emb):
X = emb.weight.data.numpy()
pca = PCA(n_components=2)
components = pca.fit_transform(X)
plt.figure(figsize=(6, 6))
club1_x = []
club1_y = []
club2_x = []
club2_y = []
for node in G.nodes(data=True):
if node[1]['club'] == 'Mr. Hi':
club1_x.append(components[node[0]][0])
club1_y.append(components[node[0]][1])
else:
club2_x.append(components[node[0]][0])
club2_y.append(components[node[0]][1])
plt.scatter(club1_x, club1_y, color="red", label="Mr. Hi")
plt.scatter(club2_x, club2_y, color="blue", label="Officer")
plt.legend()
plt.show()
# Visualize the initial random embeddding
visualize_emb(emb)
Question 7: Training the embedding! What is the best performance you can get? Please report both the best loss and accuracy on Gradescope. (20 Points)¶
We want to optimize our embeddings for the task of classifying edges as positive or negative. Given an edge and the embeddings for each node, the dot product of the embeddings, followed by a sigmoid, should give us the likelihood of that edge being either positive (output of sigmoid > 0.5) or negative (output of sigmoid < 0.5).
Note that we’re using the functions you wrote in the previous questions, as well as the variables initialized in previous cells. If you’re running into issues, make sure your answers to questions 1-6 are correct.
[23]:
from torch.optim import SGD
import torch.nn as nn
def accuracy(pred, label):
# TODO: Implement the accuracy function. This function takes the
# pred tensor (the resulting tensor after sigmoid) and the label
# tensor (torch.LongTensor). Predicted value greater than 0.5 will
# be classified as label 1. Else it will be classified as label 0.
# The returned accuracy should be rounded to 4 decimal places.
# For example, accuracy 0.82956 will be rounded to 0.8296.
accu = 0.0
############# Your code here ############
# torch.round and sum
accu = (torch.round(pred) == label).sum() / label.size(0)
# rounded to 4 decimal places.
accu = round(accu.item(), 4)
#########################################
return accu
def train(emb, loss_fn, sigmoid, train_label, train_edge):
# TODO: Train the embedding layer here. You can also change epochs and
# learning rate. In general, you need to implement:
# (1) Get the embeddings of the nodes in train_edge
# (2) Dot product the embeddings between each node pair
# (3) Feed the dot product result into sigmoid
# (4) Feed the sigmoid output into the loss_fn
# (5) Print both loss and accuracy of each epoch
# (6) Update the embeddings using the loss and optimizer
# (as a sanity check, the loss should decrease during training)
epochs = 500
learning_rate = 0.1
optimizer = SGD(emb.parameters(), lr=learning_rate, momentum=0.9)
for i in range(epochs):
############# Your code here ############
optimizer.zero_grad()
# 1. Get the embeddings of the nodes in train_edge
node_emb = emb(train_edge)
# 2. Dot product the embeddings between each node pair
dot_product = torch.sum(node_emb[0] * node_emb[1], -1)
# 3. Feed the dot product result into sigmoid
result = sigmoid(dot_product)
# 4. Feed the sigmoid output into the loss_fn
loss = loss_fn(result, train_label)
# 5. Print both loss and accuracy of each epoch
print(f"Epoch: {i} Loss: {loss.item()} Accuracy: {accuracy(result, train_label)}")
# 6. Update the embeddings using the loss and optimizer
loss.backward()
optimizer.step()
#########################################
loss_fn = nn.BCELoss()
sigmoid = nn.Sigmoid()
print(pos_edge_index.shape)
# Generate the positive and negative labels
pos_label = torch.ones(pos_edge_index.shape[1], )
neg_label = torch.zeros(neg_edge_index.shape[1], )
# Concat positive and negative labels into one tensor
train_label = torch.cat([pos_label, neg_label], dim=0)
# Concat positive and negative edges into one tensor
# Since the network is very small, we do not split the edges into val/test sets
train_edge = torch.cat([pos_edge_index, neg_edge_index], dim=1)
print(train_edge.shape)
train(emb, loss_fn, sigmoid, train_label, train_edge)
torch.Size([2, 78])
torch.Size([2, 156])
Epoch: 0 Loss: 2.1047844886779785 Accuracy: 0.5
Epoch: 1 Loss: 2.0906684398651123 Accuracy: 0.5
Epoch: 2 Loss: 2.0640647411346436 Accuracy: 0.5
Epoch: 3 Loss: 2.0266172885894775 Accuracy: 0.5
Epoch: 4 Loss: 1.9799463748931885 Accuracy: 0.5
Epoch: 5 Loss: 1.925624132156372 Accuracy: 0.5
Epoch: 6 Loss: 1.8651514053344727 Accuracy: 0.5
Epoch: 7 Loss: 1.799947738647461 Accuracy: 0.5
Epoch: 8 Loss: 1.731334924697876 Accuracy: 0.5
Epoch: 9 Loss: 1.660534381866455 Accuracy: 0.5
Epoch: 10 Loss: 1.5886584520339966 Accuracy: 0.5
Epoch: 11 Loss: 1.5167080163955688 Accuracy: 0.5
Epoch: 12 Loss: 1.44556725025177 Accuracy: 0.5
Epoch: 13 Loss: 1.3760021924972534 Accuracy: 0.5
Epoch: 14 Loss: 1.3086607456207275 Accuracy: 0.5
Epoch: 15 Loss: 1.2440712451934814 Accuracy: 0.5
Epoch: 16 Loss: 1.182645320892334 Accuracy: 0.5
Epoch: 17 Loss: 1.124683141708374 Accuracy: 0.5
Epoch: 18 Loss: 1.070379614830017 Accuracy: 0.5
Epoch: 19 Loss: 1.019834041595459 Accuracy: 0.5
Epoch: 20 Loss: 0.97306227684021 Accuracy: 0.5
Epoch: 21 Loss: 0.9300077557563782 Accuracy: 0.5
Epoch: 22 Loss: 0.8905553817749023 Accuracy: 0.5064
Epoch: 23 Loss: 0.8545448184013367 Accuracy: 0.5128
Epoch: 24 Loss: 0.8217823505401611 Accuracy: 0.5128
Epoch: 25 Loss: 0.7920518517494202 Accuracy: 0.5192
Epoch: 26 Loss: 0.7651250958442688 Accuracy: 0.5256
Epoch: 27 Loss: 0.740769624710083 Accuracy: 0.5321
Epoch: 28 Loss: 0.7187548279762268 Accuracy: 0.5385
Epoch: 29 Loss: 0.6988576650619507 Accuracy: 0.5513
Epoch: 30 Loss: 0.6808658242225647 Accuracy: 0.5833
Epoch: 31 Loss: 0.6645800471305847 Accuracy: 0.5833
Epoch: 32 Loss: 0.6498157978057861 Accuracy: 0.5897
Epoch: 33 Loss: 0.6364036798477173 Accuracy: 0.5897
Epoch: 34 Loss: 0.6241897344589233 Accuracy: 0.5897
Epoch: 35 Loss: 0.6130346655845642 Accuracy: 0.5833
Epoch: 36 Loss: 0.6028133630752563 Accuracy: 0.6154
Epoch: 37 Loss: 0.5934141874313354 Accuracy: 0.6218
Epoch: 38 Loss: 0.5847375392913818 Accuracy: 0.6346
Epoch: 39 Loss: 0.5766951441764832 Accuracy: 0.6346
Epoch: 40 Loss: 0.5692087411880493 Accuracy: 0.6474
Epoch: 41 Loss: 0.5622095465660095 Accuracy: 0.6538
Epoch: 42 Loss: 0.5556365847587585 Accuracy: 0.6538
Epoch: 43 Loss: 0.549436628818512 Accuracy: 0.6731
Epoch: 44 Loss: 0.5435625910758972 Accuracy: 0.7051
Epoch: 45 Loss: 0.5379732251167297 Accuracy: 0.7244
Epoch: 46 Loss: 0.5326324701309204 Accuracy: 0.7244
Epoch: 47 Loss: 0.5275087356567383 Accuracy: 0.7372
Epoch: 48 Loss: 0.5225741863250732 Accuracy: 0.75
Epoch: 49 Loss: 0.5178046822547913 Accuracy: 0.7564
Epoch: 50 Loss: 0.5131789445877075 Accuracy: 0.7628
Epoch: 51 Loss: 0.5086786150932312 Accuracy: 0.7692
Epoch: 52 Loss: 0.5042873620986938 Accuracy: 0.7756
Epoch: 53 Loss: 0.4999912977218628 Accuracy: 0.7756
Epoch: 54 Loss: 0.49577796459198 Accuracy: 0.7885
Epoch: 55 Loss: 0.491636723279953 Accuracy: 0.7949
Epoch: 56 Loss: 0.4875582158565521 Accuracy: 0.8077
Epoch: 57 Loss: 0.48353445529937744 Accuracy: 0.8077
Epoch: 58 Loss: 0.47955819964408875 Accuracy: 0.8077
Epoch: 59 Loss: 0.47562360763549805 Accuracy: 0.8141
Epoch: 60 Loss: 0.4717252254486084 Accuracy: 0.8333
Epoch: 61 Loss: 0.46785852313041687 Accuracy: 0.8333
Epoch: 62 Loss: 0.46401968598365784 Accuracy: 0.8397
Epoch: 63 Loss: 0.46020516753196716 Accuracy: 0.8397
Epoch: 64 Loss: 0.45641228556632996 Accuracy: 0.8462
Epoch: 65 Loss: 0.4526384770870209 Accuracy: 0.8526
Epoch: 66 Loss: 0.44888168573379517 Accuracy: 0.859
Epoch: 67 Loss: 0.4451402425765991 Accuracy: 0.859
Epoch: 68 Loss: 0.4414125084877014 Accuracy: 0.8782
Epoch: 69 Loss: 0.4376974403858185 Accuracy: 0.9038
Epoch: 70 Loss: 0.43399399518966675 Accuracy: 0.9038
Epoch: 71 Loss: 0.4303014278411865 Accuracy: 0.9038
Epoch: 72 Loss: 0.4266190230846405 Accuracy: 0.9038
Epoch: 73 Loss: 0.4229464828968048 Accuracy: 0.9038
Epoch: 74 Loss: 0.419283390045166 Accuracy: 0.9038
Epoch: 75 Loss: 0.4156295657157898 Accuracy: 0.9103
Epoch: 76 Loss: 0.4119848608970642 Accuracy: 0.9167
Epoch: 77 Loss: 0.40834927558898926 Accuracy: 0.9167
Epoch: 78 Loss: 0.4047229588031769 Accuracy: 0.9167
Epoch: 79 Loss: 0.40110594034194946 Accuracy: 0.9231
Epoch: 80 Loss: 0.3974985182285309 Accuracy: 0.9423
Epoch: 81 Loss: 0.39390090107917786 Accuracy: 0.9423
Epoch: 82 Loss: 0.39031335711479187 Accuracy: 0.9487
Epoch: 83 Loss: 0.38673633337020874 Accuracy: 0.9487
Epoch: 84 Loss: 0.3831700384616852 Accuracy: 0.9551
Epoch: 85 Loss: 0.37961503863334656 Accuracy: 0.9551
Epoch: 86 Loss: 0.3760716915130615 Accuracy: 0.9551
Epoch: 87 Loss: 0.3725404143333435 Accuracy: 0.9551
Epoch: 88 Loss: 0.3690217435359955 Accuracy: 0.9551
Epoch: 89 Loss: 0.36551612615585327 Accuracy: 0.9615
Epoch: 90 Loss: 0.36202409863471985 Accuracy: 0.9679
Epoch: 91 Loss: 0.3585461676120758 Accuracy: 0.9679
Epoch: 92 Loss: 0.35508280992507935 Accuracy: 0.9744
Epoch: 93 Loss: 0.35163459181785583 Accuracy: 0.9744
Epoch: 94 Loss: 0.3482019603252411 Accuracy: 0.9744
Epoch: 95 Loss: 0.3447854816913605 Accuracy: 0.9744
Epoch: 96 Loss: 0.34138572216033936 Accuracy: 0.9744
Epoch: 97 Loss: 0.33800309896469116 Accuracy: 0.9744
Epoch: 98 Loss: 0.33463814854621887 Accuracy: 0.9808
Epoch: 99 Loss: 0.33129143714904785 Accuracy: 0.9808
Epoch: 100 Loss: 0.32796338200569153 Accuracy: 0.9808
Epoch: 101 Loss: 0.32465454936027527 Accuracy: 0.9808
Epoch: 102 Loss: 0.3213653564453125 Accuracy: 0.9808
Epoch: 103 Loss: 0.3180963099002838 Accuracy: 0.9872
Epoch: 104 Loss: 0.314847856760025 Accuracy: 0.9936
Epoch: 105 Loss: 0.31162041425704956 Accuracy: 0.9936
Epoch: 106 Loss: 0.3084144592285156 Accuracy: 0.9936
Epoch: 107 Loss: 0.30523034930229187 Accuracy: 0.9936
Epoch: 108 Loss: 0.3020685613155365 Accuracy: 0.9936
Epoch: 109 Loss: 0.2989294230937958 Accuracy: 0.9936
Epoch: 110 Loss: 0.29581335186958313 Accuracy: 0.9936
Epoch: 111 Loss: 0.2927207052707672 Accuracy: 0.9936
Epoch: 112 Loss: 0.2896518111228943 Accuracy: 0.9936
Epoch: 113 Loss: 0.28660696744918823 Accuracy: 0.9936
Epoch: 114 Loss: 0.2835865020751953 Accuracy: 0.9936
Epoch: 115 Loss: 0.2805907726287842 Accuracy: 0.9936
Epoch: 116 Loss: 0.27761995792388916 Accuracy: 0.9936
Epoch: 117 Loss: 0.27467435598373413 Accuracy: 0.9936
Epoch: 118 Loss: 0.2717542052268982 Accuracy: 0.9936
Epoch: 119 Loss: 0.26885974407196045 Accuracy: 0.9936
Epoch: 120 Loss: 0.2659911811351776 Accuracy: 0.9936
Epoch: 121 Loss: 0.2631486654281616 Accuracy: 0.9936
Epoch: 122 Loss: 0.2603324353694916 Accuracy: 0.9936
Epoch: 123 Loss: 0.25754258036613464 Accuracy: 0.9936
Epoch: 124 Loss: 0.25477927923202515 Accuracy: 0.9936
Epoch: 125 Loss: 0.252042680978775 Accuracy: 0.9936
Epoch: 126 Loss: 0.24933280050754547 Accuracy: 0.9936
Epoch: 127 Loss: 0.2466498166322708 Accuracy: 0.9936
Epoch: 128 Loss: 0.24399375915527344 Accuracy: 0.9936
Epoch: 129 Loss: 0.24136467278003693 Accuracy: 0.9936
Epoch: 130 Loss: 0.23876263201236725 Accuracy: 0.9936
Epoch: 131 Loss: 0.23618771135807037 Accuracy: 0.9936
Epoch: 132 Loss: 0.23363983631134033 Accuracy: 0.9936
Epoch: 133 Loss: 0.23111900687217712 Accuracy: 0.9936
Epoch: 134 Loss: 0.22862525284290314 Accuracy: 0.9936
Epoch: 135 Loss: 0.2261585295200348 Accuracy: 0.9936
Epoch: 136 Loss: 0.2237188071012497 Accuracy: 0.9936
Epoch: 137 Loss: 0.2213059961795807 Accuracy: 0.9936
Epoch: 138 Loss: 0.2189200520515442 Accuracy: 0.9936
Epoch: 139 Loss: 0.21656093001365662 Accuracy: 0.9936
Epoch: 140 Loss: 0.21422846615314484 Accuracy: 0.9936
Epoch: 141 Loss: 0.21192260086536407 Accuracy: 0.9936
Epoch: 142 Loss: 0.20964321494102478 Accuracy: 0.9936
Epoch: 143 Loss: 0.2073902040719986 Accuracy: 0.9936
Epoch: 144 Loss: 0.2051633894443512 Accuracy: 0.9936
Epoch: 145 Loss: 0.20296266674995422 Accuracy: 0.9936
Epoch: 146 Loss: 0.20078787207603455 Accuracy: 0.9936
Epoch: 147 Loss: 0.19863887131214142 Accuracy: 0.9936
Epoch: 148 Loss: 0.19651545584201813 Accuracy: 0.9936
Epoch: 149 Loss: 0.19441744685173035 Accuracy: 1.0
Epoch: 150 Loss: 0.19234472513198853 Accuracy: 1.0
Epoch: 151 Loss: 0.19029705226421356 Accuracy: 1.0
Epoch: 152 Loss: 0.18827421963214874 Accuracy: 1.0
Epoch: 153 Loss: 0.18627609312534332 Accuracy: 1.0
Epoch: 154 Loss: 0.1843024045228958 Accuracy: 1.0
Epoch: 155 Loss: 0.18235298991203308 Accuracy: 1.0
Epoch: 156 Loss: 0.18042761087417603 Accuracy: 1.0
Epoch: 157 Loss: 0.17852602899074554 Accuracy: 1.0
Epoch: 158 Loss: 0.17664805054664612 Accuracy: 1.0
Epoch: 159 Loss: 0.17479346692562103 Accuracy: 1.0
Epoch: 160 Loss: 0.1729619950056076 Accuracy: 1.0
Epoch: 161 Loss: 0.17115344107151031 Accuracy: 1.0
Epoch: 162 Loss: 0.16936755180358887 Accuracy: 1.0
Epoch: 163 Loss: 0.16760413348674774 Accuracy: 1.0
Epoch: 164 Loss: 0.16586287319660187 Accuracy: 1.0
Epoch: 165 Loss: 0.1641436368227005 Accuracy: 1.0
Epoch: 166 Loss: 0.162446066737175 Accuracy: 1.0
Epoch: 167 Loss: 0.1607699990272522 Accuracy: 1.0
Epoch: 168 Loss: 0.15911515057086945 Accuracy: 1.0
Epoch: 169 Loss: 0.15748129785060883 Accuracy: 1.0
Epoch: 170 Loss: 0.15586821734905243 Accuracy: 1.0
Epoch: 171 Loss: 0.15427559614181519 Accuracy: 1.0
Epoch: 172 Loss: 0.15270325541496277 Accuracy: 1.0
Epoch: 173 Loss: 0.15115094184875488 Accuracy: 1.0
Epoch: 174 Loss: 0.14961838722229004 Accuracy: 1.0
Epoch: 175 Loss: 0.14810538291931152 Accuracy: 1.0
Epoch: 176 Loss: 0.14661166071891785 Accuracy: 1.0
Epoch: 177 Loss: 0.1451369822025299 Accuracy: 1.0
Epoch: 178 Loss: 0.143681138753891 Accuracy: 1.0
Epoch: 179 Loss: 0.14224383234977722 Accuracy: 1.0
Epoch: 180 Loss: 0.14082491397857666 Accuracy: 1.0
Epoch: 181 Loss: 0.13942402601242065 Accuracy: 1.0
Epoch: 182 Loss: 0.13804104924201965 Accuracy: 1.0
Epoch: 183 Loss: 0.13667571544647217 Accuracy: 1.0
Epoch: 184 Loss: 0.1353277862071991 Accuracy: 1.0
Epoch: 185 Loss: 0.13399700820446014 Accuracy: 1.0
Epoch: 186 Loss: 0.13268320262432098 Accuracy: 1.0
Epoch: 187 Loss: 0.13138610124588013 Accuracy: 1.0
Epoch: 188 Loss: 0.13010552525520325 Accuracy: 1.0
Epoch: 189 Loss: 0.12884123623371124 Accuracy: 1.0
Epoch: 190 Loss: 0.127592995762825 Accuracy: 1.0
Epoch: 191 Loss: 0.12636059522628784 Accuracy: 1.0
Epoch: 192 Loss: 0.1251438409090042 Accuracy: 1.0
Epoch: 193 Loss: 0.12394252419471741 Accuracy: 1.0
Epoch: 194 Loss: 0.12275642156600952 Accuracy: 1.0
Epoch: 195 Loss: 0.12158531695604324 Accuracy: 1.0
Epoch: 196 Loss: 0.12042900174856186 Accuracy: 1.0
Epoch: 197 Loss: 0.11928731948137283 Accuracy: 1.0
Epoch: 198 Loss: 0.11816001683473587 Accuracy: 1.0
Epoch: 199 Loss: 0.11704691499471664 Accuracy: 1.0
Epoch: 200 Loss: 0.11594784259796143 Accuracy: 1.0
Epoch: 201 Loss: 0.11486257612705231 Accuracy: 1.0
Epoch: 202 Loss: 0.11379092931747437 Accuracy: 1.0
Epoch: 203 Loss: 0.11273273080587387 Accuracy: 1.0
Epoch: 204 Loss: 0.11168777942657471 Accuracy: 1.0
Epoch: 205 Loss: 0.11065591871738434 Accuracy: 1.0
Epoch: 206 Loss: 0.10963691771030426 Accuracy: 1.0
Epoch: 207 Loss: 0.10863065719604492 Accuracy: 1.0
Epoch: 208 Loss: 0.10763691365718842 Accuracy: 1.0
Epoch: 209 Loss: 0.1066555380821228 Accuracy: 1.0
Epoch: 210 Loss: 0.10568637400865555 Accuracy: 1.0
Epoch: 211 Loss: 0.10472921282052994 Accuracy: 1.0
Epoch: 212 Loss: 0.10378391295671463 Accuracy: 1.0
Epoch: 213 Loss: 0.10285031795501709 Accuracy: 1.0
Epoch: 214 Loss: 0.1019282266497612 Accuracy: 1.0
Epoch: 215 Loss: 0.10101751983165741 Accuracy: 1.0
Epoch: 216 Loss: 0.10011804103851318 Accuracy: 1.0
Epoch: 217 Loss: 0.09922961890697479 Accuracy: 1.0
Epoch: 218 Loss: 0.0983520895242691 Accuracy: 1.0
Epoch: 219 Loss: 0.09748531877994537 Accuracy: 1.0
Epoch: 220 Loss: 0.09662915766239166 Accuracy: 1.0
Epoch: 221 Loss: 0.09578343480825424 Accuracy: 1.0
Epoch: 222 Loss: 0.09494806081056595 Accuracy: 1.0
Epoch: 223 Loss: 0.09412283450365067 Accuracy: 1.0
Epoch: 224 Loss: 0.09330764412879944 Accuracy: 1.0
Epoch: 225 Loss: 0.09250236302614212 Accuracy: 1.0
Epoch: 226 Loss: 0.09170682728290558 Accuracy: 1.0
Epoch: 227 Loss: 0.09092090278863907 Accuracy: 1.0
Epoch: 228 Loss: 0.09014448523521423 Accuracy: 1.0
Epoch: 229 Loss: 0.08937741070985794 Accuracy: 1.0
Epoch: 230 Loss: 0.08861956000328064 Accuracy: 1.0
Epoch: 231 Loss: 0.08787082880735397 Accuracy: 1.0
Epoch: 232 Loss: 0.0871310755610466 Accuracy: 1.0
Epoch: 233 Loss: 0.08640015125274658 Accuracy: 1.0
Epoch: 234 Loss: 0.08567798137664795 Accuracy: 1.0
Epoch: 235 Loss: 0.08496440947055817 Accuracy: 1.0
Epoch: 236 Loss: 0.08425934612751007 Accuracy: 1.0
Epoch: 237 Loss: 0.08356264978647232 Accuracy: 1.0
Epoch: 238 Loss: 0.08287421613931656 Accuracy: 1.0
Epoch: 239 Loss: 0.08219393342733383 Accuracy: 1.0
Epoch: 240 Loss: 0.08152167499065399 Accuracy: 1.0
Epoch: 241 Loss: 0.08085735887289047 Accuracy: 1.0
Epoch: 242 Loss: 0.08020085096359253 Accuracy: 1.0
Epoch: 243 Loss: 0.0795520469546318 Accuracy: 1.0
Epoch: 244 Loss: 0.07891084998846054 Accuracy: 1.0
Epoch: 245 Loss: 0.07827717065811157 Accuracy: 1.0
Epoch: 246 Loss: 0.07765087485313416 Accuracy: 1.0
Epoch: 247 Loss: 0.07703190296888351 Accuracy: 1.0
Epoch: 248 Loss: 0.07642010599374771 Accuracy: 1.0
Epoch: 249 Loss: 0.07581540197134018 Accuracy: 1.0
Epoch: 250 Loss: 0.07521773129701614 Accuracy: 1.0
Epoch: 251 Loss: 0.07462694495916367 Accuracy: 1.0
Epoch: 252 Loss: 0.07404298335313797 Accuracy: 1.0
Epoch: 253 Loss: 0.0734657496213913 Accuracy: 1.0
Epoch: 254 Loss: 0.07289513200521469 Accuracy: 1.0
Epoch: 255 Loss: 0.07233106344938278 Accuracy: 1.0
Epoch: 256 Loss: 0.07177344709634781 Accuracy: 1.0
Epoch: 257 Loss: 0.07122218608856201 Accuracy: 1.0
Epoch: 258 Loss: 0.07067721337080002 Accuracy: 1.0
Epoch: 259 Loss: 0.07013843953609467 Accuracy: 1.0
Epoch: 260 Loss: 0.0696057602763176 Accuracy: 1.0
Epoch: 261 Loss: 0.06907911598682404 Accuracy: 1.0
Epoch: 262 Loss: 0.06855840981006622 Accuracy: 1.0
Epoch: 263 Loss: 0.06804358214139938 Accuracy: 1.0
Epoch: 264 Loss: 0.06753452867269516 Accuracy: 1.0
Epoch: 265 Loss: 0.06703118979930878 Accuracy: 1.0
Epoch: 266 Loss: 0.06653348356485367 Accuracy: 1.0
Epoch: 267 Loss: 0.06604133546352386 Accuracy: 1.0
Epoch: 268 Loss: 0.0655546709895134 Accuracy: 1.0
Epoch: 269 Loss: 0.0650734081864357 Accuracy: 1.0
Epoch: 270 Loss: 0.0645974725484848 Accuracy: 1.0
Epoch: 271 Loss: 0.06412681937217712 Accuracy: 1.0
Epoch: 272 Loss: 0.0636613517999649 Accuracy: 1.0
Epoch: 273 Loss: 0.06320100277662277 Accuracy: 1.0
Epoch: 274 Loss: 0.06274572014808655 Accuracy: 1.0
Epoch: 275 Loss: 0.06229541078209877 Accuracy: 1.0
Epoch: 276 Loss: 0.061850033700466156 Accuracy: 1.0
Epoch: 277 Loss: 0.061409514397382736 Accuracy: 1.0
Epoch: 278 Loss: 0.06097378581762314 Accuracy: 1.0
Epoch: 279 Loss: 0.06054277345538139 Accuracy: 1.0
Epoch: 280 Loss: 0.06011643633246422 Accuracy: 1.0
Epoch: 281 Loss: 0.05969469994306564 Accuracy: 1.0
Epoch: 282 Loss: 0.059277504682540894 Accuracy: 1.0
Epoch: 283 Loss: 0.05886480212211609 Accuracy: 1.0
Epoch: 284 Loss: 0.05845651403069496 Accuracy: 1.0
Epoch: 285 Loss: 0.058052580803632736 Accuracy: 1.0
Epoch: 286 Loss: 0.057652976363897324 Accuracy: 1.0
Epoch: 287 Loss: 0.057257603853940964 Accuracy: 1.0
Epoch: 288 Loss: 0.05686642602086067 Accuracy: 1.0
Epoch: 289 Loss: 0.05647939443588257 Accuracy: 1.0
Epoch: 290 Loss: 0.05609644949436188 Accuracy: 1.0
Epoch: 291 Loss: 0.055717527866363525 Accuracy: 1.0
Epoch: 292 Loss: 0.05534258857369423 Accuracy: 1.0
Epoch: 293 Loss: 0.054971568286418915 Accuracy: 1.0
Epoch: 294 Loss: 0.054604433476924896 Accuracy: 1.0
Epoch: 295 Loss: 0.0542411208152771 Accuracy: 1.0
Epoch: 296 Loss: 0.05388158932328224 Accuracy: 1.0
Epoch: 297 Loss: 0.05352577939629555 Accuracy: 1.0
Epoch: 298 Loss: 0.053173646330833435 Accuracy: 1.0
Epoch: 299 Loss: 0.052825137972831726 Accuracy: 1.0
Epoch: 300 Loss: 0.05248021334409714 Accuracy: 1.0
Epoch: 301 Loss: 0.05213884264230728 Accuracy: 1.0
Epoch: 302 Loss: 0.051800940185785294 Accuracy: 1.0
Epoch: 303 Loss: 0.051466502249240875 Accuracy: 1.0
Epoch: 304 Loss: 0.05113545432686806 Accuracy: 1.0
Epoch: 305 Loss: 0.05080777406692505 Accuracy: 1.0
Epoch: 306 Loss: 0.05048339068889618 Accuracy: 1.0
Epoch: 307 Loss: 0.050162289291620255 Accuracy: 1.0
Epoch: 308 Loss: 0.04984442517161369 Accuracy: 1.0
Epoch: 309 Loss: 0.049529749900102615 Accuracy: 1.0
Epoch: 310 Loss: 0.04921821877360344 Accuracy: 1.0
Epoch: 311 Loss: 0.04890979826450348 Accuracy: 1.0
Epoch: 312 Loss: 0.04860444366931915 Accuracy: 1.0
Epoch: 313 Loss: 0.048302121460437775 Accuracy: 1.0
Epoch: 314 Loss: 0.04800278693437576 Accuracy: 1.0
Epoch: 315 Loss: 0.04770641773939133 Accuracy: 1.0
Epoch: 316 Loss: 0.04741295427083969 Accuracy: 1.0
Epoch: 317 Loss: 0.047122374176979065 Accuracy: 1.0
Epoch: 318 Loss: 0.046834640204906464 Accuracy: 1.0
Epoch: 319 Loss: 0.0465497151017189 Accuracy: 1.0
Epoch: 320 Loss: 0.0462675616145134 Accuracy: 1.0
Epoch: 321 Loss: 0.04598815366625786 Accuracy: 1.0
Epoch: 322 Loss: 0.045711442828178406 Accuracy: 1.0
Epoch: 323 Loss: 0.04543740302324295 Accuracy: 1.0
Epoch: 324 Loss: 0.0451660081744194 Accuracy: 1.0
Epoch: 325 Loss: 0.044897209852933884 Accuracy: 1.0
Epoch: 326 Loss: 0.0446309857070446 Accuracy: 1.0
Epoch: 327 Loss: 0.04436729475855827 Accuracy: 1.0
Epoch: 328 Loss: 0.044106125831604004 Accuracy: 1.0
Epoch: 329 Loss: 0.04384743422269821 Accuracy: 1.0
Epoch: 330 Loss: 0.04359118267893791 Accuracy: 1.0
Epoch: 331 Loss: 0.04333735629916191 Accuracy: 1.0
Epoch: 332 Loss: 0.04308591037988663 Accuracy: 1.0
Epoch: 333 Loss: 0.04283682629466057 Accuracy: 1.0
Epoch: 334 Loss: 0.042590077966451645 Accuracy: 1.0
Epoch: 335 Loss: 0.042345624417066574 Accuracy: 1.0
Epoch: 336 Loss: 0.042103447020053864 Accuracy: 1.0
Epoch: 337 Loss: 0.041863519698381424 Accuracy: 1.0
Epoch: 338 Loss: 0.04162580519914627 Accuracy: 1.0
Epoch: 339 Loss: 0.04139029607176781 Accuracy: 1.0
Epoch: 340 Loss: 0.041156940162181854 Accuracy: 1.0
Epoch: 341 Loss: 0.040925733745098114 Accuracy: 1.0
Epoch: 342 Loss: 0.0406966358423233 Accuracy: 1.0
Epoch: 343 Loss: 0.04046962410211563 Accuracy: 1.0
Epoch: 344 Loss: 0.040244679898023605 Accuracy: 1.0
Epoch: 345 Loss: 0.040021780878305435 Accuracy: 1.0
Epoch: 346 Loss: 0.039800889790058136 Accuracy: 1.0
Epoch: 347 Loss: 0.03958198428153992 Accuracy: 1.0
Epoch: 348 Loss: 0.03936506062746048 Accuracy: 1.0
Epoch: 349 Loss: 0.03915007412433624 Accuracy: 1.0
Epoch: 350 Loss: 0.03893701359629631 Accuracy: 1.0
Epoch: 351 Loss: 0.0387258417904377 Accuracy: 1.0
Epoch: 352 Loss: 0.03851655498147011 Accuracy: 1.0
Epoch: 353 Loss: 0.038309115916490555 Accuracy: 1.0
Epoch: 354 Loss: 0.03810351341962814 Accuracy: 1.0
Epoch: 355 Loss: 0.037899721413850784 Accuracy: 1.0
Epoch: 356 Loss: 0.037697724997997284 Accuracy: 1.0
Epoch: 357 Loss: 0.03749749809503555 Accuracy: 1.0
Epoch: 358 Loss: 0.0372990146279335 Accuracy: 1.0
Epoch: 359 Loss: 0.037102263420820236 Accuracy: 1.0
Epoch: 360 Loss: 0.036907222121953964 Accuracy: 1.0
Epoch: 361 Loss: 0.0367138646543026 Accuracy: 1.0
Epoch: 362 Loss: 0.03652217984199524 Accuracy: 1.0
Epoch: 363 Loss: 0.0363321453332901 Accuracy: 1.0
Epoch: 364 Loss: 0.036143749952316284 Accuracy: 1.0
Epoch: 365 Loss: 0.03595695644617081 Accuracy: 1.0
Epoch: 366 Loss: 0.03577176108956337 Accuracy: 1.0
Epoch: 367 Loss: 0.03558814525604248 Accuracy: 1.0
Epoch: 368 Loss: 0.03540608659386635 Accuracy: 1.0
Epoch: 369 Loss: 0.03522557020187378 Accuracy: 1.0
Epoch: 370 Loss: 0.035046570003032684 Accuracy: 1.0
Epoch: 371 Loss: 0.03486909344792366 Accuracy: 1.0
Epoch: 372 Loss: 0.034693095833063126 Accuracy: 1.0
Epoch: 373 Loss: 0.03451857343316078 Accuracy: 1.0
Epoch: 374 Loss: 0.03434550017118454 Accuracy: 1.0
Epoch: 375 Loss: 0.0341738760471344 Accuracy: 1.0
Epoch: 376 Loss: 0.03400367125868797 Accuracy: 1.0
Epoch: 377 Loss: 0.033834874629974365 Accuracy: 1.0
Epoch: 378 Loss: 0.03366747871041298 Accuracy: 1.0
Epoch: 379 Loss: 0.03350144624710083 Accuracy: 1.0
Epoch: 380 Loss: 0.033336784690618515 Accuracy: 1.0
Epoch: 381 Loss: 0.03317347541451454 Accuracy: 1.0
Epoch: 382 Loss: 0.03301149234175682 Accuracy: 1.0
Epoch: 383 Loss: 0.032850828021764755 Accuracy: 1.0
Epoch: 384 Loss: 0.03269147127866745 Accuracy: 1.0
Epoch: 385 Loss: 0.032533396035432816 Accuracy: 1.0
Epoch: 386 Loss: 0.032376598566770554 Accuracy: 1.0
Epoch: 387 Loss: 0.03222106024622917 Accuracy: 1.0
Epoch: 388 Loss: 0.03206678107380867 Accuracy: 1.0
Epoch: 389 Loss: 0.03191373124718666 Accuracy: 1.0
Epoch: 390 Loss: 0.03176189213991165 Accuracy: 1.0
Epoch: 391 Loss: 0.03161126375198364 Accuracy: 1.0
Epoch: 392 Loss: 0.031461842358112335 Accuracy: 1.0
Epoch: 393 Loss: 0.031313590705394745 Accuracy: 1.0
Epoch: 394 Loss: 0.03116651624441147 Accuracy: 1.0
Epoch: 395 Loss: 0.031020594760775566 Accuracy: 1.0
Epoch: 396 Loss: 0.03087582066655159 Accuracy: 1.0
Epoch: 397 Loss: 0.030732175335288048 Accuracy: 1.0
Epoch: 398 Loss: 0.030589653179049492 Accuracy: 1.0
Epoch: 399 Loss: 0.03044823743402958 Accuracy: 1.0
Epoch: 400 Loss: 0.030307918787002563 Accuracy: 1.0
Epoch: 401 Loss: 0.030168689787387848 Accuracy: 1.0
Epoch: 402 Loss: 0.03003053553402424 Accuracy: 1.0
Epoch: 403 Loss: 0.02989344671368599 Accuracy: 1.0
Epoch: 404 Loss: 0.02975740097463131 Accuracy: 1.0
Epoch: 405 Loss: 0.029622405767440796 Accuracy: 1.0
Epoch: 406 Loss: 0.029488425701856613 Accuracy: 1.0
Epoch: 407 Loss: 0.029355483129620552 Accuracy: 1.0
Epoch: 408 Loss: 0.02922353707253933 Accuracy: 1.0
Epoch: 409 Loss: 0.02909259870648384 Accuracy: 1.0
Epoch: 410 Loss: 0.028962649405002594 Accuracy: 1.0
Epoch: 411 Loss: 0.028833670541644096 Accuracy: 1.0
Epoch: 412 Loss: 0.02870565466582775 Accuracy: 1.0
Epoch: 413 Loss: 0.02857859991490841 Accuracy: 1.0
Epoch: 414 Loss: 0.02845250628888607 Accuracy: 1.0
Epoch: 415 Loss: 0.028327345848083496 Accuracy: 1.0
Epoch: 416 Loss: 0.028203114867210388 Accuracy: 1.0
Epoch: 417 Loss: 0.028079796582460403 Accuracy: 1.0
Epoch: 418 Loss: 0.027957402169704437 Accuracy: 1.0
Epoch: 419 Loss: 0.027835901826620102 Accuracy: 1.0
Epoch: 420 Loss: 0.0277152881026268 Accuracy: 1.0
Epoch: 421 Loss: 0.02759556844830513 Accuracy: 1.0
Epoch: 422 Loss: 0.0274767205119133 Accuracy: 1.0
Epoch: 423 Loss: 0.02735874056816101 Accuracy: 1.0
Epoch: 424 Loss: 0.02724161185324192 Accuracy: 1.0
Epoch: 425 Loss: 0.027125339955091476 Accuracy: 1.0
Epoch: 426 Loss: 0.027009902521967888 Accuracy: 1.0
Epoch: 427 Loss: 0.026895299553871155 Accuracy: 1.0
Epoch: 428 Loss: 0.026781519874930382 Accuracy: 1.0
Epoch: 429 Loss: 0.02666855789721012 Accuracy: 1.0
Epoch: 430 Loss: 0.026556402444839478 Accuracy: 1.0
Epoch: 431 Loss: 0.026445046067237854 Accuracy: 1.0
Epoch: 432 Loss: 0.026334485039114952 Accuracy: 1.0
Epoch: 433 Loss: 0.026224708184599876 Accuracy: 1.0
Epoch: 434 Loss: 0.026115702465176582 Accuracy: 1.0
Epoch: 435 Loss: 0.02600747160613537 Accuracy: 1.0
Epoch: 436 Loss: 0.02590000256896019 Accuracy: 1.0
Epoch: 437 Loss: 0.0257932897657156 Accuracy: 1.0
Epoch: 438 Loss: 0.025687318295240402 Accuracy: 1.0
Epoch: 439 Loss: 0.025582091882824898 Accuracy: 1.0
Epoch: 440 Loss: 0.025477595627307892 Accuracy: 1.0
Epoch: 441 Loss: 0.025373823940753937 Accuracy: 1.0
Epoch: 442 Loss: 0.02527078613638878 Accuracy: 1.0
Epoch: 443 Loss: 0.025168443098664284 Accuracy: 1.0
Epoch: 444 Loss: 0.025066813454031944 Accuracy: 1.0
Epoch: 445 Loss: 0.024965884163975716 Accuracy: 1.0
Epoch: 446 Loss: 0.024865640327334404 Accuracy: 1.0
Epoch: 447 Loss: 0.024766091257333755 Accuracy: 1.0
Epoch: 448 Loss: 0.02466721460223198 Accuracy: 1.0
Epoch: 449 Loss: 0.024569014087319374 Accuracy: 1.0
Epoch: 450 Loss: 0.024471474811434746 Accuracy: 1.0
Epoch: 451 Loss: 0.02437460608780384 Accuracy: 1.0
Epoch: 452 Loss: 0.024278385564684868 Accuracy: 1.0
Epoch: 453 Loss: 0.02418280951678753 Accuracy: 1.0
Epoch: 454 Loss: 0.024087881669402122 Accuracy: 1.0
Epoch: 455 Loss: 0.023993590846657753 Accuracy: 1.0
Epoch: 456 Loss: 0.023899927735328674 Accuracy: 1.0
Epoch: 457 Loss: 0.02380688674747944 Accuracy: 1.0
Epoch: 458 Loss: 0.023714469745755196 Accuracy: 1.0
Epoch: 459 Loss: 0.02362266182899475 Accuracy: 1.0
Epoch: 460 Loss: 0.023531462997198105 Accuracy: 1.0
Epoch: 461 Loss: 0.023440862074494362 Accuracy: 1.0
Epoch: 462 Loss: 0.023350868374109268 Accuracy: 1.0
Epoch: 463 Loss: 0.02326146326959133 Accuracy: 1.0
Epoch: 464 Loss: 0.023172637447714806 Accuracy: 1.0
Epoch: 465 Loss: 0.02308439463376999 Accuracy: 1.0
Epoch: 466 Loss: 0.022996731102466583 Accuracy: 1.0
Epoch: 467 Loss: 0.022909637540578842 Accuracy: 1.0
Epoch: 468 Loss: 0.02282310090959072 Accuracy: 1.0
Epoch: 469 Loss: 0.022737130522727966 Accuracy: 1.0
Epoch: 470 Loss: 0.02265171706676483 Accuracy: 1.0
Epoch: 471 Loss: 0.022566847503185272 Accuracy: 1.0
Epoch: 472 Loss: 0.022482529282569885 Accuracy: 1.0
Epoch: 473 Loss: 0.02239874005317688 Accuracy: 1.0
Epoch: 474 Loss: 0.02231549844145775 Accuracy: 1.0
Epoch: 475 Loss: 0.02223278395831585 Accuracy: 1.0
Epoch: 476 Loss: 0.022150592878460884 Accuracy: 1.0
Epoch: 477 Loss: 0.022068927064538002 Accuracy: 1.0
Epoch: 478 Loss: 0.021987777203321457 Accuracy: 1.0
Epoch: 479 Loss: 0.0219071377068758 Accuracy: 1.0
Epoch: 480 Loss: 0.021827004849910736 Accuracy: 1.0
Epoch: 481 Loss: 0.02174738049507141 Accuracy: 1.0
Epoch: 482 Loss: 0.02166825346648693 Accuracy: 1.0
Epoch: 483 Loss: 0.0215896163135767 Accuracy: 1.0
Epoch: 484 Loss: 0.02151147462427616 Accuracy: 1.0
Epoch: 485 Loss: 0.02143382467329502 Accuracy: 1.0
Epoch: 486 Loss: 0.021356645971536636 Accuracy: 1.0
Epoch: 487 Loss: 0.021279949694871902 Accuracy: 1.0
Epoch: 488 Loss: 0.021203726530075073 Accuracy: 1.0
Epoch: 489 Loss: 0.02112797647714615 Accuracy: 1.0
Epoch: 490 Loss: 0.021052684634923935 Accuracy: 1.0
Epoch: 491 Loss: 0.020977865904569626 Accuracy: 1.0
Epoch: 492 Loss: 0.02090349793434143 Accuracy: 1.0
Epoch: 493 Loss: 0.0208295825868845 Accuracy: 1.0
Epoch: 494 Loss: 0.02075611613690853 Accuracy: 1.0
Epoch: 495 Loss: 0.020683100447058678 Accuracy: 1.0
Epoch: 496 Loss: 0.020610524341464043 Accuracy: 1.0
Epoch: 497 Loss: 0.020538389682769775 Accuracy: 1.0
Epoch: 498 Loss: 0.02046668902039528 Accuracy: 1.0
Epoch: 499 Loss: 0.020395418629050255 Accuracy: 1.0
Visualize the final node embeddings¶
Visualize your final embedding here! You can visually compare the figure with the previous embedding figure. After training, you should oberserve that the two classes are more evidently separated. This is a great sanitity check for your implementation as well.
[24]:
# Visualize the final learned embedding
visualize_emb(emb)
Submission¶
In order to get credit, you must go submit your answers on Gradescope.