Jupyter notebook crashing server - jupyter-notebook

Question

I just bought a GPU to put in a computer where I'm hosting a jupyter notebook. I tunnel the output of the jupyter notebook from the tower to my laptop via ssh. I'm running some code and the jupyter notebook freezes at the same line every time. Not only does the jupyter notebook freeze, but everything on the tower. If I have any other ssh connections, they freeze. If I use the GUI on the tower directly it's frozen. Nothing is responsive until I hit the power button to reset the computer.
The odd thing is, that while nothing is responding, nothing is timing out either. the ssh sessions keep their connections. the jupyter notebook homepage claims it's still connected. It's very odd, and I'm not sure this is a problem with the code or the tower somehow, so I'm not sure if I should post this here or somewhere else. But here's the code
def show_img(x):
x = x.clone().detach().permute(1,2,0).numpy()
print(x.shape)
x = rio.convert_tensor_to_rgb(x)
print(x.shape)
plt.figure(figsize=(8, 8))
plt.axis('off')
_ = plt.imshow(x)
# define generator discriminator, dataloader, other stuff....
G.cuda()
D.cuda()
g_optim = optim.RMSprop(G.parameters(), lr=lr)
d_optim = optim.RMSprop(D.parameters(), lr=lr)
g_losses = []
d_losses = []
i_losses = []
for epoch in range(n_epochs):
dataloader = DataLoader(train_dataset, batch_size=batch_size,
shuffle=True, pin_memory=True,
num_workers=num_workers)
g_loss = 0.0 # g_loss is the generator's loss
d_loss = 0.0 # d_loss is the discriminator's loss
i_loss = 0.0 # i_loss is the generator's loss for not being invertable
i_weight = 10 # prioritize being invertible ten times more than minimizing g_loss
x,y,z = train_dataset[0]
print("image")
show_img(x)
print("target")
show_img(y)
print("generated")
x,y = x.cuda(), y.cuda()
g = G(x.unsqueeze(0),y.unsqueeze(0))
print(g.shape)
show_img(g.squeeze().cpu())
loop = tqdm(total=len(dataloader), position=0, file=sys.stdout)
print("just to be sure") #prints this
for minibatch, (image, batchImage, exp_batch) in enumerate(dataloader): #this is the line it freezes on?
print("image ", image.shape, " batchImage ", batchImage.shape, " experiment batch ", exp_batch) # doesn't print this. already frozen
EDIT: GUI and ssh are responsive, but unusually slow. I guess the main problem is that the code is still freezing on said line of code, and I don't know why.

Answer 1

I found The problem. The batch size was 32, which was much to large given the size of the images. I think it justed crashed after trying to load them all.