dask bokeh port not reused - bokeh

Question

I'm running some extended tests under dask. I create a LocalCluster and a Client from it, do some processing and then shutdown the Client and close the LocalCluster. On the first test, port 8787 is used for bokeh as expected. However on subsequent tests, the port number is not 8787 but instead some random number. Here's a script that illustrates the problem:
from distributed import Client
import time
if __name__ == '__main__':
max_n_workers = 8
print('Maximum number of workers is ', max_n_workers)
n_workers = max_n_workers
while n_workers > 1:
c = Client(n_workers=n_workers)
print(c)
addr = c.scheduler_info()['address']
services = c.scheduler_info()['services']
if 'bokeh' in services.keys():
bokeh_addr = 'http:%s:%s' % (addr.split(':')[1], services['bokeh'])
print('Diagnostic pages available on port %s' % bokeh_addr)
c.shutdown()
n_workers = n_workers // 2
time.sleep(10)
exit()
And an example output is
Maximum number of workers is 8
<Client: scheduler='tcp://127.0.0.1:41049' processes=8 cores=16>
Diagnostic pages available on port http://127.0.0.1:8787
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
<Client: scheduler='tcp://127.0.0.1:34152' processes=4 cores=16>
Diagnostic pages available on port http://127.0.0.1:39621
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed
<Client: scheduler='tcp://127.0.0.1:37583' processes=2 cores=16>
Diagnostic pages available on port http://127.0.0.1:45183
distributed.nanny - INFO - Worker closed
distributed.nanny - INFO - Worker closed

Answer 1

When you close the LocalCluster it shuts down all services. This gives back all previously used ports to the operating system. When you start up a new LocalCluster it asks the operating system for these ports again. If the bokeh port is not yet accessible then it will choose a new random port rather than fail.
In your case I suspect that the operating system has not yet registered the default port as being open. I suspect that if you gave the operating system some time that things would clear up.