Renderer not updating glyph on plot in bokehjs - bokeh

Question

UPDATE 25-09-2018
In a bid to create a simpler demo (posted here: http://jsfiddle.net/bLgj4vc7/3/) for this question, I finally discovered the root cause of the problem. I have also managed to work around it, which I have posted as an answer.
I am using standalone bokehjs in my webapp. My problem is that the renderer is not updating its line glyph with updated datasource (or at least this is what the problem looks like to me).
This jsfiddle: http://jsfiddle.net/uwnqcotg/8/ demonstrates the problem. Please ignore any bad coding conventions in it; its a quick, one-off demo.
In the fiddle, the plot is loaded with a dataseries which is rendered fine and dandy. The plot can be updated in two ways:
Update Plot Once By Overwriting CDS.data Object overwrites the ColumnDataSource.data object at one-go, thus adhering to the bokeh principle of maintaining same column lengths.
Start Stream With CDS.stream employs the ColumnDataSource.stream() and updates the plot at a regular interval with randomly generated, but incremental dataseries.
In the first approach, it looks like the glyph is being updated - the axes sure are updated in accordance with the dataseries printed in the console for reference. But, the plotted glyph and the dataseries show a mismatch. To me, it looks like the glyph plotted when the plot initialized simply re-adjusted to the new axes, though I cannot be sure.
In the second approach, again, the axes are updated, but the glyph isn't. Again, the originally plotted glyph seem to re-adjust to the new axes ranges. Moreover, the rollover attr seems to be ignored as the renderer simply keeps accumulating data.
Overall, my observation is that any data provided to the renderer during its initialization is plotted correctly, but any subsequent streamed update or even an overwrite are not. Though I may be wrong here.
On the other hand, between me and bokeh, I am inclined to believe I am doing something wrong or am missing something. So what is it?

Answer 1

As updated in the question, in a bid to create a simpler demo posted here http://jsfiddle.net/bLgj4vc7/3/, I ran across the root cause of this problem:
It looks like bokehjs has a bug where a renderer, once initialized with a certain length of dataset, sticks to that length even for subsequent updates in dataset. In other words,
If a renderer was initialized with an empty dataset {x:[], y:[]}, even if the renderer.data_source is updated with a populated dataset later on, the renderer sticks to the zero length and nothing is drawn on the plot.
For an initial dataset of length 5, only first 5 points will be read from any subsequent updates and the glyph will be updated accordinly. Rest of the points are not acknowledged.
I don't know whether the problem is whether renderer doesn't read beyond the initial length or whether it does but fails to redraw the glyph on the plot. The ColumnDataSource sure is updated correctly.
The fiddle linked in this answer best demonstrates the problem.
So what is the solution? At the moment, this workaround: initialize the renderer with a dataset filled with NaNs, which has a length equal or more than the maximum number of datapoints you want to plot for a single dataseries.
So, if in a line glyph, I wish to plot maximum 100 datapoints, then:
const plot = Bokeh.Plotting.figure()
const maxDataPoints = 100 // maximum number of points I want for this glyph
const emptyData = Array().fill().map(_ => NaN)
const cds = new Bokeh.ColumnDataSource({x: emptyData, y: emptyData})
const renderer = plot.line({field: 'x'}, {field: 'y'}, {source: cds, line_width: 2})
// now any subsequent updates to datasource of renderer will update its glyph on the plot.
renderer.data_source.data = {
x: dataLengthLessOrEqualToMaxDataPoints.x,
y: dataLengthLessOrEqualToMaxDataPoints.y
}
Of course, I think this behaviour of bokehjs is either a bug or I am missing an undocumented step in initialization. Either way, its worth creating an issue on github: https://github.com/bokeh/bokeh/issues/8277