Blazor preload data even though the data is not completed - asynchronous

Question

Is there a way to show the user the data that has been processed in real time?
I mean: suppose this scenario: I created a default Blazor app to use it as example. Go to the default page FetchData, it uses as model the following:
public class WeatherForecast
{
public DateOnly Date { get; set; }
public int TemperatureC { get; set; }
public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
public string? Summary { get; set; }
}
The page uses this mock service as default:
public class WeatherForecastService
{
private static readonly string[] Summaries = new[]
{
"Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};
public Task<WeatherForecast[]> GetForecastAsync(DateOnly startDate)
{
return Task.FromResult(Enumerable.Range(1, 5).Select(index => new WeatherForecast
{
Date = startDate.AddDays(index),
TemperatureC = Random.Shared.Next(-20, 55),
Summary = Summaries[Random.Shared.Next(Summaries.Length)]
}).ToArray());
}
}
Here's the page where the service is used:
#page "/fetchdata"
#using BlazorApp1.Data
#inject WeatherForecastService ForecastService
<PageTitle>Weather forecast</PageTitle>
<h1>Weather forecast</h1>
<p>This component demonstrates fetching data from a service.</p>
#if (forecasts == null)
{
<p><em>Loading...</em></p>
}
else
{
<table class="table">
<thead>
<tr>
<th>Date</th>
<th>Temp. (C)</th>
<th>Temp. (F)</th>
<th>Summary</th>
</tr>
</thead>
<tbody>
#foreach (var forecast in forecasts)
{
<tr>
<td>#forecast.Date.ToShortDateString()</td>
<td>#forecast.TemperatureC</td>
<td>#forecast.TemperatureF</td>
<td>#forecast.Summary</td>
</tr>
}
</tbody>
</table>
}
#code {
private WeatherForecast[]? forecasts;
protected override async Task OnInitializedAsync()
{
forecasts = await ForecastService.GetForecastAsync(DateOnly.FromDateTime(DateTime.Now));
}
}
Okay so now suppose that the service instead of using 5 items we put a high number such as 2000000 by example.
public Task<WeatherForecast[]> GetForecastAsync(DateOnly startDate)
{
return Task.FromResult(Enumerable.Range(1, 20000000).Select(index => new WeatherForecast
{
Date = startDate.AddDays(index),
TemperatureC = Random.Shared.Next(-20, 55),
Summary = Summaries[Random.Shared.Next(Summaries.Length)]
}).ToArray());
}
When we reload the page it wont render until all the weather forecasts are loaded.
What is the best way to render the data in real time to show the user the page even though the service is working backwards?

Answer 1

If you'd like to receive each part of the data as soon as it arrives, rather than waiting for the complete data to be available, you can use AsAsyncEnumerable. You wouldn't like to block the CPU while waiting for the chunks of data. This is where IAsyncEnumerable can help you.
You can use AsAsyncEnumerable in your query to achieve the better performance, such as bellow:
public IAsyncEnumerable<WeatherForecast> GetForecastAsync()
{
return _context.WeathreForecasts
.AsAsyncEnumerable();
}
The photo below shows this:
General async method (e.g. .ToListAsync()):
AsAsyncEnumerable method:
For more info you can refer to IAsyncEnumerable with yield in C#

Answer 2

In your example, where you need to handle very large data sets in an expedite manner, the simplest way is to use the Virtualize component. This demonstrates several key principles in handling lists:
Only get the data you actually need.
Pass request objects into the data pipeline and get result objects back.
Use IEnumerable in the pipeline to minimize materialization.
The data pipeline needs restructuring to better resemble a real world example.
Break out the data provider.
public class WeatherForecastProvider
{
private IEnumerable<WeatherForecast> _records = Enumerable.Empty<WeatherForecast>();
private static readonly string[] Summaries = new[]
{
"Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};
public WeatherForecastProvider()
=> GetForecasts();
public IEnumerable<WeatherForecast> WeatherForecasts
=> _records;
private void GetForecasts()
{
DateOnly startDate = DateOnly.FromDateTime(DateTime.Now);
_records = Enumerable.Range(1, 2000000).Select(index => new WeatherForecast
{
Date = startDate.AddDays(index),
TemperatureC = Random.Shared.Next(-20, 55),
Summary = Summaries[Random.Shared.Next(Summaries.Length)]
});
}
}
WeatherForecastService now looks like this. It's the data broker between the application and the data source.
public class WeatherForecastService
{
private WeatherForecastProvider _weatherForecastProvider;
public WeatherForecastService(WeatherForecastProvider weatherForecastProvider)
=> _weatherForecastProvider= weatherForecastProvider;
public async ValueTask<ItemsProviderResult<WeatherForecast>> GetForecastAsync(ItemsProviderRequest request)
{
// normally request will be async calling into a DbContext or API, so pretend it is
await Task.Delay(250);
var query = _weatherForecastProvider.WeatherForecasts;
var count = query.Count();
query = query
.Skip(request.StartIndex)
.Take(request.Count);
return new ItemsProviderResult<WeatherForecast>(query, count);
}
}
And then FetchData looks like this.
#page "/fetchdata"
#using SO75219173;
#inject WeatherForecastService ForecastService
<PageTitle>Weather forecast</PageTitle>
<h1>Weather forecast</h1>
<p>This component demonstrates fetching data from a service.</p>
<table class="table">
<thead>
<tr>
<th>Date</th>
<th>Temp. (C)</th>
<th>Temp. (F)</th>
<th>Summary</th>
</tr>
</thead>
<tbody>
<Virtualize Context="forecast" ItemsProvider=this.ForecastService.GetForecastAsync>
<tr>
<td>#forecast.Date.ToShortDateString()</td>
<td>#forecast.TemperatureC</td>
<td>#forecast.TemperatureF</td>
<td>#forecast.Summary</td>
</tr>
</Virtualize>
</tbody>
</table>
#code {
}
Note the use of IEnumerable throughout. The methods don't pass Lists or Arrays.
We do this to minimize what is known as Materialization. An IEnumerable is any object that can be enumerated.
In this code query:
query = query
.Skip(request.StartIndex)
.Take(request.Count);
Doesn't actually get enumerated (materialized) until the Virtualize does it's internal foreach loop. Put in some break points to check.
We do have to do one materialization: getting the record count to pass back in the ItemsProviderResult instance.
You can also use paging controls. See QuickGrid - https://aspnet.github.io/quickgridsamples/
Note: You should always use paging in any List request. Never get an unconstrained dataset from a data store. Could your code handle 2 million records gracefully? If you want to get the lot, such as for a select and know there should be say 30 records, set a realistic maximum page size, say 1000 records.