Reduce CDORemapper() warnings when multiple cores are used
Following up #157 (closed), the CDORemapper() repetitive warnings are reduced, but the function .withWarnings() only works well when one core is used (num_procs = 1
). But when multiple cores are used (see line3854), there is no warning being returned by parallel::clusterApplyLB
. The warnings show later at bigmemory::as.matrix(data_array)
, e.g., line3941 I've tried to use .withWarnings() to catch the warnings but it doesn't work. The following code can show the repeated warnings.
# Load data
library(startR)
obs_path <- '/esarchive/recon/ecmwf/era5/monthly_mean/$var$_f1h/$var$_$sdate$.nc'
var_name <- 'sfcWind'
lons.min <- 10
lons.max <- 20
lats.min <- 0
lats.max <- 10
obs <- Start(dat = obs_path,
var = var_name,
sdate = '201811',
time = 'all',
latitude = values(list(lats.min, lats.max)),
latitude_reorder = Sort(decreasing = T),
longitude = values(list(lons.min, lons.max)),
longitude_reorder = CircularSort(0, 360),
synonims = list(longitude = c('lon', 'longitude'),
latitude = c('lat', 'latitude')),
transform = CDORemapper,
transform_extra_cells = 2,
transform_params = list(grid = 'r360x181', method = 'conservative', crop = T),
transform_vars = c('latitude', 'longitude'),
return_vars = list(time = NULL, latitude = 'dat', longitude = 'dat'),
num_procs = 2,
retrieve = T)
Before the final warning messages from Start(), two warnings are generated during the data loading process.
starting worker pid=21846 on localhost:11793 at 17:06:13.703
starting worker pid=21845 on localhost:11793 at 17:06:13.704
Warning messages:
1: In (function (data_array, variables, file_selectors = NULL, crop_domain = NULL, :
Argument 'crop' in 'transform_params' for CDORemapper() is deprecated. It is automatically assigned as the selected domain in Start() call.
2: ! Warning: CDORemap: Using CDO version 1.9.8.
* Successfully retrieved data.
Warning messages:
...
A side note: When using the whole workflow, the warning messages are generated by each chunk, so if there are 10 warnings (not only the one from CDORemapper) and we chunk data into 4, in the end of Compute(), the warnings are 40. It is not a bug since the 4 chunks are running independently, so it's normal that they all generate warnings. But if there is a way to improve the user experience, it would be good.